S

Sebastien

Back to My Journey
Innovate

Create and Run a Golden Evaluation

Not started
25 points
Advanced
~60 min
Innovate
Self-report

Create a golden dataset with repeatable scenarios and assertions, run it against a production HostedAgent, and use the results to identify an evidence-based improvement.

Why this challenge?

Evaluation turns “it seems to work” into a repeatable quality signal. Golden scenarios give teams a shared way to detect regressions before expanding an agent's scope.

How do you want to work?

Choose your preferred workflow. All paths reach the same outcome — pick what fits your style.

The Console path is the recommended route for this challenge: create and run the golden dataset from the agent Quality tab without opening a repository.

Console journey

Move through this journey one milestone at a time. Each step opens the place you need next.

  1. 1
    Open Quality

    Open the managed HostedAgent and select Quality. Evaluation authoring lives on the agent profile.

    Choose an agent
  2. 2
    Create a golden dataset

    Create a stable lowercase dataset, select its run credential, and add scenarios with inputs and assertions.

    Read Help
  3. 3
    Run the dataset

    Start the golden evaluation from Quality and wait for the scenario results.

  4. 4
    Diagnose a result

    Open a failed scenario or trace, identify one improvement, and record what evidence led to it.

    Read Help
Ask the Deere AI Assistant

Stuck? The Deere AI Assistant in the Console doesn't know about this quest by default. Download the AI Quest Challenges guide below and upload it into the Assistant so it has context, then ask for help building or evaluating this.

Download the AI Quest Challenges guide
Choose an agent View Documentation

This challenge can't be auto-detected yet — mark it complete once you've done the steps above.