S

Sebastien

Back to Challenges
Innovate

Measure Your Agent with Live Evaluations

Not started
20 points
Intermediate
~45 min
Innovate
Auto-verified from Console activity

Attach an evaluation to an agent you manage so real traffic is scored continuously, then read the resulting pass rate to see whether the agent behaves the way you intended in production.

Why this challenge?

Shipping an agent is not the same as knowing it works. A live evaluation turns every real interaction into evidence, so regressions show up as a falling pass rate instead of as a surprised user.

How do you want to work?

Choose your preferred workflow. All paths reach the same outcome — pick what fits your style.

The Console is the whole path for this challenge: live evaluations are configured on the agent's Quality tab, and AI Quest detects the configured evaluation automatically once it exists.

Console journey

Move through this journey one milestone at a time. Each step opens the place you need next.

  1. 1
    Open Quality

    Open an agent you manage and select Quality, where evaluation configuration lives.

    Choose an agent
  2. 2
    Add a live evaluation

    Create the evaluation, choose its run credential, and define the assertions that should hold for real requests.

    Read Help
  3. 3
    Try the agent

    Run a few realistic requests so the evaluation starts scoring live interactions.

    Agent name

    Copy the qualified name shown at the top of your agent.

    Test agent
  4. 4
    Check the scores

    Return to Quality and review the scored and passed counts to see how the agent is actually performing.

    Choose an agent
Ask the Deere AI Assistant

Stuck? The Deere AI Assistant in the Console doesn't know about this quest by default. Download the AI Quest Challenges guide below and upload it into the Assistant so it has context, then ask for help configuring your evaluation.

Download the AI Quest Challenges guide
Next up

Once an evaluation is scoring your agent, the golden-evaluation challenge is the natural next step: the same assertions, run against a fixed dataset you control.

Choose an agent View Documentation

This challenge is verified automatically from your real Console activity.