Test and improve the Agent

Validate Agent work on representative cases, review real user feedback and improve the layer that caused the problem.

An Agent is ready for real work when it behaves dependably on representative cases, uses the right systems and evidence, and surfaces uncertainty at the right time.

Do not judge readiness from one good conversation.

Start with representative work

Choose cases that cover:

• the normal path;

• the important exception boundaries;

• missing or conflicting information;

• the actions that carry the most consequence;

• the outputs operators actually need.

Define what “correct” means before you run the case.

That can include a system change, a generated file, a recommendation, a deliberate decision to ask a person, or a verified no-action result.

Inspect how the result was produced

Ask more than “did the answer sound right?”

Check:

• Did the intended Skill run?

• Did the Agent query the right source?

• Did it use a weak source when a stronger one was available?

• Did it create the expected file or system change?

• Did it stop at the right human boundary?

• Did the business post-condition actually happen?

Use the session's visible activity and outputs, then use Monitoring when you need deeper execution evidence.

Test reusable Skills explicitly

When the behaviour is encoded in a Skill, eligible Project builders can use the Skill's Tests surface.

Build a small regression set from representative work. When a change fixes one case, rerun the previously-good cases that could have been affected.

Do not treat an inconclusive or errored test run as proof that the Skill itself is wrong. Establish what actually happened, then correct the relevant layer.

Learn from real use

Project members can leave persistent Agent-session feedback. Builders can review that feedback alongside the session in Activity.

Use feedback together with:

• the transcript;

• tool and source activity;

• produced files;

• the Skill/version evidence available for the run;

• the actual business outcome.

Feedback and technical status are different signals. A technically successful run can still deserve negative outcome feedback.

When feedback exposes a recurring failure mode, reproduce it as a representative test before publishing the fix where practical.

Fix the layer that owns the problem

A failure can come from different parts of the setup:

• wrong general behaviour → Project description;

• wrong recurring method → Skill;

• missing or stale structured context → Table or source data;

• weak maintained reference → Knowledge;

• wrong persistent context → Memory;

• unavailable or over-broad system authority → Connection;

• stable sequence needs tighter control → Workflow;

• genuinely human decision → preserve the human boundary.

Avoid fixing every failure by adding another general instruction.

Publish improvement deliberately

Corrections and feedback are evidence for a change, not permission for silent self-modification.

A dependable loop is:

observe → reproduce → change the owning layer → test → review → publish → watch real use

That is how the Agent gets better without turning production behaviour into an undocumented collection of one-off fixes.