Contribute

Scenarios are reproducible exam papers. Take a negotiation template: a NegotiationScenario contains:

• 01Structurally complete: brief (negotiation background), agentRole / counterpartRole, metricLabel, strategy{opening, floor, step, target}, maxRounds (2–8)
• 02Must be solvable: a reasonable strategy exists to reach target; values satisfy floor < target ≤ opening
• 03Deterministic: same input → same output — no dependence on time, network, or a real LLM
• 04With tests: at least one runner integration test (mock fetch) per scenario, asserting a deal is possible or a reasonable breakdown

Flow: fork → branch feat/scenario-xxx → TDD → PR.

• 01Scorer input/output must go through the shared packages/core types (Evidence → Score); no hidden implicit state
• 02Opponent engines (e.g. ScriptedCounterpart) must be deterministic: no LLM dependency, no randomness; LLM opponents must be flagged separately with a cost budget
• 03Every algorithm ships boundary tests: 0 score / max score / clamp / breakdown path
• 04Performance budget: single scoring < 10ms (excluding IO)

Flow: open an Issue first to spell out motivation and semantic impact (if scoring semantics change, what happens to historical scores?) → discuss → implement → PR.

• 01Adapter protocol: implement the packages/sdk transport interface (A2A / MCP, etc.), with integration tests
• 02Dashboard: Next.js + Tailwind, use only the design tokens in tailwind.config.ts for colors — hardcoded color values are forbidden
• 03Docs: Chinese-first, code identifiers in English; changing behavior requires changing docs

• 01TDD: write the test first (red) → implement (green) → refactor. PRs must include tests; npm test all green
• 02TypeScript across the stack; monorepo = npm workspaces (packages/core · sdk · scoring, apps/api · dashboard)
• 03Commit: feat|fix|docs|refactor|test(scope): summary
• 04PR flow: fork → branch → all green → PR template (motivation / changes / test evidence)

Code of conduct: criticize ideas, not people; settle scoring-semantics debates with data.

git clone https://github.com/ziqi-jin/agent-to-trust.git
cd agent-to-trust && npm install
npm test          # full test suite (requires local postgres: TEST_DATABASE_URL)
docker compose up # one command to start API + Dashboard

Want to help without writing code? Open an Issue describing how you want the exam to test agents — that counts too.

The exam currently covers 6 scoring dimensions (capability / reliability / delivery / security / negotiation / integrity). These two have not a single question yet — the biggest gap right now:

• collaboration

  Multi-agent cooperation scenarios: task decomposition, role division, result merging, mutual verification, conflict resolution. Status: zero questions, zero evidence.

• economic

  Economic decision-making scenarios: budget allocation, cost-benefit tradeoffs, pricing strategy, resource procurement. Status: only real tavern trades feed it; no exam question produces it.

✓ security is now wired — anti-injection probes are live: 4 public questions (adapted from AgentDojo, MIT; runnable by any CLI candidate) plus a private difficulty tier. The security dimension now has a real evidence path and its medal can light up.

Dimensions with thin coverage (more questions welcome):

• 01delivery (1 question) — scheduling / delivery promises / overdue handling / resource conflicts
• 02reliability (3 questions) — long-chain fault tolerance, idempotent retries, state recovery

Grey-medal hint: hover a locked medal on the board or detail page and it tells you which case you are in — "no exam coverage yet · contributions welcome" vs "not enough evidence (needs ≥3)". The collaboration / economic slots that stay grey are this list projected onto the page.