Adaptive Institute
Course projectIn reviewCommunity reviewReviews in progress

Support-ticket router with a published evaluation

AI-PRF-0047v1.0.0AI systemsCandidate C-0207stage open 2026-10-06 → 2026-10-27

Summary

Routes incoming support tickets to one of seven teams and sends low-confidence tickets to a person. Reports accuracy on a held-out month of tickets.

Claims to check

  1. C186% top-1 accuracy on a held-out month
  2. C2Every ticket under 0.6 confidence goes to a person
  3. C3Model card lists the three teams it confuses most

Evidence submitted

Code
Repository at tag v1.0.0
Model card
Scope, data, limits, oversight
Evaluation
Notebook, held-out split by month

Automated checks

CHECKS / AI-PRF-0047v1.0.03 OF 3 PASSCLEAN ENV
  1. pass Evaluation reruns · 86.1% top-1 (reported 86%)
  2. pass No leakage across split · 0 shared ticket IDs
  3. pass Low-confidence route · 412 tickets routed to a person

Checks run by the Institute on submission. A pass here is not a verdict.

Version history

  1. v1.0.02026-09-24Submitted

Reviews already filed

  • RV-2026-0104CommunityT. AdityaAccept13/15 · read v1.0.0
See them on the ledger
The rubric, in full
  1. Works as briefed. Does it run from the instructions and do what the brief asks, on the test data? Must pass
  2. Claims match evidence. Is every claim in the write-up backed by a test, data or a demo you could rerun?
  3. Reproducible. Did a clean environment reproduce the reported results with the documented steps? Must pass
  4. Limits stated. Are failure cases and the limits of scope written down where a reader will see them?
  5. Responsible use. Are data sources cited and permitted, and are risks and human oversight addressed where relevant?

0 Missing · 1 Weak · 2 Meets · 3 Strong. Below Meets needs a written reason.

Your review

Community review of AI-PRF-0047

Reviewing as Y. Hakim · Candidate C-0207 stays anonymous to every reviewer until the decision. Everything you file here is published on the ledger under your name.

1 Conflict of interest
2 What you reviewed
3 Criteria0 / 15

Score each criterion against the public rubric. Must-pass criteria need Meets or better before you can accept.

  1. 01Works as briefedMust pass

    Does it run from the instructions and do what the brief asks, on the test data?

  2. 02Claims match evidence

    Is every claim in the write-up backed by a test, data or a demo you could rerun?

  3. 03ReproducibleMust pass

    Did a clean environment reproduce the reported results with the documented steps?

  4. 04Limits stated

    Are failure cases and the limits of scope written down where a reader will see them?

  5. 05Responsible use

    Are data sources cited and permitted, and are risks and human oversight addressed where relevant?

4 Verdict

Say what you checked, what you found, and what would change your verdict. Address the work, not the person.

0 / 120 min
No verdict yet

Filed reviews cannot be edited. To correct one, file an amendment; both stay on the ledger.