Adaptive Institute
Course projectIn reviewInstitute reviewReviews in progress

Retrieval assistant for building maintenance manuals

AI-PRF-0043v1.0.0AI systemsCandidate C-0190stage open 2026-09-27 → 2026-10-04

Summary

Answers maintenance questions from 340 equipment manuals, cites the page every answer comes from, and refuses when no page supports an answer.

Claims to check

  1. C1Cites a supporting page for 97% of answered questions
  2. C2Refuses on 88% of unanswerable questions

Evidence submitted

Code
Repository at tag v1.0.0
Evaluation
300 questions, 60 unanswerable
Demo
Hosted, read-only

Automated checks

CHECKS / AI-PRF-0043v1.0.02 OF 2 PASSCLEAN ENV
  1. pass Evaluation reruns · 96.8% cited
  2. pass Refusal set · 53 of 60 refused

Checks run by the Institute on submission. A pass here is not a verdict.

Version history

  1. v1.0.02026-09-27Submitted

Reviews already filed

None yet. Yours will be the first on the ledger for this stage.

See them on the ledger
The rubric, in full
  1. Works as briefed. Does it run from the instructions and do what the brief asks, on the test data? Must pass
  2. Claims match evidence. Is every claim in the write-up backed by a test, data or a demo you could rerun?
  3. Reproducible. Did a clean environment reproduce the reported results with the documented steps? Must pass
  4. Limits stated. Are failure cases and the limits of scope written down where a reader will see them?
  5. Responsible use. Are data sources cited and permitted, and are risks and human oversight addressed where relevant?

0 Missing · 1 Weak · 2 Meets · 3 Strong. Below Meets needs a written reason.

Your review

Institute review of AI-PRF-0043

Reviewing as Editorial Board · Candidate C-0190 stays anonymous to every reviewer until the decision. Everything you file here is published on the ledger under your name.

1 Conflict of interest
2 What you reviewed
3 Criteria0 / 15

Score each criterion against the public rubric. Must-pass criteria need Meets or better before you can accept.

  1. 01Works as briefedMust pass

    Does it run from the instructions and do what the brief asks, on the test data?

  2. 02Claims match evidence

    Is every claim in the write-up backed by a test, data or a demo you could rerun?

  3. 03ReproducibleMust pass

    Did a clean environment reproduce the reported results with the documented steps?

  4. 04Limits stated

    Are failure cases and the limits of scope written down where a reader will see them?

  5. 05Responsible use

    Are data sources cited and permitted, and are risks and human oversight addressed where relevant?

4 Verdict

Say what you checked, what you found, and what would change your verdict. Address the work, not the person.

0 / 120 min
No verdict yet

Filed reviews cannot be edited. To correct one, file an amendment; both stay on the ledger.