Pharos Production ยท Technical reference

Citation support and quantity checks

Compare a claim with its passage, then inspect quantity matching and its semantic limits.

Use these manual checks before treating a score as evidence. This page does not execute the scorer or validate an answer. The linked articles describe historical implementation records.

A resolved marker still needs a supported claim

First locate the answer span that makes the claim. Follow its marker to the exact cited passage, then compare the claim with that passage. A marker that resolves to a retrieved fragment can still accompany an unsupported statement.

Original fictional fixture: a fan inspection interval
RecordTextManual observation
Passage AInspect the filter every 12 days. This note does not specify a replacement interval.Inspection interval is stated.
AnswerReplace the filter every 12 days [A].The marker resolves. The replacement claim exceeds the passage.
  1. Save the exact answer span and source revision.
  2. Record marker resolution separately from claim support.
  3. Mark the replacement claim unsupported in this fictional pair. Keep a reviewer decision separate from the parser result.
  4. Repeat with a supported answer: "Inspect the filter every 12 days [A]." A useful check must distinguish this pair.

Read Valid Citations, Unsupported Claims: What Fifteen Questions Revealed About Our RAG for the historical case and its sampling limits. Its selected questions do not establish a general model error rate.

Open the existing stale-evidence fixture to inspect source freshness as a separate dimension. A current source can still fail to support a claim.

Keep the quantity check beside its blind spot

For each answer quantity, retain its value and unit together. Compare against quantities in the cited passages. Preserve unit-prefix case where it changes scale. Pair membership alone does not bind a quantity to the correct subject.

Original fictional electronics records for manual review
PassageAnswerExpected manual finding
LED A consumes 8 mW.LED A consumes 8 MW.Changed unit prefix. Unsupported pair.
LED A consumes 8 mW. LED B consumes 20 mW.LED A consumes 20 mW.Pair occurs in source. Subject assignment is wrong.
LED A consumes 8 mW.LED A consumes eight milliwatts.Meaning is equivalent. Parser handling must be measured separately.
{"record_id":"manual-quantity-example","fixture_origin":"synthetic",
 "parser_observation":"NOT_RUN","review_status":"NOT_REVIEWED",
 "model_run":false,"release_eligible":false}

Read What Our Number-and-Unit Scorer Could Actually Detect for the implemented scorer's normalization and metric boundaries. Its parser fixture counts are not live model accuracy.

When recording a rate, name the eligible population, numerator and excluded errors. Keep an empty applicable population distinct from a measured zero.

What to retain after review

Copy the answer, cited passage and immutable revisions into a review record. Keep syntax resolution, quantity matching and human claim adjudication as separate fields. A complete record does not establish production effectiveness.

Return to the evidence inventory to document evidence references and gaps. The original inventory remains a draft preparation tool.