Pharos Production Audits Its Portfolio: 12-Week Evidence Report

The gate results

A published repository is an inspectable deliverable. The program also required evidence of use, complete source lineage and affordable maintenance. These are separate claims, so a successful fixture run cannot compensate for an unmet use target.

For the release-evidence problem, the Pharos Production control-mapping methodology supplies an inspectable method. Its presence does not establish that an adopter has implemented a control or that an evidence reference is authentic.

What the portfolio actually contains

The Production AI Evidence Collection connects four artifacts with different jobs. This audit keeps catalog records separate from synthetic cases: adding them together would blur methods, proposals and executed fixtures.

All 97 catalog rows retain NOT_REVIEWED. The September 17 owner review authorized public candidate publication. It did not convert each frozen row into an independently reviewed operational control. The original catalogs remain at their recorded candidate schema, with upstream proposal adoption still disclosed as incomplete.

Scenario bundle v1.1 is an additive manifest version. It does not upgrade the original catalogs, remove their warnings or represent a native Hub tag. This audit adds a separate report version and preserves that distinction.

Evidence Pack, checked against its own limits

The hands-on check used the hosted tool's built-in fictional support-agent example. It selected 4 controls and supplied an accepted reference for identity. AIBOM scope, input drift and governance inventory had no evidence. Recovery fields were blank.

The result was EVIDENCE_GAPS_OPEN: 3 controls with gaps and 1 with supplied references. It also listed missing recovery fields and the frozen source-review, rights and canonical-adoption conditions. The raw result retained release_eligible=false and the exact original dataset revision.

We then changed only the identity reference from example://identity-exercise to audit://not-validated-document. Editing cleared the old result. Rechecking still classified identity as REFERENCES_SUPPLIED_USER_ATTESTED, with the same aggregate counts. No reference contents were fetched. This confirms the documented self-attestation boundary, rather than a bypass of an authenticated evidence check.

The two exact raw JSON results displayed by the hosted page are included in the supporting evidence file. A browser download capture timed out before yielding a file, so this audit does not claim fresh downloaded-file parity. Earlier scenario export checks remain historical evidence, not a substitute for a fresh check.

What we used and what it proves

Immutable Hub revisions supplied the comparison boundary. Release manifests supplied the original catalog digests. Embedded source snapshots allowed their recorded file hashes to be recomputed without substituting a later upstream revision. All inspected embedded source files matched their recorded digests.

Scenario 001's public runner matched its 10 authority cases. Scenario 002 completed its 16-test suite, and Scenario 003 completed its 17-test suite. These suites include their respective 10-case datasets. Their scope is deterministic authored fixtures: no model inference, retrieval evaluation, customer incident, deployed authorization boundary or recovery exercise was executed for this report.

For availability, the link scan checked 7 source Pages, 7 distinct source-data URLs, 4 repositories, the Collection and 3 scenario pages. HTTP 200 establishes reachability at capture time. It does not establish that an old framework profile still describes current features, or that a source rule is adequate for a particular system.

Where the program fell short

Our all-row source requirement was stricter than the released candidate. The 8 architecture proposals openly lack a canonical source. The 8 AIBOM proposals do have a source-context link, but the underlying comparison does not itself contain those proposed controls. A provenance field can be present while its mapping still needs adoption and review.

Lifecycle wording also has two layers. The public release record says publication was owner-authorized, while the frozen license ledger and application output retain pending-review language. Those older flags belong to the preserved candidate contract. The report records this consistency gap instead of rewriting historical data or implying that every pending condition has been cleared.

The usefulness target remains unestablished. Across the 4 repository discussion indexes, the only 2 threads were automated Parquet-conversion notices. They are infrastructure activity. We found no substantive external issue or documented reuse there. Private inquiries and off-platform reuse were not inspected. Zero verified signals in this scope does not prove that nobody used the work.

Maintenance cost has no defensible average without a time ledger. A short test runtime is not a maintenance estimate. The planned 6-hour weekly threshold therefore receives no pass.

What stays, and what stops

Keep the existing artifacts available. The datasets distribute pinned records, the Spaces apply those records, and the Collection provides navigation. No artifact was shown to lack a distinct job as well as demonstrated use, so this audit does not delete, archive or unpin one to manufacture a pruning result.

Pause new Failure Scenarios in this portfolio. The demand rule required a reassessment if no evidence-use signal was established 45 days after the second Space appeared. Both Spaces were public on September 17. That checkpoint fell on November 1. Next, review distribution and the decision jobs before proposing another scenario.

• Stop using release count or synthetic case count as a proxy for external usefulness.
• Keep production-readiness, validated-control and operating-effectiveness claims withheld.
• Require a separately reviewed version for source adoption or application changes. Preserve historical records.
• Record maintenance work by week before evaluating the budget criterion. A maintenance role is suggested, not an accepted assignment.

The later target of 3 evidence-use signals at 90 days from first public release falls on December 16, 2026. That is a criterion for a future review, not a forecast or a scheduled automation.

Inspect the evidence

Download the machine-readable audit verdict and supporting measurement evidence. The latter contains repository revisions, file digests, per-row provenance checks, the bounded URL scan and the two hosted self-audit results. A Markdown copy of this report is also available.

1. Fetch the files at the exact revisions listed below, rather than substituting current main.
2. Recompute SHA-256 for each catalog's data.json and its configured JSONL files against release.json.
3. Inspect missing provenance fields separately from review status. An original proposal is not a measured source result.
4. Run the published scenario reproduction scripts, then inspect the expected outcomes and their synthetic limitations.
5. Repeat the hosted example and edit one evidence reference. Read the resulting raw JSON and its release eligibility before drawing a conclusion.

Correct a source fact upstream. Report a reproducible discrepancy in the relevant repository's Community tab, with its revision and case or row ID. Do not include private production data. This dossier is a self-audit of public artifacts, without new independent human technical sign-off.