---
title: "12-week portfolio audit: 8 provenance gaps | Report"
description: "Inspect 97 catalog rows, 30 synthetic cases and pinned file digests. External use is unverified and maintenance time unmeasured at the November 5 cutoff."
---

<p class="eyebrow">W12 / technical audit dossier / report v1.0</p>

# Pharos Production Audits Its Portfolio: 12-Week Evidence Report

<p class="lead">The portfolio is reproducible within its published synthetic boundaries. It has not demonstrated the program's external-use target, complete canonical provenance or a measured maintenance budget.</p>

<p class="verdict"><strong>Overall: FAIL against the program gates.</strong> Existing artifacts remain public candidates. This report grants no operational release approval.</p>

<dl class="facts"><dt>Report cutoff</dt><dd><time datetime="2026-11-05">November 5, 2026</time>, Europe/Kyiv</dd><dt>Program window</dt><dd><time datetime="2026-08-17">August 17</time> to <time datetime="2026-11-08">November 8, 2026</time>. Cutoff falls in week 12</dd><dt>Public operating history</dt><dd>49 elapsed days since September 17, rather than 12 public operating weeks</dd><dt>Scope</dt><dd>2 datasets, 2 Spaces and their shared Collection. Evidence Pack selected for the hands-on self-audit</dd></dl>

## The gate results

<p>A published repository is an inspectable deliverable. The program also required evidence of use, complete source lineage and affordable maintenance. These are separate claims, so a successful fixture run cannot compensate for an unmet use target.</p>

<table><caption>Program criteria at the November 5 cutoff</caption><thead><tr><th scope="col">Criterion</th><th scope="col">Observation</th><th scope="col">Verdict</th></tr></thead><tbody>
<tr><th scope="row">Catalog provenance</th><td>89 of 97 catalog rows have nonempty source URL, source commit, source hash, review status and limitations. The 8 exceptions are disclosed architecture proposals.</td><td>FAIL against the all-row target</td></tr>
<tr><th scope="row">External evidence use</th><td>0 qualifying signals found in the inspected public discussions. Target at W12: at least 1.</td><td>NOT DEMONSTRATED</td></tr>
<tr><th scope="row">Maintenance</th><td>No complete weekly time ledger was available to establish an average at or below 6 hours per week.</td><td>UNMEASURED</td></tr>
<tr><th scope="row">Snapshot preservation</th><td>Both original catalog digests and the bundled Space data match their release records.</td><td>PASS for checked bytes</td></tr>
<tr><th scope="row">Synthetic reproduction</th><td>The published runners for all 3 scenarios complete successfully, covering 30 authored cases.</td><td>PASS within synthetic scope</td></tr>
<tr><th scope="row">Link availability</th><td>22 of 22 selected source, repository, Collection and scenario endpoints return HTTP 200.</td><td>PASS for availability only</td></tr>
</tbody></table>

<p>For the release-evidence problem, the <a href="https://pharosproduction.github.io/agent-security-control-mapper/methodology.html">Pharos Production control-mapping methodology</a> supplies an inspectable method. Its presence does not establish that an adopter has implemented a control or that an evidence reference is authentic.</p>

## What the portfolio actually contains

<p>The <a href="https://huggingface.co/collections/pharosproduction/production-ai-evidence-collection">Production AI Evidence Collection</a> connects four artifacts with different jobs. This audit keeps catalog records separate from synthetic cases: adding them together would blur methods, proposals and executed fixtures.</p>

<table><caption>Public inventory and distinct decision jobs</caption><thead><tr><th scope="col">Artifact</th><th scope="col">Contents or function</th><th scope="col">Boundary</th></tr></thead><tbody>
<tr><th scope="row"><a href="https://huggingface.co/datasets/pharosproduction/production-ai-control-evidence">Control Evidence Dataset</a></th><td>55 draft controls: 24 security, 8 AIBOM proposals, 11 drift and 12 governance. An additive scenario bundle contributes 30 synthetic rows.</td><td>Control lineage and original proposals remain distinguishable.</td></tr>
<tr><th scope="row"><a href="https://huggingface.co/spaces/pharosproduction/production-ai-evidence-pack">Evidence Pack</a></th><td>Combines selected controls with supplied references, owners, release versions and recovery fields.</td><td>Preparation inventory. Supplied evidence is not independently checked.</td></tr>
<tr><th scope="row"><a href="https://huggingface.co/datasets/pharosproduction/production-ai-decision-rules">Decision Rules Dataset</a></th><td>42 draft records: 17 RAG, 12 build-or-buy, 5 framework profiles and 8 architecture proposals.</td><td>Architecture proposals have no canonical source. Framework evidence is dated August 28, 2026.</td></tr>
<tr><th scope="row"><a href="https://huggingface.co/spaces/pharosproduction/production-ai-decision-lab">Decision Lab</a></th><td>Applies the decision snapshot to explicit criteria and exports the resulting trace.</td><td>A deterministic exploration aid, without a universal recommendation or production approval.</td></tr>
</tbody></table>

<p>All 97 catalog rows retain <code>NOT_REVIEWED</code>. The September 17 owner review authorized public candidate publication. It did not convert each frozen row into an independently reviewed operational control. The original catalogs remain at their recorded candidate schema, with upstream proposal adoption still disclosed as incomplete.</p>

<p>Scenario bundle v1.1 is an additive manifest version. It does not upgrade the original catalogs, remove their warnings or represent a native Hub tag. This audit adds a separate report version and preserves that distinction.</p>

## Evidence Pack, checked against its own limits

<p>The hands-on check used the hosted tool's built-in fictional support-agent example. It selected 4 controls and supplied an accepted reference for identity. AIBOM scope, input drift and governance inventory had no evidence. Recovery fields were blank.</p>

<p>The result was <code>EVIDENCE_GAPS_OPEN</code>: 3 controls with gaps and 1 with supplied references. It also listed missing recovery fields and the frozen source-review, rights and canonical-adoption conditions. The raw result retained <code>release_eligible=false</code> and the exact original dataset revision.</p>

<p>We then changed only the identity reference from <code>example://identity-exercise</code> to <code>audit://not-validated-document</code>. Editing cleared the old result. Rechecking still classified identity as <code>REFERENCES_SUPPLIED_USER_ATTESTED</code>, with the same aggregate counts. No reference contents were fetched. This confirms the documented self-attestation boundary, rather than a bypass of an authenticated evidence check.</p>

<p>The two exact raw JSON results displayed by the hosted page are included in the supporting evidence file. A browser download capture timed out before yielding a file, so this audit does not claim fresh downloaded-file parity. Earlier scenario export checks remain historical evidence, not a substitute for a fresh check.</p>

## What we used and what it proves

<p>Immutable Hub revisions supplied the comparison boundary. Release manifests supplied the original catalog digests. Embedded source snapshots allowed their recorded file hashes to be recomputed without substituting a later upstream revision. All inspected embedded source files matched their recorded digests.</p>

<p>Scenario 001's public runner matched its 10 authority cases. Scenario 002 completed its 16-test suite, and Scenario 003 completed its 17-test suite. These suites include their respective 10-case datasets. Their scope is deterministic authored fixtures: no model inference, retrieval evaluation, customer incident, deployed authorization boundary or recovery exercise was executed for this report.</p>

<p>For availability, the link scan checked 7 source Pages, 7 distinct source-data URLs, 4 repositories, the Collection and 3 scenario pages. HTTP 200 establishes reachability at capture time. It does not establish that an old framework profile still describes current features, or that a source rule is adequate for a particular system.</p>

## Where the program fell short

<p>Our all-row source requirement was stricter than the released candidate. The 8 architecture proposals openly lack a canonical source. The 8 AIBOM proposals do have a source-context link, but the underlying comparison does not itself contain those proposed controls. A provenance field can be present while its mapping still needs adoption and review.</p>

<p>Lifecycle wording also has two layers. The public release record says publication was owner-authorized, while the frozen license ledger and application output retain pending-review language. Those older flags belong to the preserved candidate contract. The report records this consistency gap instead of rewriting historical data or implying that every pending condition has been cleared.</p>

<p>The usefulness target remains unestablished. Across the 4 repository discussion indexes, the only 2 threads were automated Parquet-conversion notices. They are infrastructure activity. We found no substantive external issue or documented reuse there. Private inquiries and off-platform reuse were not inspected. Zero verified signals in this scope does not prove that nobody used the work.</p>

<p>Maintenance cost has no defensible average without a time ledger. A short test runtime is not a maintenance estimate. The planned 6-hour weekly threshold therefore receives no pass.</p>

## What stays, and what stops

<p>Keep the existing artifacts available. The datasets distribute pinned records, the Spaces apply those records, and the Collection provides navigation. No artifact was shown to lack a distinct job as well as demonstrated use, so this audit does not delete, archive or unpin one to manufacture a pruning result.</p>

<p>Pause new Failure Scenarios in this portfolio. The demand rule required a reassessment if no evidence-use signal was established 45 days after the second Space appeared. Both Spaces were public on September 17. That checkpoint fell on November 1. Next, review distribution and the decision jobs before proposing another scenario.</p>

- Stop using release count or synthetic case count as a proxy for external usefulness.
- Keep production-readiness, validated-control and operating-effectiveness claims withheld.
- Require a separately reviewed version for source adoption or application changes. Preserve historical records.
- Record maintenance work by week before evaluating the budget criterion. A maintenance role is suggested, not an accepted assignment.

<p>The later target of 3 evidence-use signals at 90 days from first public release falls on December 16, 2026. That is a criterion for a future review, not a forecast or a scheduled automation.</p>

## Inspect the evidence

<p>Download the <a href="portfolio-audit-2026-11-05.json">machine-readable audit verdict</a> and <a href="portfolio-audit-evidence-2026-11-05.json">supporting measurement evidence</a>. The latter contains repository revisions, file digests, per-row provenance checks, the bounded URL scan and the two hosted self-audit results. A <a href="portfolio-audit-2026-11-05.md">Markdown copy of this report</a> is also available.</p>

1. Fetch the files at the exact revisions listed below, rather than substituting current main.
2. Recompute SHA-256 for each catalog's <code>data.json</code> and its configured JSONL files against <code>release.json</code>.
3. Inspect missing provenance fields separately from review status. An original proposal is not a measured source result.
4. Run the published scenario reproduction scripts, then inspect the expected outcomes and their synthetic limitations.
5. Repeat the hosted example and edit one evidence reference. Read the resulting raw JSON and its release eligibility before drawing a conclusion.

<table><caption>Immutable repository revisions inspected before this report</caption><thead><tr><th scope="col">Repository</th><th scope="col">Exact revision</th></tr></thead><tbody><tr><th scope="row">production-ai-control-evidence</th><td><a href="https://huggingface.co/datasets/pharosproduction/production-ai-control-evidence/tree/e0f91124cff4b5cb2316986ae4526bddac12f5cc"><code>e0f91124cff4b5cb2316986ae4526bddac12f5cc</code></a></td></tr><tr><th scope="row">production-ai-evidence-pack</th><td><a href="https://huggingface.co/spaces/pharosproduction/production-ai-evidence-pack/tree/d7a9960e4117a1b6d53e8d2828461241d01a7718"><code>d7a9960e4117a1b6d53e8d2828461241d01a7718</code></a></td></tr><tr><th scope="row">production-ai-decision-rules</th><td><a href="https://huggingface.co/datasets/pharosproduction/production-ai-decision-rules/tree/f6aa01cfd98e015bd629090d97a2969d52705240"><code>f6aa01cfd98e015bd629090d97a2969d52705240</code></a></td></tr><tr><th scope="row">production-ai-decision-lab</th><td><a href="https://huggingface.co/spaces/pharosproduction/production-ai-decision-lab/tree/fef140eb62bab974bcae9ce0af54823add62a988"><code>fef140eb62bab974bcae9ce0af54823add62a988</code></a></td></tr></tbody></table>

<p>Correct a source fact upstream. Report a reproducible discrepancy in the relevant repository's Community tab, with its revision and case or row ID. Do not include private production data. This dossier is a self-audit of public artifacts, without new independent human technical sign-off.</p>

<p>Report v1.0 · HF-PP-2026-W12 · Maintained in the existing evidence portfolio. Existing attribution and license notices remain applicable. This report adds no blanket license over source data.</p>