#4839 Verifying Completeness and Correctness of LLM Response

#4839 After prompting an LLM to evaluate whether a NASA launch provider software development document complies with NASA standards and requirements, how can I verify the response is complete and correct?

Verifying an LLM's compliance evaluation for mission-critical aerospace documentation requires a systematic audit across both the authoritative standards and the target document.

1. Verification Checklist

Dimension	Primary Failure Mode	How to Verify
Standard Baseline	Hallucinated clauses, outdated revisions	Cross-reference cited paragraph numbers against the contractually binding baseline (e.g., NPR 7150.2D vs. earlier revisions).
Exemption & Tailoring	Flagging waived clauses as violations	Check the program’s approved Compliance Matrix / Tailoring Plan for explicitly waived or modified requirements.
Traceability & Grounding	Plausible but fabricated document quotes	Search the provider's document for exact quote strings and cited section numbers.
Completeness (Recall)	Hallucination of omission (skipping hard requirements)	Reconcile the LLM’s evaluated items against the complete set of mandatory "shall" statements for that software classification.
Logic & Classification	Misjudged compliance status	Manually inspect borderline verdicts (e.g., marked "Compliant" based only on intent rather than objective evidence).

2. Systematic Audit Protocol

Audit the Governing Standard Baseline
- Confirm the specific document identifier and revision: NPR 7150.2 (NASA Software Engineering Requirements), NASA-STD-8739.8 (Software Assurance and Safety), and NASA-STD-8719.13 (Software Safety Standard).
- Verify software classification (Class A human-rated vs. Class B/C non-crewed). An LLM often cites requirements that only apply to Class A systems when assessing Class B software, or vice versa.
Verify Bidirectional Citation Grounding
- Source Document Grounding: Ensure every claim of non-compliance quotes exact, verbatim passages with verifiable section/page numbers from the provider's document.
- Standard Grounding: Check that every NASA standard cited exists verbatim in the official standard text, particularly verifying that numerical clause IDs match the applicable revision.
Check Tailoring and Approved Variances
- NASA programs routinely grant deviations, waivers, and tailoring allowances (documented via the NASA Software Compliance Matrix).
- Compare the LLM's finding list against the approved compliance matrix. LLMs frequently flag missing processes that were formally tailored out or satisfied by alternative commercial industry standards (such as DO-178C or CMMI Level 3+).
Run an Automated "Second-Pass" Differential Verification
- Extraction Pass: Use an independent, deterministic script or an isolated LLM prompt strictly tasked with extracting all mandatory requirements from the governing standard into a structured CSV:
  [Req ID] | [Mandatory "Shall" Text] | [Classification Applicability]
- Verification Matrix: Prompt a separate instance with access to the original evaluation to verify only:
  Does citation [X] exist in [Document]? (Yes/No + Verbatim Quote) Does standard clause [Y] match the exact wording of [NPR 7150.2]? (Yes/No)
Examine Evidence Sufficiency vs. Process Intent
- Distinguish between process commitment ("The vendor will perform static code analysis") and objective quality evidence ("Static code analysis reports, rule sets, and threshold criteria are defined in Section 4.2").
- LLMs often mark requirements compliant if the text merely mentions the topic, ignoring whether actionable verification criteria exist.

3. Subject-Matter Expert (SME) Sampling Strategy

Do not review the LLM output in arbitrary order. Stratify human review:

100% Review of Safety-Critical Findings: Any finding intersecting software safety hazard analyses, fault tolerance, or command/control inhibits.
Random 20% Spot-Check of "Compliant" Items: LLM false positives typically hide here (marking ambiguous provider language as fully compliant).
100% Review of Identified Non-Compliances: Validate before escalating findings to the contractor or joint control board.

All steps, initial and verification, need to be AI-automated

An autonomous, dual-engine AI verification architecture replaces manual spot-checking by decoupling requirement extraction, compliance evaluation, adversarial verification, and deterministic schema gating into independent execution nodes.

Architecture Overview

[Governing Baseline DB]  ──┐
(NPR 7150.2D, NASA-STD)    │
                           ▼
                  [1. Auditor Node]
                  (Candidate Audit)
                           │
                           ▼
[Provider Target Doc] ───► [2. Adversarial Verifier] ◄─── [Compliance Matrix / Waivers]
                           (Independent Model)
                           │
                           ▼
                  [3. Reconciliation Gating]
                  (Pydantic / Deterministic AST)
                           │
                     ┌─────┴─────┐
                     ▼           ▼
              [Verified Pass] [Flagged Discrepancy]

1. Verification Dimensions & Automated Tests

Verification Dimension	Failure Mode Caught	Automated Verification Check
Citation Hallucination	Fabricated clauses or vendor quotes	Exact substring matching against raw document text & authoritative standards database.
Omission / Recall Gap	Skipped mandatory "shall" statements	Set difference analysis: `{Applicable Class Requirements} \ {Audited Requirements}`.
Baseline Mismatch	Using wrong standard revision	Primary key check against official requirement baseline tables (e.g., NPR 7150.2D clause IDs).
Tailoring False Positive	Flagging formally approved waivers	Automated lookup against approved program Tailoring Matrix / DIDs.
Weak Evidence Acceptance	Accepting intent instead of objective proof	Adversarial prompt checking for concrete artifacts (DIDs, test records, tool IDs, section tables).

2. Multi-Agent Automated Pipeline Execution

1. Deterministic Requirement Scope Resolution: Pre-audit stage: Eliminates baseline hallucinations.

Query a relational or graph database of NASA standards (e.g., NPR 7150.2D, NASA-STD-8739.8B) filtered strictly by contractual scope (e.g., Class A / Crewed or Class B / Uncrewed).

Output a fixed JSON array of required Requirement IDs (SWE-XXX, SMA-XXX) and verbatim "shall" text.
Load the vendor's approved Software Compliance Matrix to tag pre-waived or tailored-out clauses as EXEMPT.

2. Primary Compliance Audit Node: Evaluates document against unresolved mandatory requirements.

Feed the vendor documentation chunks alongside the resolved requirement scope into the Primary Auditor LLM.

Enforce a strict Pydantic JSON schema requiring for each evaluated clause:
- requirement_id: Exact standard ID.
- verdict: Compliant, Non-Compliant, or Partial.
- vendor_citations: Array of exact verbatim quote strings, section numbers, and page numbers.
- objective_evidence_description: Identified concrete artifacts (plans, metrics, toolchains).

3. Deterministic Citation Grounding Verification:AST / Substring validation without LLM inference.

Execute a local Python script to validate all returned citations before calling the second LLM:

def verify_grounding(audit_findings, raw_vendor_text, standards_db):
    for item in audit_findings:
        # 1. Verify standard clause exists
        assert item.requirement_id in standards_db, f"Unknown ID: {item.requirement_id}"
        
        # 2. String-match citations in provider text
        for quote in item.vendor_citations:
            norm_quote = " ".join(quote.split())
            assert norm_quote in raw_vendor_text, f"Hallucinated quote: {quote}"

Any finding failing string grounding is automatically marked REJECTED_HALLUCINATED_CITATION and routed back for re-extraction.

4. Adversarial Secondary Verifier Node:Independent LLM cross-examines evidence sufficiency.

Pass the verified citations and primary verdict to an isolated, distinct model (e.g., cross-checking between different model families) acting strictly as an Adversarial Reviewer:

Task: Evaluate whether the cited vendor excerpt constitutes objective verification evidence or merely aspirational intent (e.g., "Contractor will track defects" vs. "Defect tracking metrics and severity thresholds defined in Section 3.4").
Output an adversarial score (Concur, Diverge - Insufficient Evidence, Diverge - Overly Strict).

5. Coverage & Completeness Reconciler:Guarantees zero dropped requirements.

Execute a deterministic coverage diff:

\Delta_{\text{omitted}} = S_{\text{mandatory}} \setminus S_{\text{evaluated}}

If $\Delta_{\text{omitted}}$ is non-empty, auto-trigger a targeted audit pass specifically on the missing requirement IDs until the delta is zero.

3. Execution Schema & Concrete Next Steps

Serialize Governing Requirements: Ensure your NPR 7150.2D and NASA-STD-8739.8B records are structured with primary keys (SWE-013, SWE-020) and classification tags in an indexed data store.
Implement Dual-Model Orchestration: Run the Auditor and Adversarial Verifier on distinct model backends to eliminate shared blind spots.
Hard-Gate via Pydantic: Constrain both audit and verification responses to structured output schemas; reject and retry raw Markdown or unstructured prose.
Log Automated Discrepancy Reports: Aggregate items where the Auditor and Adversarial Verifier disagree into a dedicated discrepancy table for formal engineering baseline sign-off.

From <https://gemini.google.com/app/f3494350cda117d4> Google Gemini (3.8 Flash)

Search This Blog

Learning from ChatGPT