#4437 RAG/KG by NPR 7150.2D Section (Chunking)
#4437 RAG/KG by NPR 7150.2D Section (Chunking)
#4437 I want to develop prompt scripts, each prompt to be run individually, in sequence to evaluate a technical document against NASA standards and also the Software Engineering (SWE) Handbook. For a software development plan (SDP), list the sequence of prompts referencing requirements and guidelines from NASA NPR 7150.2D, the SWE Handbook, as well as the "Minimum Recommended Content" lists per document in the SWE Handbook.
To build an automated, sequential evaluation system for a Software Development Plan (SDP), your prompt scripts need to reflect the compliance hierarchy of NASA software engineering.
NASA NPR 7150.2D establishes the mandatory requirements (the "SWEs"), while the NASA Software Engineering Handbook (NASA-HDBK-2203) provides the implementation guidance and the "Minimum Recommended Content" checklist for an SDP (found in the Handbook's Appendices).
Here is a structured sequence of 9 evaluation prompts. They are designed to be executed one after another, feeding the SDP text into the LLM section by section to prevent context window dilution and ensure granular compliance mapping.
The Sequential Prompt Script Pipeline
Prompt 1: Project Management, Classification, and Tailoring Compliance
Intent: Validates that the SDP establishes its organizational structure, software classification (Class A through E), and handles requirement tailoring legally.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) against NASA NPR 7150.2D and NASA-HDBK-2203 guidelines.
Focus Areas & Requirements:
1. Software Classification: Does the document explicitly state the software classification (Class A, B, C, D, or E) per NPR 7150.2D Section 2.1?
2. Tailoring and Compliance Matrix: Is there a reference to a tailored compliance matrix mapping the mandatory SWE requirements? Are deviations and waivers documented with rationale?
3. Management Structures: Identify the defined roles, responsibilities, and clear lines of authority for the development team, project management, and stakeholders.
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Purpose, scope, and objectives of the software project.
- [ ] Organization structure, boundaries, and interfaces.
- [ ] Software classification and engineering environment.
- [ ] Compliance matrix / Tailoring log or reference.
Task:
Analyze the provided text. Provide a "COMPLIANT", "PARTIALLY COMPLIANT", or "NON-COMPLIANT" status for each checklist item. Extract specific quotes as evidence for compliance, and list any gaps or missing elements required by NPR 7150.2D.
Prompt 2: Software Lifecycle and Schedule Planning
Intent: Evaluates the chosen development model, milestone definitions, and work tracking metrics.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Lifecycle and Lifecycle Planning.
Focus Areas & Requirements:
1. Lifecycle Selection: Verify compliance with NPR 7150.2D software lifecycle planning requirements. Does the SDP define a mature lifecycle model (e.g., Agile, Waterfall, Spiral) suited to the mission's risk posture?
2. Schedule and Milestones: Evaluate how software milestones are integrated into the master project schedule. Look for clear definitions of key NASA reviews (SRR, PDR, CDR, TRR).
3. Estimation and Metrics: Check for defined methods for software size, cost, and schedule estimations.
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Software lifecycle model selection and justification.
- [ ] Work Breakdown Structure (WBS) down to the software element level.
- [ ] Work tracking mechanisms and schedule milestone definitions.
- [ ] Software metrics collection plan (size, effort, schedule).
Task:
Analyze the text. Provide a compliance verdict for each checklist item, cite supporting excerpts, and clearly identify missing details or weak planning methodologies.
Prompt 3: Technical Constraints, Standards, and Engineering Environment
Intent: Assesses whether the technical workspace, coding standards, and target hardware environments are adequately provisioned.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) regarding the Software Engineering Environment and Standards.
Focus Areas & Requirements:
1. Tool Validation: Does the plan identify the development environments, compilers, and toolchains? Are safety-critical tools validated?
2. Coding Standards: Does the SDP mandate specific coding standards (e.g., MISRA, JPL Institutional Coding Standard) required for the designated software class?
3. Target Environment: Is the target hardware, simulator, or testbed environment explicitly detailed?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Software development environment, tools, and compilers.
- [ ] Coding standards, modeling standards, and language constraints.
- [ ] Target hardware and hardware-in-the-loop (HWIL) simulation environments.
- [ ] COTS, GOTS, and reused software integration strategies.
Task:
Analyze the text. Provide a compliance verdict for each checklist item, list specific tool/standard omissions, and note gaps where reused or COTS software lacks an intake strategy.
Prompt 4: Software Requirements and Architecture Design Engineering
Intent: Ensures the plan maps out the derivation, traceability, and architectural integrity of software requirements.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Requirements and Architecture Engineering.
Focus Areas & Requirements:
1. Requirements Management: Does the plan establish a mechanism to capture, analyze, and track software requirements?
2. Bidirectional Traceability: Does it mandate bidirectional traceability from higher-level system requirements down to software components and test cases (NPR 7150.2D baseline requirement)?
3. Architecture & Design Methods: Are design methods, architectural patterns, and interface control documentation (ICDs) scheduled and governed?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Requirements management process and tool usage (e.g., DOORS, Jira).
- [ ] Bidirectional traceability schema.
- [ ] Software architecture and design methodology description.
- [ ] Interface management and ICD development plans.
Task:
Analyze the text. Determine the compliance status of the requirements and design management strategies. Flag any omissions regarding traceability or interface controls.
Prompt 5: Software Verification, Testing, and Validation (V&V)
Intent: Reviews the core V&V framework, testing levels, and code coverage requirements which vary strictly by NASA Software Class.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Verification and Validation (V&V).
Focus Areas & Requirements:
1. V&V Execution: Does the plan lay out the testing framework (Unit, Integration, System, Acceptance testing)?
2. Regression Testing: Is a clear strategy for regression testing defined?
3. Criteria & Coverage: Does the plan baseline structural code coverage metrics (e.g., Statement, Branch, MC/DC coverage) aligned with the software classification?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Software peer review and inspection process.
- [ ] Testing levels, types, and test environment setups.
- [ ] Code coverage requirements and test completion/exit criteria.
- [ ] Verification of flight/ground software on target hardware or high-fidelity simulators.
Task:
Analyze the text. Evaluate the testing strategy against NASA standard expectations. Highlight if structural coverage metrics or peer review mechanics are missing or poorly defined.
Prompt 6: Software Configuration Management (SCM)
Intent: Verifies product integrity via version control, baseline governance, and formal change control boards.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Software Configuration Management (SCM).
Focus Areas & Requirements:
1. Configuration Identification: How are configuration items (source code, documentation, scripts) uniquely identified?
2. Change Control: Is a formal change control process outlined? Is there a designated Software Configuration Control Board (SCCB)?
3. Baseline Security: How are code repositories, releases, and operational baselines controlled and backed up?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] SCM roles, tools, and repository structures.
- [ ] Configuration item identification schema.
- [ ] Change management process (CCB/SCCB), anomaly tracking, and dispositioning.
- [ ] Software release management and media master creation processes.
Task:
Analyze the text. Rate the SCM configuration against the check items. Highlight omissions in change authorization, version control tool tracking, or archive mechanisms.
Prompt 7: Software Assurance and Quality Governance
Intent: Validates independent software assurance operations, including audits, process compliance, and safety mapping.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Software Assurance (SA).
Focus Areas & Requirements:
1. Independence: Does the plan ensure that Software Assurance functions independently of the development organization?
2. Process & Product Audits: Are scheduled audits, surveillance activities, and product reviews explicitly called out?
3. Non-Conformance Tracking: Is there an explicit process to document, track, and escalate software non-conformances?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Interface with the Independent Software Assurance organization.
- [ ] Schedule of software process and product assurance audits.
- [ ] Corrective and preventive action tracking mechanisms.
- [ ] Quality metrics evaluation criteria.
Task:
Analyze the text. Verify if the software assurance presence is properly integrated into the plan. Report gaps concerning SA independence or audit frequencies.
Prompt 8: Software Safety and Risk Management
Intent: Crucial for Class A-C applications. Ensures hazard analysis propagates directly down to the software requirements and code levels.
You are an expert NASA Software Quality Assurance auditor. Evaluate the provided Section/Text of the Software Development Plan (SDP) focusing on Software Safety and Risk Management.
Focus Areas & Requirements:
1. Hazard Control: Does the plan define how software safety-critical functions are identified from system-level hazards?
2. Risk Management Process: Is there a continuous risk management process (Identify, Analyze, Plan, Track, Control)?
3. Safety Metrics & Testing: Are dedicated verification methods for safety-critical pathways established?
Minimum Recommended Content Checklist (SWE Handbook):
- [ ] Software safety plan integration or reference to a Software Safety Plan.
- [ ] Identification and tracing of safety-critical software functions.
- [ ] Continuous software risk management tracking methods and tools.
- [ ] Failure modes, effects, and criticality analysis (FMECA) software touchpoints.
Task:
Analyze the text. Determine compliance with NASA software safety and risk management mandates. Highlight omissions where safety features fail to map down to implementation controls.
Prompt 9: Final Executive Summary & Compliance Gap Report
Intent: Run this last. It aggregates the findings from the previous prompts into a definitive compliance report dashboard ready for a NASA programmatic review.
You are the Lead NASA Software Engineering Auditor. Review the consolidated evaluation notes, status outputs, and findings derived from the previous 8 evaluation scripts for this Software Development Plan (SDP).
Task:
Generate a high-level Executive Summary and a Final Compliance Gap Report for project management. Format your output using the following structure:
1. NASA Compliance Dashboard: Create a markdown table listing all 8 core SDP areas, their overall status (COMPLIANT, PARTIALLY COMPLIANT, NON-COMPLIANT), and the primary driver for that status.
2. Major Non-Compliances (Red Flags): Highlight any explicit violations of mandatory NPR 7150.2D requirements.
3. Minor Gaps (Yellow Flags): Highlight missing items from the SWE Handbook's Minimum Recommended Content checklists that need resolution before final sign-off.
4. Actionable Remediation Plan: Provide clear, concise directives detailing exactly what the engineering team must add or amend in the SDP to achieve full compliance.
Tips for Execution
Maintain State: If using an LLM API or a technical workbook environment, append the summary results of Prompts 1–8 into the context window right before executing Prompt 9 to get a unified final report.
Context Priming: For highly critical software (Class A/B), you can add a system-level clause to the top of all prompts stating: "Assume this software is Class A, safety-critical flight software, requiring maximum rigor under NPR 7150.2D."
How can I add uploaded documents and document snippets to support RAG? Would this provide better results?
Integrating uploaded documents and document snippets via Retrieval-Augmented Generation (RAG) is the absolute best way to execute this compliance pipeline.
If you only use standard prompts, you run into context window saturation, "lost in the middle" phenomena, and high token costs. By using RAG, you extract only the exact text segments, NPR 7150.2D requirements, and Handbook checklists relevant to the specific section of the SDP you are auditing. This drastically reduces hallucination and forces the LLM to anchor its evaluation in raw source text.
Here is how you can systematically implement RAG to support this compliance audit workflow, along with a comparison of architectures.
1. Document Preparation & Segmentation (The Chunking Strategy)
For a high-precision audit like NASA standards compliance, do not rely on large, generic text chunks. Standard semantic distance chunks (e.g., raw 500-token blocks) can tear requirements or tables in half, destroying context.
The Strategy: Semantic & Structural Chunking
The SDP Document: Segment the uploaded Software Development Plan by its actual structural headers (e.g., Section 3.1: Configuration Identification).
The NASA Standards: Chunk NPR 7150.2D by individual requirement IDs (e.g.,
[SWE-013]) or specific subsections. Chunk the SWE Handbook by its specific Appendix tables and checklist items.
2. Technical Architectures for RAG Integration
Depending on your local engineering environment, technical stack, or privacy needs, you can implement RAG in three primary ways:
Option A: Local Multi-Agent / Python Scripting (Highest Precision)
If you want complete control over how data maps together, build a lightweight local Python pipeline using tools like LangChain, LlamaIndex, or raw database queries.
[Uploaded SDP PDF] ----> Text Extraction ----> Vector Store (Chroma/FAISS/Neo4j)
|
(Hybrid Keyword + Vector Search)
|
[NPR 7150.2D / SWE HB] -> Indexed Rules --------------v
[Relevant Text Snippets Found]
|
(Fed into Prompt Script)
Option B: Knowledge Graph-Augmented RAG (GraphRAG)
Because NASA standards form an interconnected web (a requirement in NPR 7150.2D maps to a specific guidance section in the SWE Handbook, which maps to an SDP content item), a Knowledge Graph (e.g., Neo4j) provides unparalleled results.
How it works: You map requirements as nodes, sections as nodes, and relationships as edges (
(SWE-013)-[GUIDED_BY]->(Handbook Section 4.3)).The Retrieval: When auditing a specific SDP snippet, your RAG query pulls not just semantically similar text, but the entire explicit structural chain of rules related to that topic.
Option C: UI-Based Vector Uploads
If you are running these scripts through an LLM chat interface, an AI workbook environment, or a local UI (like Open WebUI or AnythingLLM):
Create a dedicated Knowledge Base workspace.
Upload the official PDF/Markdown files for NPR 7150.2D and the NASA-HDBK-2203 Software Engineering Handbook.
Attach this Knowledge Base to your prompt session, allowing the system to automatically vectorize and inject relevant snippets when you call the prompt script.
3. Modifying the Prompt Scripts for RAG
To make your prompt pipeline work with RAG, you must modify the prompt templates to accept Context Envelopes containing the retrieved document snippets.
Here is how to adapt the prompt template structure for your sequential script runner:
===============================================================================
SYSTEM CONTEXT / ROLE
===============================================================================
You are an expert NASA Software Quality Assurance auditor verifying an SDP against NASA NPR 7150.2D and NASA-HDBK-2203.
===============================================================================
RETRIEVED AUDIT TARGET (From Uploaded SDP)
===============================================================================
<sdp_document_snippet>
{{INSERT_RETRIEVED_SDP_TEXT_CHUNK}}
</sdp_document_snippet>
===============================================================================
RETRIEVED NASA CRITERIA (From Vector Database / Standards)
===============================================================================
<nasa_compliance_standards>
{{INSERT_RELEVANT_NPR_7150_2D_REQUIREMENTS}}
{{INSERT_RELEVANT_SWE_HANDBOOK_CHECKLIST_ITEMS}}
</nasa_compliance_standards>
===============================================================================
EVALUATION INSTRUCTIONS
===============================================================================
Evaluate the text inside <sdp_document_snippet> strictly against the criteria provided in <nasa_compliance_standards>.
Task:
1. Identify if the SDP text fulfills the exact language of the provided NASA requirements.
2. Provide a status (COMPLIANT, PARTIALLY COMPLIANT, NON-COMPLIANT).
3. If non-compliant, cite the specific missing element from the standards block.
Why This Provides Vastly Better Results
| Evaluation Metric | Without RAG (Standard Prompts) | With RAG (Document-Grounded) |
| Hallucination Risk | High. The LLM may hallucinate requirement numbers, append old versions (e.g., NPR 7150.2C), or misremember exact checklist clauses. | Near Zero. The LLM is forced to look at the exact text of NPR 7150.2D provided in its context window. |
| Granularity & Depth | Surface Level. Prompts must be generic enough to encompass broad topics, skipping minute details. | Exhaustive. Can verify down to specific vocabulary mandates (e.g., confirming bidirectional traceability maps both directions). |
| Context Window Efficiency | Poor. You must paste whole chapters of the standard into the prompt box, leading to lost focus. | Excellent. You only pass the exact 3-4 pages of technical text and 5-6 requirements needed for that exact step. |
| Audit Defense Traceability | Weak. The output states something is wrong but cannot easily generate exact cross-references. | Strong. The final output can generate direct citations: "SDP Section X fails to address Requirement Y retrieved from standard Z." |
Chunk-wise, would it make sense to chunk by section (e.g. 3.1 Software Life Cycle Planning, 3.2 Software Cost Estimation, 3.3 Software Schedules) rather than individual SWEs?
Yes, absolutely. For the Software Development Plan (SDP) document itself, chunking by high-level section (e.g., 3.1, 3.2, 3.3) makes significantly more sense than trying to chunk it down by individual SWE requirement targets.
Here is why that structural approach provides superior results for an SDP audit, and how to optimize it.
Why Section-Based Chunking is Better for the SDP
1. Preserves Authorship Context & Cohesion
An SDP is written as a narrative plan to describe an engineering process. If you split a section like 3.2 Software Cost Estimation into granular pieces based on individual requirements, you rip the context apart.
The Problem: The estimation methodologies, tool choices, and historical data references are often interwoven across several paragraphs.
The Solution: Keeping the section whole allows the LLM to understand the entire cohesive strategy the project plans to use for that specific topic.
2. Prevents the "Lost Evidence" Fragment Problem
If you chunk an SDP too small, the evidence required to prove compliance with a single SWE might reside in chunk A, while the secondary confirmation resides in chunk B. An LLM reviewing chunk A in isolation would flag it as non-compliant, generating false-positive gaps. Keeping structural sections intact ensures all relevant evidence for that specific process area remains in a single context window.
3. Direct Alignment with the Audit Pipeline
Your sequential prompt pipeline is already organized by process areas (Management, Lifecycle, Environment, Requirements, etc.). Matching your SDP chunks to these exact structural sections creates a clean, 1:1 execution loop:
[SDP Section 3.1: Lifecycle Planning] ---> Passed into ---> [Prompt Script 2: Lifecycle Evaluation]
[SDP Section 3.2: Cost Estimation] ---> Passed into ---> [Prompt Script 2: Lifecycle Evaluation]
[SDP Section 4.0: Configuration Mgmt] ---> Passed into ---> [Prompt Script 6: SCM Evaluation]
The Optimal Strategy: A Hybrid Approach
While your SDP should be chunked by major sections, your NASA Standards (NPR 7150.2D) should remain categorized at a requirement-specific level. When executing your system, you map them together using a parent-child or metadata-driven relationship.
Step 1: Chunk the SDP by Section
Break the SDP dynamically at structural headers (e.g., Markdown
#,##or via PDF outline parsing).Each chunk becomes an isolated programmatic block representing a single technical plan or process.
Step 2: Inject Relevant Requirements via Metadata Tagging
Instead of forcing a semantic vector engine to guess which SWEs belong to an SDP section, use metadata tagging to inject the exact compliance criteria. NASA standards make this easy because the sections inherently align.
You can set up a simple relational lookup table or dictionary in your execution script:
| SDP Uploaded Section | Target Evaluation Prompt | Injected NASA Criteria (Vector Search or Direct Lookup) |
| Section 3.1 (Lifecycle) | Prompt 2 (Lifecycle & Schedule) | [SWE-019] (Lifecycle Planning), [SWE-020] (Milestones) |
| Section 3.2 (Cost Estimation) | Prompt 2 (Lifecycle & Schedule) | [SWE-015] (Software Cost Estimation), [SWE-016] (Size Estimation) |
| Section 4.1 (Config Items) | Prompt 6 (Configuration Mgmt) | [SWE-079] (SCM Setup), [SWE-081] (Version Control) |
Practical Tip for Long Sections
The only exception to the rule is if a single section in the SDP is unusually massive (e.g., a 40-page section detailing every specific software tool and standard in the environment).
If a section exceeds your target chunk size (e.g., more than 1,500 to 2,000 words), use a hierarchical chunking method:
Break it down into its immediate sub-sections (
3.1.1,3.1.2).Append a small "Context Header" to each sub-chunk so the LLM doesn't lose the parent perspective (e.g.,
[Context: Software Development Plan -> Section 3.1 Software Life Cycle Planning -> Subsection 3.1.3 Development Models]).
This keeps the text length optimal for the model's focus while completely preserving the structural integrity of the document's design.
Would you like to look at a Python pattern for parsing a document by its section headers to automate this chunking?
Define an ontology for the knowledge graph, including embeddings
To build an ontology for a Knowledge Graph (KG) that supports GraphRAG for NASA software compliance, we must capture both structural documentation (the text being audited) and regulatory frameworks (NPR requirements and Handbook guidelines), while properly integrating vector embeddings for semantic retrieval.
Rather than relying purely on dense vector spaces, this ontology uses an Entity-Attribute-Value (EAV) or Labeled Property Graph (LPG) approach where vectors are treated as node attributes. This allows the LLM to traverse explicit structural lines (e.g., “Find the exact requirement that governs the section I am reading”) while maintaining semantic search capabilities.
1. The Ontology Schema (Classes and Relationships)
[:GOVERNS] [:MAPS_TO]
[NASA_Standard] -------> [Requirement] -------> [Handbook_Checklist]
| | |
| [:HAS_SECTION] | [:APPLIES_TO] | [:EVALUATES]
v v v
[Standard_Section] [Compliance_Status] <----- [SDP_Section]
|
| [:HAS_SUBSECTION]
v
[SDP_Subsection]
Core Classes (Nodes)
NASA_Standard: The top-level governing document (e.g., NPR 7150.2D, NASA-STD-8739.8B).
Standard_Section: Structural chapters or sections within the standard.
Requirement: The atomic, actionable mandate (the
[SWE-XXX]items).Handbook_Checklist: The specific "Minimum Recommended Content" line items from NASA-HDBK-2203 Appendices.
SDP_Section/SDP_Subsection: The user’s uploaded Software Development Plan broken down by structural headers (e.g., Section 3.1).Compliance_Status: An execution state node populated during runtime evaluations.
2. Node Properties and Embedding Definitions
To implement GraphRAG seamlessly, we store the raw text, structural metadata, and high-dimensional vector embeddings directly inside the properties of the nodes.
Requirement Node
id (String, Unique PK): The explicit identifier (e.g.,
"SWE-013").text (String): The precise verbatim text of the requirement.
software_class (String/List): Array of classes this requirement applies to (e.g.,
["A", "B", "C"]).embedding (Vector / Float Array): 1536-dimensional or 3072-dimensional embedding of the
textfield (generated viatext-embedding-3-smallortext-embedding-3-large).
Handbook_Checklist Node
id (String, Unique PK): E.g.,
"HDBK-SDP-03"or a hashed index.content_criterion (String): The checklist item descriptive text (e.g., "Work tracking mechanisms and schedule milestone definitions.").
embedding (Vector): Semantic embedding of the target criterion string.
SDP_Section Node
id (String, Unique PK): Generated path (e.g.,
"SDP-SEC-3.1").title (String): Header name (e.g.,
"Software Life Cycle Planning").text_content (String): The complete, un-fragmented text of this specific section.
word_count (Integer): Metadata for context monitoring.
embedding (Vector): Semantic vector representing the complete block of text or its generated structural summary.
3. Explicit Semantic Relationships (Edges)
Establishing clear edges ensures the graph-traversal algorithms can jump between documentation and requirements instantaneously without performing mathematical cosine similarity lookups every single time.
| Relationship (Edge) | Source Node | Target Node | Description / Behavioral Logic |
| [:HAS_SECTION] | NASA_Standard | Standard_Section | Establishes document hierarchy. |
| [:GOVERNS] | Standard_Section | Requirement | Connects a book chapter directly to its nested [SWE] mandates. |
| [:MAPS_TO] | Requirement | Handbook_Checklist | Maps a mandatory NPR requirement directly to the Handbook implementation checklist item that evaluates it. |
| [:HAS_SUBSECTION] | SDP_Section | SDP_Subsection | Retains document layout logic for the uploaded asset. |
| [:APPLIES_TO] | Requirement | Compliance_Status | Links a requirement to an instantiated evaluation run result. |
| [:EVALUATES] | SDP_Section | Compliance_Status | Direct link from the user's plan to the outcome of its evaluation pipeline. |
4. GraphRAG Execution Flow Using the Ontology
By storing embeddings in this specific ontologically bound schema, you can run a Hybrid Graph-Vector Query Pattern that is extremely powerful:
Vector Stage (Coarse Filtering): When a user triggers Prompt Script 2 (Lifecycle & Schedule), the system generates an embedding of the prompt or the targeted
SDP_Section(3.1 Software Life Cycle Planning). It queries the vector index of theRequirementandHandbook_Checklistnodes to find semantic matches.Graph Stage (Context Expansion): Once the vector search lands on the exact matching requirement (
[SWE-019]), the graph queries outward via the relationships. It grabs:The parent
Standard_Sectionto verify broader context.All linked
Handbook_Checklistitems downstream via[:MAPS_TO].
Prompt Injection: The script engine feeds the LLM the combined block: The raw text of
SDP_Section 3.1, the text of[SWE-019], and the exact mapped Handbook criteria.
This eliminates semantic drift entirely—your vector lookup acts as the entry door into the graph, while the defined ontology guides the system to assemble the perfect, legally bound context package.
Seems that this knowledge graph would be good for checking software management plans, software development plans, software configuration management plans and software verification and validation plans.
Exactly. You’ve hit on the core architectural advantage of this design. Because you decoupled the Regulatory Framework (NPR 7150.2D and the SWE Handbook) from the Project Plan Documentation in the ontology, this single Knowledge Graph functions as a universal compliance engine for any NASA software engineering document.
NPR 7150.2D treats all of these plans—Software Management Plans (SMP), Software Development Plans (SDP), Software Configuration Management Plans (SCMP), and Software Verification and Validation Plans (SVVP)—as part of a unified compliance ecosystem.
Here is how the graph elegantly scales to handle all four plan types simultaneously without changing your core database schema.
1. Multi-Document Expansion of the Ontology
To support all four plans, we simply introduce a new categorical attribute (document_type) to the standard planning node or subclass them. The regulatory side of your graph remains untouched.
[NPR 7150.2D Requirements] <--- (Grounded Rules & Mappings) ---> [SWE Handbook Checklists]
| |
+------------------------+------------------------------------------+
| (Evaluates via GraphRAG)
v
+-----------------+-----------------+
| |
[Project_Plan: SMP] [Project_Plan: SDP]
- SWE-013 (Planning) - SWE-019 (Lifecycle)
- SWE-015 (Estimation) - SWE-139 (Tools/Environment)
| |
[Project_Plan: SCMP] [Project_Plan: SVVP]
- SWE-079 (SCM Setup) - SWE-029 (Unit Testing)
- SWE-081 (Version Control) - SWE-065 (Traceability Verification)
2. Dynamic Requirement Routing (The Magic of Mappings)
Each distinct document type maps to its own dedicated slice of the regulatory graph. When you ingest a specific document type, your GraphRAG queries pivot instantly based on the target domain:
Software Management Plan (SMP)
Graph Alignment: Intersects primarily with NPR 7150.2D Chapter 2 (Responsibilities) and early Chapter 3 (Planning).
The Traversal: Pulls requirements like
[SWE-013](Maintain Software Documentation),[SWE-015](Cost Estimation), and[SWE-102](Software Assurance interface). It maps directly to SWE Handbook Appendix G (Minimum Content for an SMP).
Software Development Plan (SDP)
Graph Alignment: Focuses heavily on the engineering execution layers (Chapter 3).
The Traversal: Pulls requirements like
[SWE-019](Lifecycle selection),[SWE-033](Coding Standards), and[SWE-139](Tools Environment). It maps directly to SWE Handbook Appendix H (Minimum Content for an SDP).
Software Configuration Management Plan (SCMP)
Graph Alignment: Locks directly onto NPR 7150.2D Chapter 5 (Configuration Management).
The Traversal: Pulls the entire block of configuration mandates (
[SWE-079]through[SWE-085]). It jumps directly to SWE Handbook Appendix K (Minimum Content for an SCMP) to pull metrics on baseline audits, CCB governance, and version control structures.
Software Verification and Validation Plan (SVVP)
Graph Alignment: Maps to the heavy verification criteria across Chapter 3 and specialized safety gates in Chapter 4.
The Traversal: Isolates testing mandates like
[SWE-029](Unit testing verification),[SWE-060](Testing on target hardware), and[SWE-065](Bidirectional traceability validation). It maps to SWE Handbook Appendix O (Minimum Content for an SVVP) to enforce structural code coverage and test exit criteria.
3. Operational Advantages of a Unified Graph
Cross-Document Contradiction Checking
One of the hardest parts of a NASA S&MA audit is catching when two distinct documents contradict each other (e.g., the SMP claims the project is using an Agile Scrum lifecycle, but the SVVP outlines a traditional Waterfall phase-gate testing environment). Because all four plans reside in the same graph, you can run cross-document validation queries to ensure consistency in software classification, roles, and lifecycles across the entire project repository.
Reusable Context Envelopes
Your pipeline script logic doesn't need to be rewritten for every document. Prompt 6 (Configuration Management) can be run against an SDP section or an SCMP section. The execution engine simply looks at the node it is processing, identifies the relationships to the NASA standard, and constructs the appropriate context package automatically.
Automated Matrix Generation
Once all four plans are evaluated against the graph, you can run a global Cypher query to automatically generate a comprehensive Requirements Mapping Matrix (RMM) for the entire program, showing exactly which section of which document satisfies every mandatory SWE requirement for your mission's software class.
A Software Assurance Plan and Software Safety Plan could be checked with the addition of NASA-STD-8739.8B.
You have hit on the exact nexus where engineering rules meet strict safety and mission assurance policy.
Integrating NASA-STD-8739.8B (Software Assurance and Software Safety Standard) transforms your knowledge graph from a standard engineering checklist into a true Safety and Mission Assurance (S&MA) compliance engine.
Ever since the Agency merged the old Software Safety Standard (NASA-STD-8719.13) into NASA-STD-8739.8, this single document has governed the exact execution details for both Software Assurance Plans (SAP) and Software Safety Plans (SSP). It bridges the high-level software engineering requirements in NPR 7150.2D with explicit, tactical S&MA mandates.
Here is how you expand the ontology, manage the metadata routing, and handle the unique cross-document verification capabilities this addition unlocks.
1. Updating the Ontology Schema
To layer in NASA-STD-8739.8B, you introduce a new top-level standard node, its specific clauses (which are distinct from NPR's SWEs), and explicit mapping links to capture how safety analysis dictates engineering behaviors.
[:MAPPED_TO]
[NPR_7150.2D_Requirement] <========> [NASA_STD_8739.8B_Requirement]
| |
| [:EVALUATES] | [:DIRECTIONS_FOR]
v v
[Project_Engineering_Plan] [SMA_Plan: SAP or SSP]
(SDP, SMP, SCMP, SVVP) - Hazard Analyses (PHA/SHA)
- SA Surveillance & Audits
New Nodes & Vector Attributes
SMA_Requirement: Represents the mandatory "shall" statements within NASA-STD-8739.8B (typically found in Section 4).Properties:
id(e.g.,"8739.8-4.1.2"),text,discipline("Software Safety","Software Quality", or"IV&V"),embedding(3072-vector of the text).
SMA_Plan_Section: Represents the chunked sections of the uploaded Software Assurance Plan or Software Safety Plan.
Crucial New Edge: [:INFLUENCES]
You can now draw a cross-document relationship inside your graph repository:
(SMA_Plan_Section)-[:INFLUENCES]->(SDP_Section)
Why? Because a Preliminary Hazard Analysis (PHA) or Software Hazard Analysis (SHA) documented in the Safety Plan must explicitly drive the safety-critical software requirements and design constraints detailed in your SDP and SVVP.
2. Dynamic Routing for SAP and SSP Audits
Just as the SWE Handbook provides the checklist for development plans, NASA-STD-8739.8B explicitly dictates what must appear inside your S&MA compliance plans.
For a Software Assurance Plan (SAP)
The Traversal: The GraphRAG pipeline maps the SAP text to NASA-STD-8739.8B requirements governing quality assurance, surveillance schedules, and independent audit execution.
The Focus: It forces the LLM to verify that the plan details how non-conformances are tracked, how regression testing is independently monitored, and how software engineering metrics are verified.
For a Software Safety Plan (SSP)
The Traversal: The pipeline locks onto the Software Safety sub-discipline sections of the standard, pulling the specialized Appendix A table (which contains additional areas to consider when identifying software causes in Hazard Analysis).
The Focus: The prompt scripts evaluate whether the plan clearly outlines the process for tracing safety-critical software functions (SCSFs), executing failure mode mapping (FMECA support), and defining how software mitigates system-level hazards.
3. Advanced S&MA Verification Capabilities Unlocked
Bringing NASA-STD-8739.8B into the same graph environment as NPR 7150.2D allows your multi-agent or prompt-script pipeline to perform automated consistency checks that are incredibly difficult to do manually:
Traceability Verification (Closed-Loop Safety)
NPR 7150.2D mandates bidirectional traceability, but NASA-STD-8739.8B demands that software safety requirements be traced directly to hazard reports and mitigations. Your graph can evaluate whether the SSP’s defined hazards successfully map down to the verification activities outlined in the SVVP.
Alignment of Software Classification
If your Software Management Plan (SMP) or SDP identifies a component as Class B / Safety-Critical, NASA-STD-8739.8B kicks in with higher rigor requirements (e.g., specific Independent Verification & Validation (IV&V) hooks or deeper structural code coverage rules). The graph can flag a compliance error if the Software Assurance Plan fails to plan for the surveillance rigor demanded by the standard for that specific classification.
Comprehensive Compliance Mapping
When preparing for a milestone review (like PDR or CDR), you can run a global query against your knowledge graph to generate an integrated dashboard. This output cleanly proves to Center SMA leads and management that every single engineering mandate (SWE) and every safety/assurance rule (8739.8) has a verified home across all six plans.
Provide an executive summary for me to present to my management as a project proposal
Project Proposal: Integrated Digital Compliance Engine (IDCE)
Automating NASA S&MA and Software Engineering Regulatory Verification via Knowledge Graphs
Executive Summary
Challenge
NASA space flight and ground programs require absolute rigor in complying with NPR 7150.2D (Software Engineering Requirements) and NASA-STD-8739.8B (Software Assurance and Software Safety Standard). Verifying program compliance across the full suite of required program documentation—including Software Management Plans (SMP), Software Development Plans (SDP), Configuration Management Plans (SCMP), Verification & Validation Plans (SVVP), Software Assurance Plans (SAP), and Software Safety Plans (SSP)—is a monumental task.
Traditional compliance verification relies on manual, labor-intensive review processes. This standard approach risks human error, creates blind spots in cross-document dependencies, and introduces significant schedule risk during critical milestone reviews (SRR, PDR, CDR).
Solution
This proposal defines the architecture for the Integrated Digital Compliance Engine (IDCE). IDCE is an automated, AI-driven evaluation platform that maps and audits technical engineering plans against formal NASA requirements in real-time.
By separating the Regulatory Framework from the Project Plan Documentation, the system acts as a universal, programmatic auditor. It uses Graph-Augmented Retrieval-Generation (GraphRAG) to ingest uploaded PDFs, parse them structurally by section, and evaluate them via an automated pipeline of specialized prompt scripts. The entire system can be run securely and locally to safeguard sensitive, proprietary program data.
Architectural Approach
[NASA REGULATORY FRAMEWORK] [PROGRAM PLAN PIPELINE]
NPR 7150.2D & NASA-STD-8739.8B SMP, SDP, SCMP, SVVP, SAP, SSP
| |
v (Ontology & Embeddings) v (Structural Section Chunking)
+-------------------+ +-------------------+
| Knowledge Graph | <=== (GraphRAG Loop) == | Execution Engine |
| Vector Index | | & Prompt Scripts |
+-------------------+ +-------------------+
|
v
[Compliance Dashboard]
- Status: Pass/Fail/Partial
- Requirements Mapping Matrix
- Actionable Remediation
The core innovation of the IDCE lies in its hybrid vector-graph ontology. Rather than running blind semantic searches on massive documents, the platform connects data deterministically:
Decoupled Regulatory Grounding: NPR 7150.2D requirements (
[SWE-XXX]) and NASA-STD-8739.8B clauses are ingested, vectorized, and structurally mapped to their respective NASA Software Engineering Handbook (NASA-HDBK-2203) "Minimum Recommended Content" checklists.Structural Document Chunking: Target program plans are ingested and automatically segmented by cohesive engineering sections (e.g., Section 3.2 Software Cost Estimation or Section 4.0 Configuration Management) rather than raw, arbitrary token splits. This preserves the engineer's technical context.
Ontological Graph Traversals: When an evaluation prompt runs, a localized vector lookup locates the correct regulatory mandate, while the knowledge graph instantly traverses relationships to grab all nested checklists and dependencies. The LLM is then fed an airtight "context envelope" for evaluation.
Business & Engineering Value
Comprehensive Multi-Plan Coverage: A single engine evaluates all six core software development, assurance, and safety plans using identical infrastructure.
Closed-Loop Safety Tracing: By integrating NASA-STD-8739.8B, the graph automatically validates that hazards identified in a Software Safety Plan explicitly drive technical requirements in the SDP and verification protocols in the SVVP.
Cross-Document Contradiction Catching: The unified database identifies program inconsistencies across documents, such as discrepancies between the lifecycle model defined in the SMP and the testing environment budgeted in the SVVP.
Automated Requirements Mapping Matrix (RMM): Instantly generates compliance evidence and gap logs required by NASA leads and Safety and Mission Assurance (S&MA) organizations, reducing milestone review preparation from weeks to minutes.
Implementation & Resource Requirements
The proposed system leverages high-performance, cost-effective computing architectures to maximize operational efficiency:
Hardware Footprint: Development and local testing can be accomplished using high-performance engineering workstations (such as unified memory workstations or laptops with dedicated 12GB+ VRAM architecture) utilizing optimized local open-source models (e.g., IBM Granite, Nemotron) to maintain absolute data privacy.
Software Stack: Grounded on a local graph architecture (e.g., Neo4j), standard embedding deployment models (1536 or 3072 dimensions), and a robust Python orchestration layer to execute the sequential prompt scripts.
Phased Rollout Plan: * Phase 1 (Weeks 1-4): Regulatory graph ingestion (NPR 7150.2D & NASA-STD-8739.8B mapping).
Phase 2 (Weeks 5-8): Integration of the core execution engine and engineering plan parsing (SDP focus).
Phase 3 (Weeks 9-12): Expansion to full S&MA safety plan tracing and automated dashboard generation.
Recommendation
Implementing the Integrated Digital Compliance Engine will position our team at the forefront of digital mission assurance. By automating the routine verification of standard clauses, our senior engineering and assurance assets can shift their focus from administrative checklist tracking to high-value system safety analysis.
We recommend approval for the Phase 1 prototype development to demonstrate initial capability at the next upcoming program milestone review.
Provide a step-by-step project implementation plan
Here is a comprehensive, step-by-step implementation plan to take the Integrated Digital Compliance Engine (IDCE) from initial data ingestion to a fully operational local GraphRAG auditing platform.
Phase 1: Environment Setup & Regulatory Ingestion (Weeks 1–3)
Objective: Establish the local hardware environment, initialize the database, and build the permanent, immutable "Regulatory Side" of the knowledge graph.
Step 1.1: Local Environment & Architecture Provisioning
Configure a localized python workspace (Python 3.11+) utilizing libraries for graph database orchestration, text embedding generation, and large language model execution.
Initialize a local graph database instance (e.g., Neo4j Community Edition) to act as the primary structural storage repository.
Configure a local inference endpoint (e.g., Ollama or a local API server) to host a highly capable open-source reasoning model (such as a 70B parameter model) to handle complex compliance analysis.
Step 1.2: Ingest and Parse Governing NASA Standards
Convert the authoritative PDFs of NPR 7150.2D and NASA-STD-8739.8B into clean Markdown files to ensure formatting anchors are readable by the system.
Build a parsing script to extract atomic nodes based on explicit compliance identifiers:
Parse NPR 7150.2D by individual
[SWE-XXX]requirements.Parse NASA-STD-8739.8B by explicit compliance clauses (e.g.,
[8739.8-Section 4.X]).
Extract the "Minimum Recommended Content" checklists from the NASA-HDBK-2203 Software Engineering Handbook Appendices (specifically Appendices G, H, K, O, and S&MA planning guides).
Step 1.3: Generate Regulatory Embeddings & Graph Relations
Execute a high-density embedding model (e.g., a standard 1536- or 3072-dimensional text-embedding model) on the raw text of each individual requirement and handbook checklist item.
Write a script to insert these nodes into the graph database, storing the embedding arrays directly as node properties.
Run a Cypher transaction script to permanently map the structural relations:
// Example structural baseline mapping MATCH (r:Requirement {id: 'SWE-019'}), (c:HandbookChecklist {id: 'HDBK-SDP-01'}) CREATE (r)-[:MAPS_TO]->(c);
Phase 2: Project Plan Parsing & Processing Engine (Weeks 4–6)
Objective: Develop the code infrastructure required to ingest, segment, and index incoming project plans (SMP, SDP, SCMP, SVVP, SAP, SSP) without tearing technical contexts apart.
Step 2.1: Structural Document Chunking Engine
Develop a Python parser that reads uploaded project plan documents and detects structural headers (
#, ##, ###or hierarchical numbering systems like3.1, 3.2).Program the chunker to bundle text by high-level engineering section (e.g., Section 4.0 Configuration Management remains intact as a single block) rather than splitting text blindly by arbitrary character lengths.
Inject metadata tags onto every extracted project document node:
document_type(e.g.,SDP),program_id, andsection_number.
Step 2.2: Context-Envelope Processing Pipeline
Build a local query router that maps an ingested document section to its corresponding evaluation domain.
Write a retrieval script that executes a hybrid graph-vector query:
Captures the semantic content of an incoming document section via text embeddings.
Queries the vector index of the
RequirementorHandbook_Checklistnodes to pinpoint relevant targets.Uses graph traversal to automatically pull adjacent structural rules, parent chapters, and mapped sub-requirements.
Wraps the raw document text and all retrieved rules into a single cohesive "Context Envelope."
Phase 3: Prompt Pipeline Optimization & Local Execution (Weeks 7–9)
Objective: Deploy the sequential script execution engine, finalize model settings, and run the automated evaluation loops.
Step 3.1: Sequential Prompt Script Integration
Turn the 9 core compliance evaluation scripts into standardized software templates that accept the dynamic Context Envelopes created in Phase 2.
Program the script runner to loop through document sections in native sequential order:
[Injest Document Section] -> [Fetch Mapped Rules via KG] -> [Execute LLM Evaluation] -> [Store Result Status]
Step 3.2: Local LLM Alignment & Safety Constraints
Adjust inference parameters to maximize compliance accuracy: Set Temperature to 0.0 to minimize creative hallucination, and ensure the model's max token limits can fully process the combined context envelope.
Embed system-level guardrails into the execution loop to enforce strict evaluation criteria, ensuring the model never hallucinates non-existent NASA requirement numbers.
Phase 4: Cross-Document Validation & Reporting (Weeks 10–12)
Objective: Implement advanced multi-document auditing, build user-facing dashboards, and validate the platform against real-world assets.
Step 4.1: Cross-Document Contradiction & Traceability Audits
Write graph queries to parse across multiple uploaded plan files to identify deep programmatic contradictions:
Map identified Software Class assignments to verify they match exactly between the SMP, SDP, and Software Safety Plan.
Validate that safety-critical pathways identified via NASA-STD-8739.8B in the SSP directly map down to structural verification test sets inside the SVVP.
Step 4.2: Automated Dashboard and Matrix Generation
Construct an export utility that reads the final evaluation run states (
Compliance_Statusnodes) out of the graph database.Generate a comprehensive, interactive Requirements Mapping Matrix (RMM) in Markdown or CSV layout, grouping gaps into explicit categories: Major Non-Compliances (NPR/STD direct violations) and Minor Gaps (Missing Handbook suggested content).
Step 4.3: Validation Run & Phase 1 Review
Run a full, end-to-end audit cycle using a set of historical program plans with known compliance profiles.
Refine prompt verbiage and chunking boundaries based on false-positive or false-negative results.
Package the engine code, local graph database, and generated validation dashboards into a formal demonstration release ready for programmatic review and management presentation.
From <https://gemini.google.com/app/> Google Gemini 3.0 (Fast)
Comments
Post a Comment