#4131 NASA Surveillance of Launch Service Provider Software Development

#4131 Discuss NASA program-level and project-level surveillance (insight/oversight) of launch service providers software development

In the context of NASA’s Safety & Mission Assurance (S&MA) and Engineering technical authorities, the approach to monitoring launch service providers—particularly commercial ones like SpaceX, Blue Origin, or ULA—shifted significantly with the move toward Commercial Resupply Services (CRS) and Commercial Crew Programs (CCP).

The strategy balances Oversight (a more traditional, prescriptive "tell me what you're doing and let me approve it" approach) and Insight (a "show me how you do it so I can trust the result" approach).

1. Program-Level Surveillance: Strategy and Risk

At the program level (e.g., Launch Services Program (LSP) or Moon to Mars), NASA establishes the Surveillance Plan. This level is less about individual lines of code and more about the "Contractor’s Process Capability."

Requirements Baseline: The program defines which NASA standards apply (e.g., NASA-STD-8739.8A for Software Assurance or NASA-STD-5001 for structural design). For commercial providers, NASA often allows the use of "equivalent" industry standards (like DO-178C or IEEE/ISO 12207), provided the provider can prove the equivalence.
Risk-Based Surveillance: NASA doesn't look at everything. Programs use a risk-informed process to decide where to focus. If a provider has a flawless record with flight software but a spotty one with ground control systems, the surveillance shifts toward the latter.
Safety & Mission Assurance (S&MA) Audits: The program conducts "Deep Dives" into the provider’s Quality Management System (QMS). They verify that the provider’s software development plan (SDP) isn't just "shelf-ware" but is actually being followed by the developers.

2. Project-Level Surveillance: Tactical Execution

At the project level, NASA’s "Sub-system Managers" or "Software Assurance Engineers" (often subcontractors) get into the weeds of the development lifecycle.

Insight Activities (Monitoring)

Participation in Milestones: NASA engineers attend SRRs (System Requirements Reviews), PDRs (Preliminary Design Reviews), and CDRs (Critical Design Reviews). At the project level, the goal is to ensure that software requirements are traceable to system safety requirements.
Code & Test Data Access: Under "Insight," NASA may not "approve" every line of code, but they maintain the right to inspect it. This includes reviewing unit test results, static analysis reports, and "Hardware-in-the-loop" (HWIL) testing outcomes.
Shadowing: NASA personnel might "shadow" provider testing sessions to ensure that "test-as-you-fly" principles are being upheld.

Oversight Activities (Approval)

Criticality Analysis: NASA project-level staff verify the provider’s Software Classification. If a piece of software is deemed "Class A" (Human Rated/Mission Critical), NASA exercises much tighter oversight, often requiring formal approval of the Verification & Validation (V&V) plan.
Non-Conformance Reporting: If a provider has a "software escape" (a bug found late in the cycle), NASA project-level surveillance ensures the root cause analysis (RCA) is rigorous and that the fix doesn't introduce regression errors in other systems.

3. The "Insight vs. Oversight" Tension

The modern NASA-Provider relationship is often defined by this table of responsibilities:

Feature	Oversight (Traditional)	Insight (Commercial)
Authority	NASA approves every milestone.	NASA monitors provider milestones.
Standards	NASA-SPEC-XXXX is mandatory.	Provider standards are "vetted" by NASA.
Artifacts	NASA owns the documentation.	NASA has "viewing rights" to data.
Accountability	NASA shares high technical risk.	Provider owns the risk; NASA "accepts" it.

4. Specific Software Concerns in Launch Services

Surveillance currently focuses heavily on three high-risk areas:

Autonomous Flight Termination Systems (AFTS): Since these protect the public, the software logic that decides to "destruct" a vehicle is subject to the highest level of NASA and FAA oversight.
Regression Testing in Agile Environments: Commercial providers iterate quickly. NASA surveillance must ensure that a "quick fix" in a sprint doesn't break a safety-critical constraint established months prior.
Cybersecurity & Supply Chain: Ensuring that the software development environment (the "pipeline") is secure from injection attacks or unauthorized changes.

Would you like to dive deeper into how NASA-STD-8739.8A specifically governs the software assurance (SA) requirements for these providers?

Describe the outline and contents of a NASA Program-level Surveillance Plan of a Launch Provider Software Development

A NASA Program-Level Surveillance Plan (PLSP) for a Launch Service Provider (LSP) serves as the "Rule of Engagement" between the government and the commercial entity. Unlike a project-level plan that focuses on specific code modules, the Program-level plan defines the governance, risk-tolerance, and data-access rights required to ensure mission success.

Below is a standard outline and the expected contents for such a plan, reflecting the modern shift toward Insight-driven surveillance.

1. Introduction and Scope

This section establishes the legal and technical boundaries of NASA’s reach into the provider’s internal processes.

Purpose: Defines the "Why"—balancing the provider’s proprietary development speed with NASA’s "Safety and Mission Success" mandate.
Surveillance Philosophy: Explicitly states the ratio of Insight (monitoring/witnessing) vs. Oversight (formal approval/disapproval).
Applicability: Lists the specific launch vehicle families and software suites (Flight, Ground, Support) covered by the plan.

2. Governance and Authority

Technical Authorities (TA): Defines the roles of the NASA Engineering and S&MA Technical Authorities.
Surveillance Organization: Identifies the "Insight Team," including NASA Civil Servants and S&MA Subcontractors (who often perform the day-to-day data reviews).
Access Rights: Specifies NASA's right to access the provider’s internal tools (e.g., Jira, Confluence, Bitbucket, or Gitlab) and physical facilities for audits.

3. Applicable Standards and Equivalence

Because commercial providers often use industry standards rather than NASA-specific ones, this section is critical.

Baseline Standards: Typically references NASA-STD-8739.8A (Software Assurance) and NASA-STD-7009A (Models and Simulations).
Equivalence Matrix: A table mapping the provider’s internal processes (e.g., "SpaceX Software Quality Manual") to NASA requirements to identify "gaps" that require extra surveillance.

4. Risk-Based Surveillance (RBS) Methodology

NASA does not have the resources to watch every "sprint." This section explains how they prioritize.

Criticality Analysis: Definition of Software Class (Class A for Human-Rated, Class B for High-Value Science).
Surveillance Frequency: Based on the provider's "Process Maturity." A new provider gets high-frequency surveillance; a legacy provider with a "stable" flight software baseline gets periodic "delta" reviews.
KPIs and Triggers: Defines what events trigger an "Out-of-Cycle" audit (e.g., a software-related test failure or a major architecture change).

5. Insight and Oversight Activities

This is the "meat" of the document, broken down by the Software Development Life Cycle (SDLC).

SDLC Phase	Surveillance Activity	NASA Role
Requirements	Verification of bidirectional traceability between System Safety and Software Requirements.	Insight: Participate in Technical Exchange Meetings (TEMs).
Design	Review of architectural patterns (e.g., partition of safety-critical vs. non-critical code).	Insight: Review Design Documents / CDR packages.
Implementation	Verification of Code Standards and Static Analysis results.	Insight: Spot-check SonarQube/Coverity reports.
Test/V&V	Witnessing "Hardware-in-the-Loop" (HWIL) and "Day-of-Launch" simulations.	Oversight: Formal approval of the V&V Report for "Class A" software.
Deployment	Review of Version Description Documents (VDD) and configuration hashes.	Oversight: Sign-off on the Flight Readiness Review (FRR) software action items.

6. Non-Conformance and Issue Resolution

Material Review Board (MRB) Participation: Defines NASA’s role in the provider’s internal bug-tracking and anomaly resolution process.
Root Cause Analysis (RCA): NASA’s requirement to witness the RCA for any "Software Escape" that reaches the launch pad or flight.
Dissenting Opinions: The formal process for a NASA engineer to "Non-Concur" with a provider’s technical decision.

7. Configuration Management (CM) and Data Deliverables

Data Requirements List (DRL): A list of what the provider must send to NASA (e.g., Software Version Descriptions, Test Plans).
Data Access List (DAL): A list of what NASA can look at but doesn't necessarily "keep" (e.g., proprietary source code).

Key Focus for NASA S&MA Subcontractors

In your role, the "Surveillance Plan" is your primary lever. You are looking for the "Quality Shadow"—evidence that the provider's actual daily output matches the high-level plans they showed during the proposal phase.

Would you like me to draft a sample "Insight Checklist" specifically for the transition from Software Design (CDR) to System Integration Testing (SIT)?

Yes

Since you are operating at the interface of NASA S&MA and commercial launch providers, this checklist focuses on the high-risk transition where "paper design" meets "real hardware."

At this stage, your goal is to verify that the theoretical safety controls identified during the Critical Design Review (CDR) have actually been implemented and are ready for System Integration Testing (SIT).

NASA S&MA Insight Checklist: CDR to SIT Transition

Target: Launch Service Provider (LSP) Flight & Ground Software

Focus: Verification of Safety-Criticality and Integration Readiness

1. Requirement & Design Traceability (The "Paper to Code" Link)

[ ] Bidirectional Traceability: Verify that 100% of "Safety-Critical" requirements identified in the CDR are mapped to specific software modules and SIT test cases.
[ ] Architectural Partitioning: Confirm that the "Firewall" between Class A (Flight Critical) and Class C (Non-Critical/Telemetry) software is physically or logically enforced as designed.
[ ] Design Change Log: Review all "Design Deviations" occurred since CDR. Have any changes impacted the Hazard Reports (HRs) or Failure Modes and Effects Analysis (FMEA)?

2. Static Analysis & Code Quality (The "Pre-SIT" Gate)

[ ] Static Analysis Reports: Review outputs from tools (e.g., Coverity, Polyspace, or SonarQube). Are there any unaddressed "High" or "Critical" findings in the flight-critical path?
[ ] Compiler Warning Audit: Ensure the provider has a "Zero Warning" policy for flight builds, or documented justifications for every suppressed warning.
[ ] Complexity Metrics: Identify modules with high Cyclomatic Complexity. These are prime candidates for NASA "Deep Dive" code walkthroughs before they hit the integration lab.

3. Test Environment & HWIL Readiness

[ ] Fidelity Verification: Does the SIT environment (Hardware-in-the-Loop / HWIL) use flight-equivalent processors and sensors? If using emulators, has the provider validated that the emulator timing matches flight hardware?
[ ] Configuration Baseline: Verify the "Version Description Document" (VDD). Is the software version under test exactly what was branched from the CDR-approved baseline?
[ ] Test Script Validation: Review the automated test scripts. Do they actually test the requirement, or just the "happy path"? (Look for "off-nominal" and "boundary value" coverage).

4. Safety-Critical Functionality (The "Showstoppers")

[ ] Autonomous Flight Termination (AFTS): If applicable, witness a standalone "Dry Run" of the AFTS logic. Verify that no "non-safety" software can interrupt the termination sequence.
[ ] Fault Management (FM): Confirm the "Redline" logic. If a sensor fails during SIT, does the software correctly trigger the Safing or Abort sequence as defined in the Design?
[ ] Command & Control Integrity: Verify that "Critical Commands" (e.g., Engine Ignition, Stage Sep) require a multi-step "Arm/Fire" sequence and cannot be triggered by a single bit-flip.

5. Process & "Quality Shadow" Evidence

[ ] Peer Review Minutes: Spot-check the provider’s internal code review records. Are they finding bugs, or are they "rubber-stamping" code?
[ ] Non-Conformance (NC) Trends: Look at the bug-tracking system (e.g., Jira). Is the "Open Bug" count increasing as SIT approaches? (A massive spike may indicate the design wasn't as mature as claimed at CDR).
[ ] Cybersecurity/Supply Chain: Confirm that all 3rd-party libraries or "Legacy Code" used in the build have been scanned for vulnerabilities.

S&MA "Deep Dive" Tip

If you notice a provider is rushing into SIT to meet a milestone payment, look specifically at their Regression Test Suite. Often, they will "trim" the regression tests to save time. As an S&MA subcontractor, your "Value-Add" is insisting that the high-risk safety paths are re-verified after every minor "tweak" in the integration lab.

Would you like me to generate a template for a "NASA S&MA Surveillance Report" (SSR) that you could use to document these findings for the Program Office?

From <https://gemini.google.com/app/> Google Gemini 3.0 Pro (Thinking)

Search This Blog

Learning from ChatGPT