#4018 Innovation Flow - Python App

#4018 Innovation Flow - Python App

#4018 Analyze and improve the following Python app requirements: """Python application:

1. Read a text file for a sequence of system instructions and user prompts

2. Loop sequentially through the system instructions and user prompts, submitting each to an Ollama-supported local LLM, catenating the previous step response to the user prompt (when applicable)

3. For each result from a step, insert the text file name, sequence step number, system instructions, user prompt with previous response (as applicable), and the response to a database table

4. Provide a full report of all steps"""

This is a solid foundation for an agentic or multi-step reasoning pipeline. Given your background in software process engineering and S&MA, you likely value traceability and error handling as much as the core logic.

I have analyzed your requirements and refined them to include better handling of state, metadata, and reliability.

---

1. Requirement Analysis & Refinements

The primary challenge in your current flow is the context window and state management. As the loop progresses, the "catenated previous response" can grow exponentially if not managed, potentially exceeding the LLM's context limit or diluting the specific instructions.

Proposed Improvements:

• Context Management: Instead of simple concatenation, define a clear "Context Strategy" (e.g., sliding window or summary-based) to keep prompts efficient.

• Database Schema: Ensure the schema includes metadata such as the model name, temperature, and timestamp for reproducibility.

• Error Handling: Local LLMs (Ollama) can occasionally timeout or fail; the app needs a retry mechanism or a "save-state" to resume a failed sequence.

• Schema Validation: Use a structured format (like JSON or YAML) for your input text file to clearly distinguish between system roles and user prompts.

---

2. Improved Requirements Specification

R1: Input Parsing & Structure

• R1.1: The application shall ingest a structured input file (YAML/JSON preferred) containing a list of objects, each defining a system_instruction and a user_prompt.

• R1.2: The application shall support a placeholder syntax (e.g., {{last_response}}) within user prompts to define exactly where the previous output should be injected.

R2: LLM Interaction (Ollama API)

• R2.1: The app shall interface with the Ollama local API, allowing the user to specify the target model (e.g., llama3, mistral).

• R2.2: The app shall implement a sequential loop where the output of step $n$ is held in memory to be provided to step $n+1$.

R3: Data Persistence (Traceability)

• R3.1: For every execution step, the following record shall be persisted to a relational database (SQLite/PostgreSQL):

  • run_id: Unique UUID for the entire session.

  • step_index: The sequence number.

  • input_file_source: Filename.

  • model_version: Exact Ollama model string used.

  • full_prompt: The final rendered prompt sent to the LLM.

  • response_text: The raw output from the LLM.

  • latency: Time taken for the LLM to respond.

R4: Reporting & Output

• R4.1: The application shall generate a final report (Markdown or HTML) summarizing the "chain of thought," showing the evolution of the response across all steps.

• R4.2: The report shall highlight any steps that exceeded a predefined response time threshold.

---

3. Recommended Technical Stack

Component	Recommendation
Orchestration	Python 3.10+
LLM Interface	ollama-python library
Database	SQLite (via SQLAlchemy for easy migration to Postgres later)
Config/Input	PyYAML
CLI/Reports	Rich (for beautiful terminal output) or Pandas (for CSV export)

---

4. Logical Data Flow

graph TD
    A[Input YAML/Text File] --> B[Python Orchestrator]
    B --> C{Step Loop}
    C --> D[Construct Prompt + Prev. Response]
    D --> E[Ollama Local LLM]
    E --> F[Capture Response]
    F --> G[(SQLite Database)]
    G --> H{More Steps?}
    H -- Yes --> C
    H -- No --> I[Generate Final Report]

---

Next Step

Would you like me to provide a Python starter script using the ollama and sqlite3 libraries that implements this refined sequential logic?

Yes

Since you’re likely looking for a robust, traceable "software process" approach rather than just a quick script, I've designed this using SQLite for the database and the Official Ollama Python library.

I have implemented a Session ID system. This allows you to run multiple tests and keep them distinct in your database—critical for Safety and Mission Assurance (S&MA) style documentation and auditing.

Python Implementation: Sequential LLM Pipeline

import sqlite3
import time
import uuid
from ollama import Client

# --- Configuration ---
DB_NAME = "llm_traceability.db"
OLLAMA_MODEL = "llama3"  # Change to your preferred local model
INPUT_FILE = "instructions.txt"
CLIENT = Client(host='http://localhost:11434')

def setup_database():
    """Initializes the SQLite database with a traceability-focused schema."""
    conn = sqlite3.connect(DB_NAME)
    cursor = conn.cursor()
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS llm_logs (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            session_id TEXT,
            step_number INTEGER,
            file_name TEXT,
            system_instruction TEXT,
            user_prompt TEXT,
            full_context_sent TEXT,
            response TEXT,
            timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
        )
    ''')
    conn.commit()
    return conn

def run_pipeline():
    conn = setup_database()
    cursor = conn.cursor()
    session_id = str(uuid.uuid4())[:8]  # Unique ID for this specific run
    
    # 1. Read the text file
    # Format: Assumes System and User prompt on alternating lines or specific delimiter
    # For this example, we assume a simple 'System|User' pipe-delimited text file
    try:
        with open(INPUT_FILE, 'r') as f:
            lines = [line.strip() for line in f if line.strip()]
    except FileNotFoundError:
        print(f"Error: {INPUT_FILE} not found.")
        return

    previous_response = ""
    
    print(f"--- Starting Session: {session_id} ---")

    # 2. Loop sequentially through instructions
    for idx, line in enumerate(lines, start=1):
        # Splitting logic: "System Instruction | User Prompt"
        parts = line.split('|')
        system_content = parts[0].strip()
        user_base_prompt = parts[1].strip() if len(parts) > 1 else ""

        # Catenate previous response to user prompt if applicable
        current_user_prompt = f"{user_base_prompt}\n\nContext from previous step: {previous_response}".strip()

        print(f"Processing Step {idx}...")

        # 3. Submit to Ollama
        response = CLIENT.chat(model=OLLAMA_MODEL, messages=[
            {'role': 'system', 'content': system_content},
            {'role': 'user', 'content': current_user_prompt},
        ])

        llm_output = response['message']['content']

        # 4. Insert into Database
        cursor.execute('''
            INSERT INTO llm_logs 
            (session_id, step_number, file_name, system_instruction, user_prompt, full_context_sent, response)
            VALUES (?, ?, ?, ?, ?, ?, ?)
        ''', (session_id, idx, INPUT_FILE, system_content, user_base_prompt, current_user_prompt, llm_output))
        
        conn.commit()
        
        # Update state for next iteration
        previous_response = llm_output

    print(f"--- Pipeline Complete. Results saved to {DB_NAME} ---")
    generate_report(session_id, conn)
    conn.close()

def generate_report(session_id, conn):
    """Provides a full report of all steps for the specific session."""
    print(f"\n{'='*20} SESSION REPORT: {session_id} {'='*20}")
    cursor = conn.cursor()
    cursor.execute("SELECT step_number, response FROM llm_logs WHERE session_id = ? ORDER BY step_number", (session_id,))
    
    rows = cursor.fetchall()
    for row in rows:
        print(f"\nSTEP {row[0]}:")
        print(f"RESULT: {row[1][:200]}...") # Print first 200 chars for brevity
    print(f"\n{'='*60}")

if __name__ == "__main__":
    run_pipeline()

---

Key Features of this Script:

• Traceability: It logs the full_context_sent. This is vital because if the LLM hallucinates, you can look at the database to see exactly what prompt caused it (including the injected previous response).

• Session Management: Every time you run the script, it gets a new session_id. This allows you to compare different runs of the same instruction file to see how the LLM's "reasoning" changes.

• Structured Reporting: It ends with a summary but keeps the raw data in llm_traceability.db, which you can open with any SQLite browser to perform deeper analysis or export to Excel for NASA-style reporting.

To use this:

1. Install requirements: pip install ollama

2. Create instructions.txt: Create a file where each line looks like: Analyze the software requirements | Identify the safety-critical components Refine the components found | List potential hazards for these specific components

3. Run the script: Ensure Ollama is running in the background.

Would you like me to help you design a more complex SQL schema that handles "Hazard ID" and "Mitigation" mapping specifically for your S&MA work?

From <https://gemini.google.com/app/52ce6dd7b13f88d3>  Google Gemini 3.0 Pro (Thinking)