#3095 Improve Document Compliance Check Instructions

-

#3095 Improve Document Compliance Check Instructions

#3095 Assess the following instructions to an LLM: """<TASK SUMMARY>Assess Compliance of DOCUMENT_A to REQUIREMENTS_B</TASK SUMMARY> <INSTRUCTIONS> <STEPS> <STEP id="0">Determine whether DOCUMENT_A fully complies, partially complies, or is non-compliant to REQUIREMENT_B.</STEP> <STEP id="1">Output the following when DOCUMENT_A fully complies with the requirement: a. The exact text of the requirement. b. The section(s) of DOCUMENT_A that demonstrate full compliance.</STEP> <STEP id="2">Output the following when DOCUMENT_A partially complies with the requirement: a. The exact text of the requirement. b. The section(s) of DOCUMENT_A that partially fulfill the requirement.</STEP> <STEP id="3">Output the following when DOCUMENT_A does not comply with the requirement: a. The exact text of the requirement. b. A commentary explaining why DOCUMENT_A does not meet the requirement.</STEP> </STEPS> </INSTRUCTIONS>""" --- the document and requirement will be provided by the user. Provided improved instructions that I can submit to the LLM.

Below is an improved and clarified version of the instructions that you can submit to the LLM:

Task:
Evaluate whether a provided document (DOCUMENT_A) complies with a given requirement (REQUIREMENT_B).

Overall Objective:
Determine if DOCUMENT_A:

  • Fully complies with REQUIREMENT_B,

  • Partially complies with REQUIREMENT_B, or

  • Does not comply with REQUIREMENT_B.

Instructions:

  1. Assess Compliance:

    • Read both DOCUMENT_A and REQUIREMENT_B carefully.

    • Decide on one of the three compliance statuses:

      • Full Compliance: DOCUMENT_A completely meets every aspect of REQUIREMENT_B.

      • Partial Compliance: DOCUMENT_A meets some parts of REQUIREMENT_B, but there are identifiable gaps.

      • Non-Compliance: DOCUMENT_A does not meet the requirements outlined in REQUIREMENT_B.

  2. Output Format Based on Determination:

    • If DOCUMENT_A Fully Complies with REQUIREMENT_B:

      • Output the exact text of REQUIREMENT_B.

      • List the section(s) of DOCUMENT_A that clearly demonstrate full compliance.

    • If DOCUMENT_A Partially Complies with REQUIREMENT_B:

      • Output the exact text of REQUIREMENT_B.

      • List the section(s) of DOCUMENT_A that partially fulfill the requirement.

      • Indicate which specific aspects or components of the requirement are not completely addressed.

    • If DOCUMENT_A Does Not Comply with REQUIREMENT_B:

      • Output the exact text of REQUIREMENT_B.

      • Provide a commentary that explains why DOCUMENT_A fails to meet the requirement.

        • This commentary should reference specific sections or omissions in DOCUMENT_A and compare them directly against REQUIREMENT_B.

  3. Additional Guidance:

    • Clarity and Structure: Clearly label each part of your output (e.g., "Compliance Status," "Requirement Text," and "Supporting Details").

    • Referencing: When referring to sections of DOCUMENT_A, use section numbers or headings if available to support your evaluation.

    • Objectivity: Base your evaluation on direct comparisons between the texts of DOCUMENT_A and REQUIREMENT_B.

    • Consistency: Ensure that your output follows the same structure regardless of the compliance level determined.

Example Output Structure:

Compliance Status: [Full / Partial / Non-Compliant]

Requirement Text:
"[Exact text of REQUIREMENT_B]"

Supporting Details:
- [If Fully Compliant] Section X of DOCUMENT_A states: "..."
- [If Partially Compliant] Section Y of DOCUMENT_A partially addresses the requirement; however, it omits details regarding ...
- [If Non-Compliant] DOCUMENT_A does not address the following aspects: ... (provide detailed commentary)

These improved instructions provide a clear, structured framework for evaluating DOCUMENT_A against REQUIREMENT_B and ensure that the output is consistent and well-organized.


Rewrite the improved instructions as XML
<AssessmentInstructions>
    <Task>
        Evaluate whether a provided document (DOCUMENT_A) complies with a given requirement (REQUIREMENT_B).
    </Task>
    <OverallObjective>
        <Status type="full">
            DOCUMENT_A fully meets every aspect of REQUIREMENT_B.
        </Status>
        <Status type="partial">
            DOCUMENT_A meets some parts of REQUIREMENT_B but has identifiable gaps.
        </Status>
        <Status type="none">
            DOCUMENT_A does not meet the requirements outlined in REQUIREMENT_B.
        </Status>
    </OverallObjective>
    <Steps>
        <Step id="1">
            <Title>Assess Compliance</Title>
            <Details>
                <Instruction>
                    Read both DOCUMENT_A and REQUIREMENT_B carefully.
                </Instruction>
                <Instruction>
                    Determine the compliance status: full, partial, or non-compliant.
                </Instruction>
            </Details>
        </Step>
        <Step id="2">
            <Title>Output Format Based on Compliance Status</Title>
            <Substeps>
                <Substep compliance="full">
                    <Title>If Fully Compliant</Title>
                    <Instruction>
                        Output the exact text of REQUIREMENT_B.
                    </Instruction>
                    <Instruction>
                        List the section(s) of DOCUMENT_A that clearly demonstrate full compliance.
                    </Instruction>
                </Substep>
                <Substep compliance="partial">
                    <Title>If Partially Compliant</Title>
                    <Instruction>
                        Output the exact text of REQUIREMENT_B.
                    </Instruction>
                    <Instruction>
                        List the section(s) of DOCUMENT_A that partially fulfill the requirement.
                    </Instruction>
                    <Instruction>
                        Indicate which specific aspects or components of the requirement are not completely addressed.
                    </Instruction>
                </Substep>
                <Substep compliance="none">
                    <Title>If Non-Compliant</Title>
                    <Instruction>
                        Output the exact text of REQUIREMENT_B.
                    </Instruction>
                    <Instruction>
                        Provide a commentary explaining why DOCUMENT_A fails to meet the requirement.
                    </Instruction>
                    <Instruction>
                        Reference specific sections or omissions in DOCUMENT_A to support the commentary.
                    </Instruction>
                </Substep>
            </Substeps>
        </Step>
        <Step id="3">
            <Title>Formatting and Additional Guidance</Title>
            <Details>
                <Instruction>
                    Clearly label each part of your output (e.g., "Compliance Status", "Requirement Text", and "Supporting Details").
                </Instruction>
                <Instruction>
                    Use section numbers or headings from DOCUMENT_A when referring to specific sections.
                </Instruction>
                <Instruction>
                    Ensure your evaluation is objective and based on direct comparisons between DOCUMENT_A and REQUIREMENT_B.
                </Instruction>
                <Instruction>
                    Maintain consistency in the output structure regardless of the compliance level determined.
                </Instruction>
            </Details>
        </Step>
    </Steps>
    <ExampleOutput>
        <Structure>
            <Line>Compliance Status: [Full / Partial / Non-Compliant]</Line>
            <Line>Requirement Text: "[Exact text of REQUIREMENT_B]"</Line>
            <Line>
                Supporting Details:
                <Detail>If Fully Compliant: Section X of DOCUMENT_A states: "..."</Detail>
                <Detail>If Partially Compliant: Section Y of DOCUMENT_A partially addresses the requirement; however, it omits details regarding ...</Detail>
                <Detail>If Non-Compliant: DOCUMENT_A does not address the following aspects: ... (provide detailed commentary)</Detail>
            </Line>
        </Structure>
    </ExampleOutput>
</AssessmentInstructions>


For each requirement submitted by the user, I need a statement by the LLM that the document is "FULL COMPLIANCE", "PARTIAL COMPLIANCE", or "NON-COMPLIANCE" to the requirement, and why.

<AssessmentInstructions>

    <Task>

        Evaluate whether a provided document (DOCUMENT_A) complies with each given requirement submitted by the user.

    </Task>

    <OverallObjective>

        <Status type="full">

            DOCUMENT_A fully meets every aspect of the requirement.

        </Status>

        <Status type="partial">

            DOCUMENT_A meets some parts of the requirement but has identifiable gaps.

        </Status>

        <Status type="none">

            DOCUMENT_A does not meet the requirement.

        </Status>

    </OverallObjective>

    <Steps>

        <Step id="1">

            <Title>Assess Each Requirement</Title>

            <Details>

                <Instruction>

                    For every requirement provided by the user, read the text carefully and assess DOCUMENT_A.

                </Instruction>

                <Instruction>

                    Determine the compliance status for each requirement as either FULL COMPLIANCE, PARTIAL COMPLIANCE, or NON-COMPLIANCE.

                </Instruction>

            </Details>

        </Step>

        <Step id="2">

            <Title>Output a Compliance Statement</Title>

            <Details>

                <Instruction>

                    For each requirement, output a clear statement indicating the compliance status: "FULL COMPLIANCE", "PARTIAL COMPLIANCE", or "NON-COMPLIANCE".

                </Instruction>

                <Instruction>

                    Provide an explanation detailing why DOCUMENT_A meets, partially meets, or does not meet the requirement.

                </Instruction>

            </Details>

        </Step>

        <Step id="3">

            <Title>Detailing the Evaluation</Title>

            <Substeps>

                <Substep compliance="full">

                    <Title>If Fully Compliant</Title>

                    <Instruction>

                        Output the exact text of the requirement.

                    </Instruction>

                    <Instruction>

                        Include the section(s) of DOCUMENT_A that clearly demonstrate full compliance.

                    </Instruction>

                    <Instruction>

                        Conclude with a statement: "FULL COMPLIANCE" along with a justification.

                    </Instruction>

                </Substep>

                <Substep compliance="partial">

                    <Title>If Partially Compliant</Title>

                    <Instruction>

                        Output the exact text of the requirement.

                    </Instruction>

                    <Instruction>

                        List the section(s) of DOCUMENT_A that partially fulfill the requirement.

                    </Instruction>

                    <Instruction>

                        Identify which aspects are not fully addressed.

                    </Instruction>

                    <Instruction>

                        Conclude with a statement: "PARTIAL COMPLIANCE" along with a detailed explanation.

                    </Instruction>

                </Substep>

                <Substep compliance="none">

                    <Title>If Non-Compliant</Title>

                    <Instruction>

                        Output the exact text of the requirement.

                    </Instruction>

                    <Instruction>

                        Provide a commentary explaining why DOCUMENT_A fails to meet the requirement.

                    </Instruction>

                    <Instruction>

                        Conclude with a statement: "NON-COMPLIANCE" along with a rationale.

                    </Instruction>

                </Substep>

            </Substeps>

        </Step>

        <Step id="4">

            <Title>Formatting and Consistency</Title>

            <Details>

                <Instruction>

                    Clearly label each part of your output (e.g., "Compliance Statement", "Requirement Text", and "Evaluation Details").

                </Instruction>

                <Instruction>

                    Use section numbers or headings from DOCUMENT_A when referencing specific sections.

                </Instruction>

                <Instruction>

                    Ensure the evaluation is objective and based on direct comparisons between DOCUMENT_A and the requirement.

                </Instruction>

                <Instruction>

                    Maintain a consistent output structure for every requirement evaluated.

                </Instruction>

            </Details>

        </Step>

    </Steps>

    <ExampleOutput>

        <Structure>

            <Line>Compliance Statement: FULL COMPLIANCE / PARTIAL COMPLIANCE / NON-COMPLIANCE</Line>

            <Line>Requirement Text: "[Exact text of the requirement]"</Line>

            <Line>

                Evaluation Details:

                <Detail>If Fully Compliant: Section X of DOCUMENT_A states: "..." — therefore, FULL COMPLIANCE because ...</Detail>

                <Detail>If Partially Compliant: Section Y of DOCUMENT_A addresses part of the requirement, but it omits details regarding ... — therefore, PARTIAL COMPLIANCE because ...</Detail>

                <Detail>If Non-Compliant: DOCUMENT_A does not address the following aspects: ... — therefore, NON-COMPLIANCE because ...</Detail>

            </Line>

        </Structure>

    </ExampleOutput>

</AssessmentInstructions>


Simplify these instructions so that the LLM will not get confused. The key point is to determine and state clearly "FULLY COMPLIANT", "PARTIALLY COMPIANT" or "NON-COMPLIANT". Also quote the document text supporting the conclusion.
<Instructions>
    <Task>
        For each requirement provided, determine if DOCUMENT_A is:
        - FULLY COMPLIANT
        - PARTIALLY COMPLIANT
        - NON-COMPLIANT
    </Task>
    <Steps>
        <Step id="1">
            Read the requirement and DOCUMENT_A carefully.
        </Step>
        <Step id="2">
            Decide which compliance status applies:
            <Status>FULLY COMPLIANT</Status>
            <Status>PARTIALLY COMPLIANT</Status>
            <Status>NON-COMPLIANT</Status>
        </Step>
        <Step id="3">
            Output the following:
            <Item>The exact text of the requirement.</Item>
            <Item>A clear statement: "FULLY COMPLIANT", "PARTIALLY COMPLIANT", or "NON-COMPLIANT".</Item>
            <Item>Quoted text from DOCUMENT_A that supports your conclusion.</Item>
        </Step>
    </Steps>
</Instructions>

From <https://chatgpt.com/>   ChatGPT o3-mini

Comments

Post a Comment

Popular posts from this blog

#2892 Example of ChatGPT o1 Prompt

-
#2892 Example of ChatGPT o1 Prompt #2892 Consider this prompt example from a tutorial:  """The Anatomy of an o1 Prompt Goal: I want a list of the best medium-length hikes within two hours of San Francisco. Each hike should provide an unique adventure, and be lesser know. Return Format: For each hike, return the name of the hike as I'd find it on AllTrails, then provide the starting address of the hike, the ending address of the hike, distance, drive time, hike duration, and what makes it a cool and unique adventure. Warnings: Be careful to make sure the name of the trail is correct, that it actually exists, and that the time is correct! Context Dump: For context: my girlfriend and I hike a ton! We've don pretty much all of the local SF hikes, whether that's Presidio or Golden Gate Park. we definitely want to get out of town -- we did mount tam pretty recently, the whole thing from the beginning of the stairs to Stinson -- it was really long and we are def...
Read more

#3105 Federal and State Spending Idiocy

-
#3105 Federal and State Spending Idiocy A historic highway overpass for animals cost nearly $100 million and is almost done https://www.yahoo.com/news/historic-highway-overpass-animals-cost-100115264.html #3105 Considering that a state is building a 100-foot wide grass covered overpass across 6 lanes of interstate and shoulders, describe all budget items and the total budget for this task. Reasoned about overpass construction budgeting for a few seconds I'm mapping out a budget for a 100-foot wide grass-covered overpass. Categories include design, permitting, site preparation, structural construction, landscaping, signage, safety features, environmental mitigation, traffic management, and contingency, totaling an approximate $20-30 million. Below is a sample, high-level breakdown of typical budget items for constructing a 100-foot-wide, grass-covered overpass across six interstate lanes (plus shoulders). Actual costs vary significantly depending on location, terrain, materials, lab...
Read more

#3106 Google Gemini Advanced 2.0 Flash: NASA Software Process Requirements Checklist Template

-
#3106 Google Gemini Advanced 2.0 Flash: NASA Software Process Requirements Checklist Template #3106 Provide a template for a NASA software process requirements checklist *NASA Software Process Requirements Checklist** **Project Name:** _________________________ **Project Number:** _________________________ **Software Name/Component:** _________________________ **Version/Release:** _________________________ **Prepared By:** _________________________ **Date:** _________________________ **Applicable NASA Standards/Policies:** (e.g., NASA-STD-8739.8, NPR 7150.2, etc.) **Instructions:** For each requirement, indicate compliance with "Yes," "No," "Partial," or "N/A" (Not Applicable). Provide comments and evidence (e.g., document references, tool outputs) to support the assessment. **I. Software Development Lifecycle (SDLC) Processes** | **Requirement Area** | **Requirement Description** | **Compliance (Yes/No/Partial/N/A)** | **Comments/Evidence**...
Read more