Documentation That Validates

In Qualityfolio, tests aren't just code—they're assurance artifacts. Every test is Markdown with YAML metadata, versioned in Git, executable automatically, and stored with integrity in structured SQL tables. The same artifact powers documentation, validation execution, and audit-ready evidence with accountability.

These examples show real assurance test structures for regulated industries.

Example 1: Qualityfolio – Examples and Schema Reference

This document provides examples and schema templates for all test artifacts maintained in the Qualityfolio project. Each section aligns with the Qualityfolio Hierarchy Standard: strategy → suite → plan → cycle → case → evidence

1. Strategy

File Path: /qualityfolio/strategy/authentication-strategy/authentication-strategy.md

yaml
---
strategy_id: STR-001
type: test_strategy
title: "Authentication & Login Testing Strategy"
description: "Defines the quality assurance goals, principles, scope, testing types, and tooling strategy for authentication and login functionality in the OWASP GlueUp application. This strategy aligns with OWASP standards and Qualityfolio QA governance."
created_by: "your-email@example.com"
created_at: "2025-10-31"
tags: ["QA strategy", "authentication", "security testing", "OWASP", "functional testing", "automation policy"]
version: "1.0"
status: "Approved"
related_requirements: ["REQ-101", "REQ-102", "REQ-103", "REQ-104"]
---

## 1. Introduction

This **Authentication & Login Testing Strategy** defines the comprehensive approach for validating all authentication modules in the **OWASP GlueUp** application.
It ensures login, credential handling, and session management are tested under functional, non-functional, and security perspectives in compliance with **OWASP guidelines**.

## 2. Testing Goals and Principles

### 2.1 Testing Goals

- Validate authentication and session management functionality across browsers and devices.
- Detect defects related to credential handling, error feedback, and access control.
- Verify compliance with OWASP Authentication and Session Management Cheat Sheets.

### 2.2 Testing Principles

1. **Security First:** Prioritize security over convenience in authentication validation.
2. **Shift-Left QA:** Begin validation at early development stages to reduce risk.
3. **Traceability:** Maintain direct mapping from requirements → plan → cycle → case → evidence.

2. Suite

File Path: /qualityfolio/strategy/authentication-strategy/suites/login-suite/login-suite.md

yaml
---
test_suite_id: TS-001
title: "OWASP GlueUp Login Suite"
type: test_suite
strategy: STR-001
test_plan: TP-001
owner: qa.engineer@example.com
description: "This test suite groups all authentication and login-related test plans for the OWASP GlueUp application."
created_by: "your-email@example.com"
created_at: "2025-10-31"
tags: ["authentication", "login", "security", "OWASP compliance", "test suite"]
version: "1.0"
status: "Active"
---

## 1. Introduction

This test suite consolidates all testing activities related to the **Login and Authentication** functionality of the OWASP GlueUp application.

## 2. Scope

The login suite covers:
- Functional validation of login and redirection flow.
- Negative testing for invalid, missing, or malformed inputs.
- Account lockout and brute-force protection behavior.

3. Test Plan

File Path: /qualityfolio/strategy/authentication-strategy/suites/login-suite/plans/login-plan/login-plan.md

yaml
---
test_plan_id: TP-001
title: "OWASP GlueUp Login Functionality Test Plan"
type: test_plan
parent_id: TS-001
owner: qa.tester@example.com
description: "This test plan defines the approach for validating the login functionality of the OWASP GlueUp application."
created_by: "your-email@example.com"
created_at: "2025-10-31"
tags: ["functional testing", "security testing", "authentication", "login validation"]
version: "1.0"
status: "Active"
---

## 1. Introduction

This test plan outlines the **test approach, objectives, and scope** for verifying the login functionality of the OWASP GlueUp web application.

## 2. Scope of Work

The plan covers the following test areas:
- **Functional Testing:** Successful login and redirection for valid users.
- **Negative Testing:** Validation of invalid, unregistered, and empty input handling.
- **Security & Lockout:** Enforcement of lockout after repeated failed attempts.

4. Test Cycle

File Path: qualityfolio/strategy/authentication-strategy/suites/login-suite/plans/login-plan/cycles/TCY-0001/TCY-0001.cycle.md

yaml
---
test_cycle_id: "TCY-0001"
type: test_cycle
test_plan: "TP-001"
test_suite: "TS-001"
strategy: "STR-001"
title: "Cycle 1 - Functional Login Validation"
description: "Functional and security validation cycle for login functionality on OWASP GlueUp application."
created_by: "your-email@example.com"
created_at: "2025-10-31"
approved_by: "approver-email@example.com"
approved_on: "2025-10-31"
tags: ["Functional", "Authentication", "Security", "UI"]
status: "Active"
version: "1.0"
test_cases: ["TC-001", "TC-002", "TC-003", "TC-004", "TC-005", "TC-006"]
---

### Objective
To validate the complete login process of OWASP GlueUp web application, ensuring functional correctness, input validation, session handling, and account lock behavior.

### Scope
Includes:
- Positive authentication flow
- Error handling for invalid inputs
- UI validation (password masking)
- Account lockout behavior

5. Test Case

File Path: /qualityfolio/strategy/authentication-strategy/suites/login-suite/plans/login-plan/cycles/TCY-0001/cases/TC-001.case.md

yaml
---
test_case_id: "TC-001"
title: "Verify successful login using registered email and correct password"
type: test_case
test_plan: TP-001
created_by: "your-email@example.com"
created_at: "2025-10-30"
approved_by: "approver-email@example.com"
approved_on: "2025-10-30"
test_type: "Manual"
tags: ["Login", "Positive", "Authentication"]
priority: "High"
scenario_type: "happy path"
version: "1.0"
status: "Ready"
---

### Description
Verify that a user can successfully log in using a registered email and the correct password in the OWASP GlueUp application.

### Pre-Conditions
1. User must have a valid registered account on https://owasp.glueup.com/.
2. The account must be active and not locked.

### Test Steps
1. Navigate to https://owasp.glueup.com/account/login.
2. Enter a valid **registered email**.
3. Enter the **correct password**.
4. Click on the **Login** button.

### Expected Result
1. User is successfully authenticated.
2. Redirected to the dashboard or home page.
3. Session remains active until logout or timeout.
4. No error messages displayed.

6. Evidence → Test Case Execution Result

File Path: /qualityfolio/evidence/TCY-0001-TC-001-results-2025-10-31.json

json
{
  "case": "TC-001",
  "cycle": "TCY-0001",
  "title": "Verify successful login using registered email and correct password",
  "status": "passed",
  "start_time": "2025-10-30T03:42:23.787Z",
  "end_time": "2025-10-30T03:42:36.474Z",
  "total_duration": "12.69 seconds",
  "steps": [
    {
      "step": 1,
      "stepname": "Navigate to https://owasp.glueup.com/account/login.",
      "status": "passed",
      "start_time": "2025-10-30T03:42:23.792Z",
      "end_time": "2025-10-30T03:42:36.190Z"
    },
    {
      "step": 2,
      "stepname": "Enter a valid **registered email**.",
      "status": "passed",
      "start_time": "2025-10-30T03:42:36.194Z",
      "end_time": "2025-10-30T03:42:36.286Z"
    }
  ]
}

7. Evidence → Test Run Status

File Path: /qualityfolio/evidence/TCY-0001-TC-001-TR-001.run.md

yaml
---
run_id: "TCY-0001-TC-001-TR-001"
cycle: "TCY-0001"
test_case_id: "TC-001"
run_date: "2025-10-31"
environment: "Staging"
---

### Actual Result

1. User is successfully authenticated.
2. Redirected to the dashboard.
3. Session remains active until logout or timeout.
4. No error messages displayed.

### Run Summary

- Status: Passed
- Notes: All steps were executed successfully.

8. Evidence → Bug

File Path: /qualityfolio/evidence/TCY-0001-TC-003.bug

yaml
---
issue_id: "BUG-TCY-0001-TC-003"
test_case_id: "TC-003"
run_id: "TCY-0001-TC-003-TR-001"
titleName: "Login fails with timeout error even when valid credentials are used"
endpoint: "Login page"
created_by: "your-email@example.com"
created_at: "2025-10-30"
test_type: "Manual"
priority: "High"
severity: "Low"
assigned: "Mahesh"
status: "REJECTED"
---

### Description

During login attempts with valid credentials, the system fails to authenticate the user due to a network timeout.

### Test Steps

Step 1: Launch the application and navigate to the login page.
Step 2: Enter valid credentials and attempt to log in.
Step 3: Simulate network delay or partial disconnection to cause timeout.
Step 4: Observe the system's response.

### Expected Result

1. The system should either retry the request or display a clear "Network Timeout" message.
2. The user should not be shown incorrect "Invalid credentials" error messages.

### Actual Result

1. Login attempt fails with a timeout error.
2. System displays a generic "Login failed" or "Invalid credentials" message.

9. Folder Description Summary

Folder / File Purpose
/strategyHolds all test strategies that define testing goals, scope, and tools.
/authentication-strategyA specific testing strategy related to authentication mechanisms.
/suitesContains grouped test suites under a particular strategy.
/login-suiteSuite dedicated to login and authentication-related plans.
/plans/login-planSpecific test plan containing multiple test cycles.
/cycles/TCY-0001Test cycle representing a single execution batch.
/casesIndividual test case files under each cycle.
/evidenceStores proof of execution — logs, screenshots, results, and reports.
*.json, *.run.md, *.bug.mdEvidence files linked to specific test executions.

Example 2: Assurance Test with Full Metadata

Every assurance test includes YAML frontmatter with validation category, risk level, and requirement traceability for governance and accountability.

markdown
---
test_id: MFA-001
validation_category: computer_system_validation
risk_level: high
requirement_ids:
  - SOC2-CC6.1
  - HIPAA-164.312(a)(2)(i)
regulatory_context:
  - SOC 2 Type II (Access Controls)
  - HIPAA Technical Safeguards
---

# Multi-Factor Authentication Enforcement Test

## Test Objective
Verify that multi-factor authentication (MFA) is required for all user 
access to systems containing PHI/PII, satisfying SOC 2 CC6.1 and 
HIPAA §164.312(a)(2)(i) requirements.

## Setup
- Test user account: test-user@example.com
- System: Production authentication service
- MFA method: TOTP (Time-based One-Time Password)

## Test Procedure

\`\`\`bash
# Step 1: Attempt login with only password
curl -X POST https://api.example.com/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email": "test-user@example.com", "password": "ValidPassword123!"}'

# Step 2: Submit valid MFA code
curl -X POST https://api.example.com/auth/mfa/verify \
  -H "Content-Type: application/json" \
  -d '{"session_id": "$SESSION_ID", "mfa_code": "$TOTP_CODE"}'
\`\`\`

## Pass Criteria
1. Initial login returns HTTP 200 with 
requires_mfa: true
2. Access without MFA verification returns HTTP 403
3. MFA verification with valid code grants session token
4. MFA verification logged with timestamp for accountability

## Evidence Captured
- Authentication attempt timestamp
- MFA challenge issued confirmation
- MFA verification success/failure
- Session creation with MFA flag
- Full request/response logs with integrity

Captured Evidence (stored in SQL with integrity):

json
{
                "test_id": "MFA-001",
                "validation_category": "computer_system_validation",
                "status": "PASSED",
                "requirement_ids": ["SOC2-CC6.1", "HIPAA-164.312(a)(2)(i)"],
                "execution_timestamp": "2025-10-27T14:23:17Z",
                "response_time_ms": 142,
                "mfa_enforced": true,
                "evidence_hash": "sha256:a4f2...d8e1",
                "provenance": {
                  "git_commit": "7f3a21c",
                  "executor": "ci-runner-prod",
                  "environment": "production"
                }
              }

Example 3: Qualityfolio's SQL Evidence Schema

All test execution is captured in structured orchestration tables with integrity and accountability. This open SQL schema lets you query any evidence, anytime, for governance and audit.

sql
-- Main execution record table
            CREATE TABLE orchestration_session (
              orchestration_session_id TEXT PRIMARY KEY,
              device_id TEXT NOT NULL,
              orchestration_nature_id TEXT NOT NULL,
              version TEXT NOT NULL,
              orch_started_at TIMESTAMPTZ NOT NULL,
              orch_finished_at TIMESTAMPTZ,
              diagnostics_json JSONB,
              diagnostics_md TEXT
            );

            -- Individual test execution records
            CREATE TABLE orchestration_session_exec (
              orchestration_session_exec_id TEXT PRIMARY KEY,
              orchestration_session_id TEXT NOT NULL,
              exec_nature TEXT NOT NULL,
              exec_code TEXT NOT NULL,
              exec_status TEXT NOT NULL,
              input_text TEXT,
              output_text TEXT,
              exec_error_text TEXT,
              narrative_md TEXT,
              FOREIGN KEY (orchestration_session_id) 
                REFERENCES orchestration_session(orchestration_session_id)
            );
            

These tables provide immutable records with complete provenance for accountability. Every test execution is timestamped, hashed, and linked to Git commits for governance requirements.

Example 4: Real-World Assurance Query Scenarios

Evidence stored in Qualityfolio's SQL database is queryable with standard SQL. Auditors can self-serve, compliance officers can generate reports on demand, and governance teams can monitor continuous assurance state.

Scenario 1: SOC 2 Auditor Request

"Show me all access control tests executed during Q3 2025 for accountability"

sql
SELECT 
                ose.exec_nature AS test_name,
                ose.exec_status AS status,
                os.orch_started_at AS execution_time,
                ose.narrative_md AS evidence_summary
              FROM orchestration_session_exec ose
              JOIN orchestration_session os 
                ON ose.orchestration_session_id = os.orchestration_session_id
              WHERE ose.exec_nature LIKE '%access-control%'
                AND os.orch_started_at BETWEEN '2025-07-01' AND '2025-09-30'
              ORDER BY os.orch_started_at DESC;

Scenario 2: FDA Validation Report

"Generate validation evidence package for computer system validation submission"

sql
SELECT 
                  json_extract(ose.diagnostics_json, '$.test_id') AS test_id,
                  json_extract(ose.diagnostics_json, '$.validation_category') AS category,
                  ose.exec_status,
                  os.orch_started_at,
                  json_extract(ose.diagnostics_json, '$.requirement_ids') AS requirements,
                  ose.output_text AS evidence
                FROM orchestration_session_exec ose
                JOIN orchestration_session os 
                  ON ose.orchestration_session_id = os.orchestration_session_id
                WHERE json_extract(ose.diagnostics_json, '$.validation_category') = 'computer_system_validation'
                  AND os.orch_finished_at IS NOT NULL
                ORDER BY test_id;

Scenario 3: Identify Flaky Tests (Assurance Risk)

"Find tests with inconsistent results that may impact assurance confidence"

sql
SELECT 
                ose.exec_nature AS test_name,
                COUNT(*) AS total_runs,
                SUM(CASE WHEN ose.exec_status = 'PASSED' THEN 1 ELSE 0 END) AS passes,
                ROUND(100.0 * SUM(CASE WHEN ose.exec_status = 'PASSED' THEN 1 ELSE 0 END) / COUNT(*), 2) AS pass_rate
              FROM orchestration_session_exec ose
              GROUP BY ose.exec_nature
              HAVING COUNT(*) > 5 
                AND pass_rate < 95
              ORDER BY pass_rate ASC;

Quick Start: Assurance-Focused Templates

bash
# 1. Initialize with assurance framework template
qf init --template=soc2        # SOC 2 control tests
# or: qf init --template=fda   # FDA validation protocols
# or: qf init --template=hipaa # HIPAA technical safeguards

# 2. Create a new assurance test
qf create test access-controls/mfa-enforcement.test.md

# 3. Run tests with evidence capture
qf run tests/ --capture-evidence

# 4. Export assurance report for governance
qf export --format=pdf --period=q3-2025 --framework=soc2

Works offline for air-gapped validation environments, in CI/CD pipelines for continuous assurance, or on-demand for audit preparation. Markdown in, verifiable evidence with accountability out.