Principles for Gate and Acceptance Scenarios : Biligent#10

Overview of Biligent

Quality & Verification Section 10: Testing & Acceptance Definition of Done 16 min read

Biligent Quality Architecture: Multi-Layer Testing, 6 Principle Gates & 10 Final Acceptance Scenarios (DP-10)

How to guarantee zero-defect autonomous collaboration: Multi-tier testing strategies, strict architectural invariants, automated wiring audits, 6 non-negotiable Principle Gates, and the canonical Definition of Done across 10 end-to-end acceptance scenarios (S1 ~ S10).

🐺 Flawless Synchronization Under Pressure: A pack of painted wolves thrives because every maneuver is checked, cross-validated, and executed with zero room for error. The Biligent Quality Architecture establishes this exact standard through automated invariants, AST wiring audits, and strict multi-backend verification gates.

Table of Contents

• 10-1. Multi-Layer Testing Architecture
• 10-2. The 4 Core Architectural Invariants
• 10-3. Automated Wiring & Dead-Code Inspection
• 10-4. The 6 Cardinal Principle Gates
• 10-5. Definition of Done: 10 Final Acceptance Scenarios (S1 ~ S10)

10-1 Multi-Layer Testing Architecture

Testing in Biligent is segregated into distinct layers to isolate deterministic calculation, inter-agent contracts, fault resilience, and real-environment execution.

Testing Tier	Verification Methodology & Scope
Unit Testing	Pure business rules (slot validation, scoring functions, guard verifications, transition tables, context inheritance, RBAC evaluation) and agent decision logic tested using component test doubles.
Contract Testing	Public function I/O schemas, exception formats, request envelope payload schemas, all 4 result status branches (ok, partial, needs_user, failed), and deadline timeout handling.
Independence Testing	Automated verification of the 7 Independence Invariants (§4-9) including AST import boundary validation and partial answer fallback when support agents are terminated.
Integration Testing	All branches of the QA collaboration flow (§5-3: no concepts, ambiguity, slot failure, validation rejection, timeout) using fixed response stubs, plus connected pair integration tests (§5-2).
End-to-End Pipeline	Full end-to-end execution: Prior Info → Upload → DB Ingestion → Guide → Connectivity Graph → Semantic Deliberation → Querying → Feedback → Evaluation.
Invariants & Wiring	Deterministic value computation, domain isolation, constant segregation, and AST dead-code/orphan registration audits (§10-2, §10-3).
Regression & Quality	100% regression pass across historical test suites. Post-R11 rounds require zero-regression pass on eval_lab benchmarking suites.
Live Environment	Dev Environment: Live DB and real LLM gateway executions verified via logs/DB state. UI Shell: Real user interaction validation without dedicated test screens.

🛡️ Special Agent-Layer Verifications

• Trigger Replay: Replaying A4/A5 triggers (human action, events, cron schedules) deterministically yields identical execution results.
• Deduplication Invariant: Firing multiple concurrent triggers against the same domain/target initiates only a single execution under lock.
• Fault Resilience (Agent Absence): Deactivating any individual support agent forces the orchestrator (A0) to degrade gracefully with partial answers or pending holds without crashing.

10-2 The 4 Core Architectural Invariants

#	Architectural Invariant	Automated Verification Method
1	Deterministic Value Calculation	Repeated execution of the same query produces identical numerical values; swapping LLM mock responses yields zero value variation.
2	Knowledge Tier Transparency & Human Gates	Operational answers strictly display the verification tier of all referenced knowledge items. A5 halts unconditionally at human review gates.
3	Strict Domain Storage Isolation	New tables conform strictly to store partition layout; queries specifying different domain IDs fail with zero cross-tenant leakage.
4	Dual Constants Segregation	AST scan confirms zero hardcoded magic numbers or constants residing outside the Shared and Policy Constants systems (§4-8).

10-3 Automated Wiring & Dead-Code Inspection

The automated AST wiring analyzer inspects the entire codebase against the architectural specification to guarantee:

• Zero uninvoked registry entries (hooks, schema fragments, reset/carryover functions, event subscriptions).
• Strict correspondence between package declarations in Agent Specs (§8) and real package invocations.
• Zero unused public class methods or module-level constants.
• Zero orphan DDL tables lacking code references.
• Every declared contract request type possesses both a dedicated sender and a designated handling agent.
• All event publish/subscribe pairs (§5-2) are verified with zero direct publisher→subscriber module imports.

10-4 The 6 Cardinal Principle Gates

Principle Gates represent strict automated architectural barriers added to round exit conditions:

Gate 1: Zero Sample Data

Verifies 0 hardcoded sample domain identifiers exist in code, constants, or prompt files.

Gate 2: LLM Non-Dependency

Evaluation assets run across multiple LLM backends yielding identical value judgments with pass rates above quality thresholds.

Gate 3: Zero Documentation Assumption

Cold-start pipeline (Upload → Connectivity → First Query) passes completely without auxiliary data manuals.

Gate 4: Minimal User Overhead

Enforces policy constant caps on questions per round and maximum top-level UI menu views.

Gate 5: General Format Robustness

Synthetic datasets with mixed delimiters, packed keys, and overlapping ranges pass ingestion and query pipelines seamlessly.

Gate 6: Human Confirmation Supremacy

All agent pre-assignments carry "Needs Review"; promoting knowledge to Confirmed tier strictly rejects non-human actors.

10-5 Definition of Done: 10 Final Acceptance Scenarios (S1 ~ S10)

The system is officially certified as complete when all 10 acceptance scenarios pass across multiple synthetic datasets with distinct formatting combinations:

#	Acceptance Scenario	Target Verification & Pass Criteria	Target Round
S1	Zero-Manual Cold Start	Uploads raw data without documentation to construct connectivity graphs. Role classifications match ground truth above match threshold. Ambiguous roles assigned "Needs Review".	R4
S2	Semantic Deliberation & Answering	User enters/confirms node semantics; subsequent queries utilizing that semantic rule yield accurate computed values with visible knowledge tiers in evidence.	R10
S3	Complex Queries & Plan Promotion	Multi-step queries computed immediately with calculation plan, rationale, and exclusion reasons. Inferred plans promoted to confirmed definitions and reused across parameter variations.	R6
S4	Operational Mutation Consistency	Incremental uploads, status shifts, and semantic edits correctly invalidate affected historical answers while unimpacted answers remain cached. Dependent knowledge marked Under Review.	R7
S5	Multi-Tenancy & Cost Controls	Strict RBAC enforcement across 4 tiers with 0 unauthorized data leaks. Zero paid LLM calls without user consent. Read-only users generate zero metrics mutations.	R10
S6	Correction Generalization	Confirming a correction rule applies automatically to all identically formatted columns/nodes across current and incremental datasets without manual intervention.	R9
S7	Cumulative Learning & Clarification	Recurrence of identical correction patterns steadily declines over rounds. Clarification choices automatically populate future query defaults with "Prior Choice Applied" badge.	R11
S8	Question Type Coverage & Guard	Covers trends, rankings, aggregations, and mixed questions. Natural language narrative claims are verified by guard against computed features with zero hallucinations.	R11
S9	Answer Reproduction & Rollback	Historical answers reproduced point-in-time from stored dependency versions. Semantic and structural rule rollbacks restore previous states with full actor audit trails.	R6
S10	Cross-Server Migration & Reusability	Domain export bundles imported across schema versions with automatic remapping. Human confirmed items, outlier criteria, and calculation rules re-applied seamlessly on matching signatures.	R12

#QualityEngineering #DefinitionOfDone #AcceptanceTesting #Biligent #MultiAgentSystems

© Biligent Quality Architecture & Acceptance Series. All rights reserved.