Production Assurance Framework
One framework. Different commercial moments.
Production assurance uses seven core engineering domains, with three contextual overlays activated only where the operating context makes them relevant. The commercial trigger changes the depth and emphasis, not the underlying assurance logic.
The domains Lornets examines
Seven core Production Assurance domains normally apply to software carrying commercial or operational responsibility. Three contextual overlays are not additional equal domains. They activate only where the System Context Profile makes them relevant. Assessment depth and relative importance vary according to the operating context. Lornets does not apply a universal fixed weighting to every system.
Core domains 01 to 07
- 01Architecture & Maintainability
- 02Security & Access Control
- 03Reliability & Recoverability
- 04Performance & Scalability
- 05Data & Privacy Engineering
- 06Delivery & Change Control
- 07Observability & Operations
Contextual overlays
- Enterprise AssuranceContextual overlay
- AI AssuranceContextual overlay
- Product / Operational InstrumentationContextual overlay
01
Core domain
Architecture & Maintainability
What we are trying to establish
Architecture determines the cost of every future change. Decisions that were reasonable at prototype stage constrain what can safely be changed once the software carries commercial or operational responsibility.
What we examine
System boundaries, service and module structure, coupling, state management, background processing, third-party dependencies and how the current design constrains the next twelve months of change.
Typical assurance claims
- The system can absorb the next twelve months of change without redesign of a critical path.
- A failure in one component does not compromise unrelated business functions.
Questions we ask
- Which parts of the system cannot be changed without touching everything else?
- Where does business logic actually live?
- Which third-party services would halt the product if they changed terms or availability?
- What would need to change to support ten times the current usage?
Evidence we may use
- Architecture documentation and diagrams
- System boundaries, trust boundaries and data flows
- Deployment topology and failure domains
- State ownership and service dependencies
- Integration boundaries and coupling in source code
- Recorded architectural decisions and scaling assumptions
- Where technical knowledge is concentrated
Measurements where meaningful
- Dependency and coupling characteristics
- Number and criticality of external integrations
- Blast radius of individual failure domains
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC/IEEE 42010:2022, ISO/IEC 25010:2023.
What can trigger a Critical Readiness Gate
Architecture that makes a critical control impossible to enforce, or a single failure domain the business cannot survive.
02
Core domain
Security & Access Control
What we are trying to establish
Security findings turn a technical issue into a commercial one. They surface during diligence, procurement and customer review, and they carry regulatory and reputational consequences.
What we examine
Authentication, authorisation and access control, tenant isolation, secret management, input validation, dependency vulnerabilities, exposed endpoints, storage rules and administrative access paths.
Typical assurance claims
- Customer data cannot be accessed by another customer without appropriate authorisation.
- Authorisation is enforced server-side on every protected operation.
- Production secrets are not recoverable from source control or client code.
Questions we ask
- Can one customer reach another customer's data through any code path?
- Where is authorisation enforced, and is it enforced on the server for every route?
- Which secrets have ever been committed or shared outside a secret manager?
- Who holds production access, and how is that access removed?
Evidence we may use
- Authentication and authorisation implementation
- Tenant isolation and privilege boundaries
- Secret storage and session management
- Dependency and vulnerability results
- Exposed interfaces and secure configuration
- Software supply-chain and build controls
- Security test results
Measurements where meaningful
- Known vulnerabilities by severity on exposed surfaces
- Dependency currency and patch latency
- Coverage of authorisation checks on protected routes
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: OWASP Application Security Verification Standard, NIST SP 800-218, NIST Cybersecurity Framework 2.0.
What can trigger a Critical Readiness Gate
Material broken access control, inadequate tenant isolation, exposed secrets or critical known vulnerabilities on exposed surfaces.
03
Core domain
Reliability & Recoverability
What we are trying to establish
Once a business depends operationally on software, the cost of an outage stops being technical. Real usage tests failure, recovery and data restoration whether or not they have been exercised deliberately.
What we examine
Failure modes, retry and idempotency behaviour, dependency timeouts, background job durability, backup coverage, restore testing, recovery objectives and incident response practice.
Typical assurance claims
- Business-critical data can be restored within the agreed recovery objectives.
- The system degrades in a controlled way when a critical dependency fails.
Questions we ask
- What happens if a critical dependency is unavailable?
- Can failed operations be retried safely?
- Has backup restoration actually been tested?
- What is the acceptable recovery time?
- What is the acceptable amount of data loss?
Evidence we may use
- Failure-mode analysis and dependency timeouts
- Retry and idempotency behaviour
- Backup configuration and restoration evidence
- Documented recovery objectives
- Incident history and post-incident records
- Operational ownership and on-call arrangements
Measurements where meaningful
- Availability against defined objectives
- Error rates and incident frequency
- RTO and RPO against demonstrated recovery performance
- Behaviour under dependency failure
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC 25010:2023, NIST Cybersecurity Framework 2.0.
What can trigger a Critical Readiness Gate
No credible, demonstrated recovery capability for an operationally critical system, or inability to recover business-critical data.
04
Core domain
Performance & Scalability
What we are trying to establish
Infrastructure and architecture programmes are sometimes committed before the actual constraint has been measured. Identifying the real constraint changes both the scope and the cost of the work required.
What we examine
Query patterns and indexing, N+1 behaviour, hot paths, payload sizes, caching, connection limits, queue depth, rate limiting and the actual measured bottleneck rather than the assumed one.
Typical assurance claims
- The system sustains the expected workload within agreed latency and error thresholds.
- The actual constraint under load has been measured rather than assumed.
Questions we ask
- What is the measured bottleneck under realistic load?
- Which queries dominate database time?
- What breaks first as concurrency rises: the database, the queue or the runtime?
- Which costs grow linearly with users, and which grow faster?
Evidence we may use
- Query patterns, indexing and hot paths
- Caching, queueing and rate-limiting configuration
- Load-test results where they exist
- Production performance telemetry
Measurements where meaningful
- p50, p95 and p99 latency
- Throughput and concurrency
- Database latency and query behaviour
- Queue depth and connection utilisation
- Resource saturation and load-test headroom
- Degradation under load
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC 25010:2023, ISO/IEC 25040:2024.
What can trigger a Critical Readiness Gate
A measured constraint that will fail a commitment the business has already made to customers.
05
Core domain
Data & Privacy Engineering
What we are trying to establish
A weak data model or an unclear retention position becomes considerably more expensive to correct after the dataset grows and after customers begin asking where their data goes.
What we examine
Data model integrity, migrations, retention, personal data handling, residency, encryption at rest and in transit, deletion paths, logging of sensitive values and data shared with third parties including model providers.
Typical assurance claims
- Personal data is stored, shared and deleted as committed to customers.
- Sensitive values are not exposed through logs, exports or third-party integrations.
Questions we ask
- What personal data is held, and why?
- Can a customer's data be exported and deleted completely?
- Which third parties, including model providers, receive customer data?
- Are migrations reversible, and are they tested before production?
Evidence we may use
- Data model, migrations and retention configuration
- Personal data inventory and data-flow documentation
- Encryption in transit and at rest
- Deletion and export paths
- Third-party and model-provider data sharing
- Logging of sensitive values
Measurements where meaningful
- Completeness of deletion and export paths under test
- Migration reversibility verified in a non-production environment
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC 25010:2023, NIST Cybersecurity Framework 2.0.
What can trigger a Critical Readiness Gate
Sensitive data exposed beyond its intended boundary, or an inability to recover or delete customer data as committed.
06
Core domain
Delivery & Change Control
What we are trying to establish
Delivery practice determines how quickly a team can fix a serious problem. Investors and enterprise customers read it as a proxy for engineering maturity.
What we examine
Environment separation, branching and review practice, automated testing coverage on critical paths, build and deployment pipeline, rollback capability, configuration management and release frequency.
Typical assurance claims
- A harmful change can be reverted quickly and safely.
- No unreviewed change can reach production.
Questions we ask
- Can a change be reverted in minutes?
- Is there a real environment separation between development and production?
- Which paths are covered by automated tests, and which are not?
- Who can deploy, and what stops an unreviewed change reaching production?
Evidence we may use
- Build reproducibility and CI configuration
- Environment separation and configuration management
- Testing gates and release traceability
- Rollback capability
- Branch and repository controls
- Artefact provenance where appropriate
- Deployment history
Measurements where meaningful
- Change lead time
- Deployment frequency
- Failed deployment recovery time
- Change fail percentage
- Deployment rework rate
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: DORA delivery metrics, NIST SP 800-218, SLSA.
What can trigger a Critical Readiness Gate
No controlled path to production, or no ability to revert a change that is actively harming customers.
07
Core domain
Observability & Operations
What we are trying to establish
Without observability, every incident becomes an investigation. Teams cannot prove reliability to a customer or an investor if they cannot measure it.
What we examine
Structured logging, error tracking, metrics, tracing on critical flows, alerting thresholds, on-call arrangements, runbooks and the ability to answer questions about production without guessing.
Typical assurance claims
- A material failure is detected by the team before a customer reports it.
- A single request can be traced across the system during an incident.
Questions we ask
- How would the team learn about a failure before a customer reports it?
- Can a single user request be traced across the system?
- What alerts exist, and who receives them?
- What was the last incident, and what changed afterwards?
Evidence we may use
- Structured logging, metrics and tracing coverage
- Error tracking configuration
- Alert definitions and routing
- Runbooks and on-call arrangements
- Evidence that incidents were detected before customers reported them
Measurements where meaningful
- Detection time against incident history
- Trace coverage on critical flows
- Alert precision and volume
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC 25010:2023, NIST Cybersecurity Framework 2.0.
What can trigger a Critical Readiness Gate
No means of detecting or diagnosing failure in a system the business depends on operationally.
Overlay
Contextual overlay
Enterprise Assurance
What we are trying to establish
Enterprise procurement asks for evidence, not assurances. It separates two different problems: a control that does not exist, and a control that exists but cannot be demonstrated on request.
What we examine
Role and permission models, audit logging, SSO capability, tenancy model, uptime evidence, architecture and data-flow documentation, vulnerability management and change management records.
Typical assurance claims
- Relevant controls can be demonstrated with evidence, not only described.
- Privileged activity is logged and reviewable.
Questions we ask
- Can the team produce an architecture diagram and data-flow map today?
- Is privileged activity logged and reviewable?
- Does the permission model match what customers will ask for?
- What evidence exists that controls operate, rather than that they were written down?
Evidence we may use
- Role and permission models
- Audit logging of privileged activity
- Tenancy model and SSO capability
- Architecture and data-flow documentation
- Vulnerability and dependency management records
- Change-management and incident-process records
- Uptime evidence
Measurements where meaningful
- Coverage of privileged actions in audit logs
- Vulnerability remediation timeliness
- Documentation currency against the deployed system
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: OWASP Application Security Verification Standard, NIST SP 800-218, NIST Cybersecurity Framework 2.0.
What can trigger a Critical Readiness Gate
Contractual or procurement commitments the current technical controls cannot support or evidence.
Overlay
Contextual overlay
AI Assurance
What we are trying to establish
AI features can move from demonstration to something customers depend on without the controls that dependence requires. The failure modes differ from conventional software.
What we examine
Prompt and context handling, data sent to model providers, output validation, tool and action permissions, cost and rate controls, evaluation practice, fallback behaviour and provider dependency.
Typical assurance claims
- AI components cannot take privileged actions outside defined permission and validation boundaries.
- AI output quality is evaluated against defined thresholds rather than judged anecdotally.
Questions we ask
- What customer data reaches a model provider, and under which terms?
- What happens when the model returns an unusable or unsafe output?
- Which actions can an AI component take without human confirmation?
- How is quality measured over time rather than judged anecdotally?
Evidence we may use
- Intended AI task definition and evaluation datasets
- Model and prompt versioning
- Output validation and fallback behaviour
- Agent and tool permission configuration
- Data sent to model providers and applicable terms
- Evaluation results and monitoring
- Cost and consumption controls
Measurements where meaningful
- Task-specific evaluation results against defined quality thresholds
- Error and hallucination rates on evaluated tasks
- Evaluation coverage of consequential paths
- Reproducibility across model or prompt versions
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: NIST AI RMF 1.0, NIST Generative AI Profile, NIST SP 800-218A, ISO/IEC 42001.
What can trigger a Critical Readiness Gate
AI components taking privileged or consequential actions outside required permission, validation or human-review boundaries.
Overlay
Contextual overlay
Product / Operational Instrumentation
What we are trying to establish
Usage evidence supports engineering prioritisation and external conversations. Absent or unreliable instrumentation removes the basis for both.
What we examine
Event coverage on critical journeys, identity resolution, data quality, funnel instrumentation and whether the business can answer usage questions from its own systems.
Typical assurance claims
- Operational and product behaviour can be answered from the organisation's own instrumentation.
Questions we ask
- Can the business measure activation, retention and feature usage today?
- Is the same user identified consistently across the product?
- Which decisions are currently made without data that could be available?
Evidence we may use
- Event schema and instrumentation coverage
- Identity resolution logic
- Funnel and journey instrumentation
- Data-quality checks
Measurements where meaningful
- Event coverage on critical journeys
- Identity match rate
- Data-quality error rate
External foundations
Referenced and mapped against where relevant; this does not constitute a compliance claim: ISO/IEC 25019:2023.
What can trigger a Critical Readiness Gate
Rarely gate-relevant. Recorded as an evidence limitation where usage evidence is required for a commercial decision.
How the domains are applied
Not every domain applies identically to every system. The System Context Profile determines which domains apply, the maturity expected, the evidence required and their relative importance. Assessment depth and relative importance vary according to the operating context. Lornets does not apply a universal fixed weighting to every system.
The framework describes what Lornets examines. It does not describe how a conclusion is reached. Readiness follows from context, evidence, domain findings, Assurance Confidence and Critical Readiness Gates.
Framework and methodology
The framework describes what Lornets examines. The methodology describes how Lornets reaches a conclusion: system context, direct technical evidence, measurement where measurement is meaningful, evaluation against defined criteria and recognised external frameworks, and structured senior engineering judgement.
AI-assisted development can materially accelerate software creation. Functional success, however, is not sufficient evidence of production quality. Recent empirical software-engineering research has identified distinct defect and security profiles in AI-generated code, reinforcing the need for independent verification rather than assumptions based on how the software was produced.
Definitions
- Production Assurance
- The structured evaluation and strengthening of software so it can safely support the commercial and operational demands being placed on it.
- System Context Profile
- A structured description of what a system must withstand, covering operational criticality, data sensitivity, exposure, commercial scrutiny, organisational dependency, scale and AI consequence. It determines which criteria apply, the maturity expected and their relative importance.
- Assurance Claim
- A specific technical condition that needs to be true for the stated operating context, stated so that evidence can support, partially support or fail to demonstrate it.
- Evidence Level
- How strongly a claim or capability can currently be substantiated, recorded on the E0 to E4 hierarchy from asserted through artefact evidenced, verified and operationally evidenced.
- Evidence Ledger
- The record linking each material conclusion to the evidence, verification, limitations and decision behind it, so the reasoning can be traced rather than assumed.
- Assurance Confidence
- How strongly the overall assessment conclusion is supported by the evidence reviewed. It is an evidence-strength assessment, not a statistical confidence interval.
- Critical Readiness Gate
- A material technical condition that may prevent a positive Readiness Position for the defined operating context regardless of stronger findings elsewhere. Gate state is reported as Clear, Conditional or Blocked.
- Readiness Position
- The final conclusion for a defined scope and operating context: Ready, Conditionally Ready or Not Demonstrated Ready. It is not a certification and not a guarantee.
- Production Assurance Dossier
- The assessment record, covering the executive readiness position, scope and system context, material findings and Critical Readiness Gates, the Evidence Ledger and verification record, engineering decisions and priorities, and relevant technical appendices.
- Engineering Work Package
- A bounded unit of engineering work traceable to a verified finding and an agreed target condition, defined with scope, intervention, acceptance criteria, verification approach, change risk and owner.
- Production Assurance Baseline
- The known technical position from which ongoing stewardship begins, covering readiness position, Critical Readiness Gates, open findings, Assurance Confidence, architecture, dependencies, recovery and security position and the target operating context.
- Assurance Drift
- The gradual divergence between the production position previously established and the real system as it changes over time. It is assessed through review of material change, not through an automated score.