Keep the baseline
Retain the model’s existing safeguards, sandbox, identities and approval controls. Document their configuration.
Brake Pedal / research invitation
Keep your safeguards.
Test the boundary.
A proposed evaluation for independent runtime authority. One workflow, an isolated environment and acceptance criteria agreed before the run.
The invitation
Complement. Duplicate. Fail.
All are useful findings. This is an invitation to design a controlled test with SPQR, not evidence that a lab has participated or endorsed Aegis. The public preview is not a penetration-testing target or a bounty programme.
Add the boundary
Integrate Aegis at the agreed action-enforcement point. Verify interception, credentials and alternative paths.
Count both sides
Measure legitimate task completion as well as unauthorised effects and false denials. A system that blocks everything has not won.
Protocol outline · proposed, not a completed experiment
Agree the result before the test.
Choose one consequential workflow.
Describe the business objective, operation, target system and consequence. Use synthetic or approved non-production data.
Freeze the comparison.
Record model and tool versions, existing controls, prompts, policy state, data and the attack/retry budget. Compare matched baseline and governed runs.
Define what must be allowed and prohibited.
Specify authorised effects, prohibited effects, approval requirements, timing constraints and who resolves ambiguity.
Identify the trusted boundary.
Locate the decision layer, enforcement point and credential holder. Enumerate bypass routes, policy-change paths and outage behaviour.
Measure the operational outcome.
Capture the actual effect, not just a displayed verdict. Include useful completion, false denials, latency, evidence quality and recovery.
Agree disclosure and a decision.
Set stop conditions, access limits, cost, duration and the handling of vulnerabilities before testing. Report failures under the agreed disclosure process. Conclude proceed, revise or stop.
What the scorecard must retain
An outcome is more than a verdict.
| Measure | What to record |
|---|---|
| Useful completion | Authorised tasks completed / eligible authorised tasks, with the task definition fixed. |
| Unauthorised effects | Actual prohibited effects / relevant test opportunities, including alternate paths. |
| False denials | Legitimate requested actions incorrectly blocked; include the ground-truth policy decision. |
| Overhead | Decision latency distribution, end-to-end time, integration effort and operator burden. |
| Evidence quality | Whether the exact action, active authority, approval and effect can be reconstructed. |
| Recovery | Stale-policy, outage, revocation and approval-change behaviour under agreed conditions. |
These are proposed measures, not promised thresholds. Define success and failure with the evaluating team.
One clear next step
Bring one workflow.
Start with the methods.
A short methods conversation determines whether a shared test is useful. It is not permission to attack a public service.
One workflow. Defined scope. A decision about fit.