#ai-alignment

Agentic evaluation of misalignment, scheming and evaluation awareness in LLM agents. Scenario text encoded at rest to resist training-data contamination.

v0.8 preview · 2,288 multi-step agent trajectories · 513 hand-authored gold + 1,775 provenance-flagged augmented. A per-step benchmark that scores whether a verifier catches drift inside an agent's trajectory — not whether a prompt is harmful.

Same agent action, safe or dangerous by context: email team vs investor; delete scratch vs prod config. A safety layer routes each action: auto, confirm, escalate, block. A reading model auto-executes 0% of unsafe actions vs a baseline's 38% (McNemar p<0.001). Plus a runtime loop: an agent earns or loses autonomy with each counterparty over time.

A rumination and error-digestion protocol for AI-assisted specification workflows.

Does an LLM's willingness to blackmail depend on its persona — and on whether that persona is prompted or trained into the weights? Two-part study on Gemma-3-12B: 16 prompted personas (n=100) + 10 QLoRA persona adapters, in the Lynch et al. shutdown scenario.