#agent-evaluation

🔁 Build reliable recurring AI-agent systems: 968 resources, 22 operational patterns, 22 loop contracts, 8 runtime starters, an interactive atlas, and a structured dataset.

Live, open-source benchmark for comparing AI coding agents on real GitHub issues

Proof Gradient is the agent evolution protocol where every run leaves proof, every proof selects intelligence, and every selected artifact evolves the network.

Catch visible-but-unbound coding-agent handoffs: CLI + GitHub Action with proof, policy gates, SARIF, HTML, and benchmark cards.

Tests exact agent interventions—skills, prompts, models, tools, and workflows—and publishes what changed, what failed, and what decision follows.

DSH 的全自动 LLM-as-a-Verifier 模式：支持并行/串行候选、PPT 择优与实时可视化

Evidence-first, reproducible quality benchmarks for AI agents.

Guarded CLI, TypeScript client, and MCP server for JustHandled x402 agent utilities

Regression-diff eval harness for Anthropic tool-use agents that surfaces LLM-judge reasoning drift, not just pass/fail flips

Autonomous UI/game-testing agent: vision-language reasoning, browser control, action planning, failure recovery, and evaluation.

Framework open source para evaluar agentes LLM con function calling / tool use. Scoring por dimensión (tool accuracy, response quality, safety) + CLI, API y dashboard.

Do your agent pipeline's own checks catch planted faults? Measured on LangGraph, CrewAI and AutoGen: median 16.7%. One prompt-level change takes it to 55.0%. Pre-registered spec, Apache-2.0 harness, every raw trace published.

Craftonomous is an agent-agnostic Minecraft embodiment and evaluation substrate: an MCP-native body with a declared perception budget and reliability-tracked skills.

Controlled coding-agent harness ablations with auditable traces, reproducible evidence bundles, and SWE-bench interoperability.

Open measurement infrastructure for the Agent Capability Economy — how AI agents discover, choose, use, and derive value from software capabilities (Reach → Choice → Use → Utility → Value). A proposed open measurement standard.