#benchmarking

A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.

A Hermes Agent skill for turning painful fixes into replayable benchmarked workflows

Prove your Agent Skill works before you publish it.

Tests exact agent interventions—skills, prompts, models, tools, and workflows—and publishes what changed, what failed, and what decision follows.

Production-oriented Kimi K3 inference control plane with checkpoint release gates, MoE capacity planning, OpenAI gateway, NVL72 deployment, benchmarks, and observability.

Evidence-first, reproducible quality benchmarks for AI agents.

Multi-agent LLM orchestration and benchmarking toolkit -- planner-solver-reviewer workflows with provider abstraction and scoring

Load test LLM inference servers with k6. TTFT, ITL, TPOT, goodput, cost, and energy metrics for any OpenAI-compatible server. Ships to Prometheus and Grafana.

Transparent benchmark cards, JSON artifacts, and visual tools for photonic AI accelerator energy/noise claims.

Cross-platform diagnostics and live LLM performance benchmarking for Claude Code, Headroom, and system bottlenecks

A fast, git-native CLI & local Web UI to version, diff tokens/costs, and benchmark LLM system prompts across local & cloud models.

Controlled coding-agent harness ablations with auditable traces, reproducible evidence bundles, and SWE-bench interoperability.