#vllm

🏛️ Multi-Agent Debate & Brainstorm Studio | 本地多Agent辩论&头脑风暴 — run Hermes/Codex/Ollama on your machine

Asistente de IA para usuarios de HPC (Slurm+Spack): revisa sbatch antes de correr, corrige con reglas deterministicas + LLM. Casos reales: hasta 2.7x mas rapido. Produccion: IFIMAR (CONICET-UNMDP).

Agent Skills and an MCP server for GPU performance profiling, benchmarking, optimization, and reporting, with an inference focus.

Reliability for LLM agents through enforcement, not model size — Agent-Contract-Kernel + a fail-closed orchestration engine. Model-agnostic. 🇦🇪 Built in the UAE.

Forge is a local-first, privacy-respecting AI IDE forked from VS Code. Designed to run completely offline with Ollama, LM Studio, vLLM, and local OpenAI-compatible endpoints—with telemetry completely removed and open extensions via Open VSX.

Cross-platform desktop AI workspace unifying local llama.cpp, vLLM, OpenAI, Anthropic, and Gemini — with agentic file/shell tools, clinical decision support, patient cases, audit logging, and MCP integrations. Electron + React + TypeScript.

Self-hosted privacy firewall for LLM traffic. Detect, pseudonymize, route, and safely restore sensitive data through an OpenAI-compatible API.

Self-hosted pentest copilot. Substrate-first (technique catalog + playbooks + RAG + deterministic tools) with structural verifier rules and LLM-as-judge quality eval. Apache 2.0.

Claude Code provider selector for Anthropic, Ollama, Ollama Cloud, vLLM, NVIDIA hosted, and self-hosted NIM

Docker Compose stack for serving a local, OpenAI-compatible LLM (vLLM on Intel XPU) on an Intel Arc Pro B60 GPU — reproducible config with operator and developer docs.

Reproducible token-throughput benchmark for OpenAI-compatible LLM servers, tuned for NVIDIA Spark and GB10 inference.

Local-first Kubernetes platform for private LLM and coding-agent workloads.

On-Premise RAG (продвинутая версия): vLLM/Ollama + bge-m3 + reranker + Qdrant + hybrid (BM25+RRF) + PDR + Basic Auth + Prometheus/Grafana + локальная оценка. Запускается на Ollama без GPU. Тех-задача Petrel AI (Astana Hub).

Model-agnostic substrate for running small local LLMs well — code agents, chat, image/video, speech. Small capable main model + tiny fast worker.

Demonstrating an attribution gap in LLM inference telemetry: per-request tracing records the symptom of KV-cache contention and not the cause

LLM-driven evolutionary search that discovers a better qubit-layout heuristic than the one it starts with, measured by the two-qubit gate count Qiskit emits after routing. Runs on the qBraid AI Gateway or your own GPU.

Production-oriented Kimi K3 inference control plane with checkpoint release gates, MoE capacity planning, OpenAI gateway, NVL72 deployment, benchmarks, and observability.

Run open LLMs on free Colab/Kaggle GPUs and call them from your machine over an OpenAI-compatible endpoint

Production-grade LLM inference deployment on AWS using vLLM and Qwen2.5, running on NVIDIA GPU-powered K3s Kubernetes with Terraform, Helm, Prometheus, and Grafana observability.

Local-model gateway for Claude Code, Zoo Code, and Kilo: route AI coding agents through llama.cpp/Ollama/vLLM on Distrobox. No API keys, no cloud fallback, nothing auto-starts - local mode either works or fails cleanly. Classic LiteLLM proxy, or the optional single-provider Kilo mode.

Lab notebook of a German craftsman running his solar business on local AI: 2x DGX Spark, Mac Studio, MCDMA. Recipes, benchmarks, mistakes. Every source credited.

YYC³ AI Family·Comic Drama — 8拟人Agent×ReAct-C九步闭环驱动「小说→分镜→图像→视频→成片→运营」六阶段全链路工业化漫剧生产系统 | Mac M4 + 双DGX Spark GB10 + NAS | 90天3集样片·≤2元/集·一致性≥80%

Load test LLM inference servers with k6. TTFT, ITL, TPOT, goodput, cost, and energy metrics for any OpenAI-compatible server. Ships to Prometheus and Grafana.

Backend-agnostic black-box hallucination & RAG-faithfulness detection for LLMs — Probabilistic Certainty & Consistency (arXiv:2601.02574). Works on MLX / OpenAI / vLLM via token logprobs; no model internals, no training.

OpenAI uyumlu inference endpoint'lerini test etmek için çapraz platform betikler: chat, model keşfi, TTFT'li yük testi ve embeddings sağlık paketi (Bash · PowerShell · Python, bağımlılıksız)

Lightweight Chrome Extension (Manifest V3) for seamless Chinese-to-Russian web translation using local LLMs (Ollama, LM Studio, vLLM). 100% DOM-safe, SPA & React friendly, viewport-only scan & LRU caching.

Thin, zero-dependency CLI to run local LLMs on Apple Silicon (vllm-mlx or mlx_lm) with OpenAI- and Anthropic-compatible endpoints — point Claude Code at a local model.

Not a leaderboard — a way to pick your daily driver. Benchmark model + quant + serving flags + thinking mode on your own hardware, then install the config that won. Code-gen and agentic tool-calling suites, hidden executable tests, mermaid reports.

Reliable multi-agent tool calling on a small model, by giving the model less to decide. Tiered routing, schema-validated decisions, graceful tool degradation, MCP server.

MCP server that saves Claude Code tokens by delegating bounded tasks to local or cloud LLMs. Works with LM Studio, Ollama, vLLM, DeepSeek, Groq, Cerebras.

Here — 끊긴 업무의 시작점과 다음 행동을 복원하는 데스크톱 앱 · OpenAI-compatible/vLLM

Early version of coprompter: optimization code leveraging vLLM and the OpenAI API wrapper for faster, asynchronous LLM inference with coroutines.

Agentic coding CLI for any model: Claude, OpenAI, Gemini, OpenRouter, local Ollama/vLLM, or your own Colab GPU. Learns from its own mistakes.

Benchmark and compare LLM inference backends (OpenAI-compatible APIs) - llama.cpp, Ollama, LM Studio, vLLM

From-scratch LLM inference engine — continuous batching, paged KV cache, prefix caching, hand-written Triton paged attention, CUDA graphs, OpenAI-compatible API. Benchmarked against vLLM on an A100.

Never give AI companies your secrets! A local LLM-based privacy filter for LLM users. Seamless integration with your existing AI tools as a Python library / OpenAI SDK replacement / API Gatetway / Web Server.

Turn-based geopolitical simulator where 5 AI civilizations — militarist, mercantile, theocratic, democratic, and authoritarian — reason via LLM (Llama 3.3 70B / Groq) to form alliances, wage wars, conduct espionage, and pursue competing victory conditions. FastAPI game engine · Flask frontend · fantasy SVG world map · injectable event cards.

Provider-agnostic VLM query flow: one Agent that dispatches to OpenAI / Anthropic / Gemini / vLLM / Claude Code CLI / local HuggingFace model backends, returning parsed answers. Used by 3D-CoT.

Serve DeepSeek and Qwen, run an on-demand model library, and fine-tune with Unsloth QLoRA on 2x NVIDIA DGX Spark. TP=2 vLLM lanes behind one OpenAI-compatible endpoint for OpenCode, Cursor, and Hermes. Honest benchmarks, committed artifacts.