Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

Forge is an open-source reliability layer for self-hosted LLM tool-calling that dramatically improves agentic task performance on consumer hardware. It uses domain-agnostic guardrails (retry nudges, step enforcement, error recovery, VRAM management) to boost an 8B model from 53% to 99% accuracy on multi-step workflows without model changes. The framework addresses the compounding failure problem in local models—90% per-step accuracy yields only 59% success on 5-step tasks—and peer-reviewed results show an 8B model with Forge (99.3%) outperforming Claude Sonnet without guardrails (87.2%).

Key Points

• •Guardrails stack has five independently toggleable layers: retry nudges (24-49 point impact), error recovery (~10 points), step enforcement (situational), rescue parsing, and context compaction
• •Serving backend infrastructure matters significantly—same Mistral-Nemo 12B weights achieve 7% accuracy on llama-server vs. 83% on Llamafile, a 75-point swing from infrastructure alone
• •Introduced ToolResolutionError exception class to distinguish between successful tool execution with data vs. successful execution with no results, preventing silent garbage data cascade
• •VRAM-aware context management prevents silent CPU fallback by querying nvidia-smi at startup and deriving token budgets, avoiding 10-100x inference slowdowns
• •Evaluated across 97 model/backend configurations, 18 scenarios, 50 runs each; Ministral 8B with Forge reaches 99.3%, Claude Sonnet with Forge reaches 100%
• •Error recovery scores 0% for all models (local and frontier) without retry mechanism—architectural absence, not capability gap
• •Proxy server mode enables transparent guardrail application to any OpenAI-compatible client without model changes
• •v0.6.0 introduces harder eval suite (26 scenarios) designed to prevent ceiling effects; even Opus 4.6 cannot sweep it
• •Compounding accuracy problem: 90% per-step accuracy on 5-step workflow = 59% end-to-end success; Forge solves mechanical reliability without changing model weights
• •Production-ready for always-on agentic systems on consumer hardware ($600 GPU); currently deployed for home assistant (Ministral 14B) and agentic coding harness (8B)

Workflow Diagram

Concepts