Show HN: Incremental JSON parser for streaming LLM tool calls in Ruby

An incremental JSON parser for Ruby that efficiently handles streaming LLM tool calls by maintaining parsing state and processing only new characters, achieving O(n) performance instead of O(n²) reparsing. The gem is MIT licensed and designed to prevent UI lag when LLMs stream function arguments character-by-character.

Key Points

• •LLMs stream function arguments as JSON character-by-character, requiring incremental parsing rather than full reparsing
• •Traditional JSON parsers reparse from scratch on each new character, resulting in O(n²) time complexity and noticeable UI lag
• •Incremental parser maintains parsing state across updates, processing only new characters for true O(n) performance
• •O(n) incremental approach keeps response streaming imperceptible to users, eliminating performance bottlenecks
• •Solution implemented as a Ruby gem with MIT license, designed specifically for streaming LLM tool calls
• •Maintains parser state between invocations to avoid redundant work on previously-processed characters
• •Enables smooth, real-time UI updates when displaying streaming JSON responses from language models

Workflow Diagram

Concepts