Jevstiller distills AI responses into efficient local model

Jevstiller is an open source project that creates a local student model from queries to a specialized AI system called Jev, achieving 98 percent agreement while processing familiar requests on-device in as little as 15 milliseconds. The approach reduces API costs and latency by caching common requests locally and only forwarding uncertain queries upstream for auditing and processing. Jev itself represents a new AI paradigm optimized for structured decision-making tasks using Choice, Score, and Noul question types rather than free-form text generation.

Jevstiller addresses a practical problem in AI deployment by creating a lightweight local model trained on historical responses from Jev, a specialized system designed for structured decision-making rather than general text generation. The system learns patterns from repeated query types—such as email filtering or task prioritization—and handles predictable requests directly on-device, dramatically reducing both computational costs and response delays. Unknown or edge-case queries still route to upstream Jev servers for proper handling.

The architecture includes continuous quality assurance through systematic auditing. A fixed percentage of all requests bypass local processing and go directly to Jev as a control mechanism, allowing developers to measure drift and maintain confidence that local responses remain aligned with upstream results. This approach aims to balance efficiency gains against the risk of accumulated errors in local decision-making.