How is LLEV different from JSON mode or structured outputs?

Structured outputs make a generative LLM write JSON token by token. LLEV reads the probability of each answer label in a single forward pass, so you get a real probability distribution and a confidence score you can automate on, with lower latency.

Can LLEV be used for AI agent guardrails?

Yes. Ask atomic yes/no questions about a proposed tool call, such as whether it is destructive or targets production, and make the allow or block decision in your own code.

Yes. LLEV is Apache-2.0 licensed open-source software; you only pay for your own hardware.

The open-source Jev alternative for self-hosted LLM classification

Q: Is there an open-source alternative to TypeSafe AI's Jev?

Yes. LLEV is an open-source, self-hosted System One decision engine that follows the same idea as Jev (typed questions in, typed answers with confidence out) and accepts requests on a /v1/systemone path. It runs on your own hardware with open models and is not affiliated with TypeSafe AI.

Q: How do I run LLM text classification locally?

Install LLEV, download a model such as Qwen3 1.7B or Gemma 3 4B, start the server with python -m llev and POST your text and labels to /v1/decide. It runs on CPU, Apple Silicon (Metal) or NVIDIA GPUs.

LLEV is a free "System One" decision engine. Send text or JSON plus typed questions, and a small local LLM returns typed answers with calibrated confidence in about 150 ms. Use it for content moderation, ticket triage, AI agent guardrails, sentiment analysis and phishing detection, on your own hardware.

View on GitHub Quick start

# Is this comment a threat? Ask atomic questions, decide in code.
curl localhost:8088/v1/decide -H "Authorization: Bearer $KEY" -d '{
  "state": "Nice try deleting my comment. I know which gym you go to, see you there tomorrow.",
  "questions": {
    "threat":        {"type": "noul", "tier": "accurate", "instructions": "The text threatens someone, even implicitly"},
    "personal_info": {"type": "noul", "tier": "accurate", "instructions": "The text hints at knowing where someone goes"}
  }
}'
→ threat 0.999 · personal_info 1.000 · confidence 1.00 · 405 ms

Why a self-hosted System One model?

LLEV never generates text. It runs one forward pass and reads the probability of each answer label.

Private by default

Runs on your Mac, Linux server or GPU. Your text never leaves your infrastructure.

Calibrated confidence

Real probability distributions, not a sampled answer. Automate when confidence is high, send the rest to a human.

Smart cascade

A fast 1.7B model answers first. Unsure answers, and any question marked accurate, go to a 4B model.

Bias-corrected

Options are asked in several orders and averaged, which cancels small models' bias toward option A.

Four question types

Multiple choice, ordered score, yes/no and multi-label, many per request over one cached state.

Open models

Qwen, Gemma, Llama or Phi in GGUF via llama.cpp. Benchmark them on your own data with one script.

Use cases

Each of these has a ready-made preset in the built-in playground.

Content moderation

Threats, harassment, hate speech and spam, each checked separately on the accurate tier.

Support ticket triage

Route to the right team, score urgency and flag churn risk in a single request.

AI agent guardrails

Check a tool call before it runs: is it destructive, does it target production, does it match the goal?

Phishing and spam detection

Quarantine scams and sort mail into folders with confidence scores.

Lead scoring

Detect large deals, meeting requests and decision-makers, and extract enterprise requirements.

Game NPCs

Real-time reactions and mood for characters in games and simulations.

Benchmarks

56 hand-labelled English cases (support routing, urgency, sentiment, toxicity, spam, intent) on an Apple M4 Pro (Metal), Q4_K_M.

Setup	Accuracy	Accuracy when confident	Median latency
Gemma 3 4B	93%	98%	327 ms
Qwen3 4B	91%	98%	372 ms
Qwen3 1.7B → Gemma 3 4B cascade (recommended)	89%	94%	149 ms
Qwen3 1.7B	82%	91%	151 ms
Qwen2.5 1.5B	73%	91%	148 ms

"Accuracy when confident" is the accuracy of answers with confidence ≥ 0.8. Server CPUs are slower than Metal, so benchmark on your target hardware.

FAQ

Is there an open-source alternative to TypeSafe AI's Jev?

Yes, LLEV. It follows the same System One idea (typed questions in, typed answers with confidence out) and accepts requests on a /v1/systemone path, but runs entirely on your own hardware with open models. LLEV is an independent project and is not affiliated with TypeSafe AI.

How do I run LLM text classification locally?

Install LLEV, download a model (python scripts/download_model.py qwen3-1.7b), run python -m llev and POST your text and labels to /v1/decide. It runs on CPU, Apple Silicon or NVIDIA GPUs.

How is it different from JSON mode or structured outputs?

Structured outputs make a generative LLM write JSON token by token. LLEV reads each label's probability in one forward pass, so you get a real distribution and a confidence score you can automate on, with lower latency.

How is it different from zero-shot classifiers like BART-MNLI?

LLEV uses an instruction-tuned LLM that follows your instructions and option descriptions, supports ordered scales and multi-label questions, corrects option-order bias and escalates unsure answers to a bigger model.

Can I use it for AI agent guardrails?

Yes. Ask atomic yes/no questions about a proposed tool call and make the allow or block decision in your own code. Small models are reliable at atomic facts and unreliable at composite judgements.

Is it free?

Yes. LLEV is Apache-2.0 licensed; you only pay for your own hardware.