Skip to content

Use case: content classification

Problem: UGC platforms must risk-rank every submission; RAG apps must decide which retrieved passages reach the answering model. A big model per item is too expensive; keywords alone miss too much.

Jev’s role: a Score against ordered levels, with the number mapping straight to an action. “Levels are actions” is a recurring official-cookbook pattern.

{
"type": "score",
"instructions": "The risk level of this submission",
"criteria": [
"Fine as-is",
"Borderline — a human should look",
"Clearly violating — remove"
]
}

The code side is unambiguous: score < 0.5 publish, < 1.5 review queue, otherwise remove and keep probabilities for appeals. Unlike keyword lists, level descriptions track your policy — update the text, no retraining.

Official cookbook (via the llms.txt index): for each retrieved passage, ask a set of parallel questions and let code decide what reaches the answering model —

  • Keep and flag passages that contradict the question (the answer model needs to know)
  • Drop passages carrying hidden instructions / prompt injection
  • Drop the irrelevant outright

Batch the page’s passages into one request (parallel questions) — that’s the cost-optimal shape.

Official cookbook: when your answer model quotes a source, one Choice decides “does the quoted context support the claim,” and low confidence flags the citation for review — a direct fix for hallucinated citations.

Approach Cost Consistency Explainability
Keywords/regex Minimal Rigid, easy to evade Rules are the explanation
LLM per item High Drifts with prompts Requires reading output
Jev Score Low Fixed levels probabilities + legend
  • Level descriptions are policy documents — draft them like legal criteria; 3–4 levels usually operate best within the 2–10 range
  • Replay historical labeled data before launch, check agreement between score and human labels, then set thresholds
  • Classification feeds “publish/review/remove” — it isn’t the end of the pipeline; keep full logs (with confidence) for appeals

Further: the official hierarchical-classification cookbook runs parallel beam search over deep patent/retail/biomedical/source-code trees. Back to the overview, or on to Tool approval.