Use case: content classification
Problem: UGC platforms must risk-rank every submission; RAG apps must decide which retrieved passages reach the answering model. A big model per item is too expensive; keywords alone miss too much.
Jev’s role: a Score against ordered levels, with the number mapping straight to an action. “Levels are actions” is a recurring official-cookbook pattern.
Pattern 1: UGC risk tiers
Section titled “Pattern 1: UGC risk tiers”{ "type": "score", "instructions": "The risk level of this submission", "criteria": [ "Fine as-is", "Borderline — a human should look", "Clearly violating — remove" ]}The code side is unambiguous: score < 0.5 publish, < 1.5 review queue, otherwise remove and keep probabilities for appeals. Unlike keyword lists, level descriptions track your policy — update the text, no retraining.
Pattern 2: RAG passage filtering
Section titled “Pattern 2: RAG passage filtering”Official cookbook (via the llms.txt index): for each retrieved passage, ask a set of parallel questions and let code decide what reaches the answering model —
- Keep and flag passages that contradict the question (the answer model needs to know)
- Drop passages carrying hidden instructions / prompt injection
- Drop the irrelevant outright
Batch the page’s passages into one request (parallel questions) — that’s the cost-optimal shape.
Pattern 3: citation checking
Section titled “Pattern 3: citation checking”Official cookbook: when your answer model quotes a source, one Choice decides “does the quoted context support the claim,” and low confidence flags the citation for review — a direct fix for hallucinated citations.
Comparison (our framing)
Section titled “Comparison (our framing)”| Approach | Cost | Consistency | Explainability |
|---|---|---|---|
| Keywords/regex | Minimal | Rigid, easy to evade | Rules are the explanation |
| LLM per item | High | Drifts with prompts | Requires reading output |
| Jev Score | Low | Fixed levels | probabilities + legend |
Caveats (our advice)
Section titled “Caveats (our advice)”- Level descriptions are policy documents — draft them like legal criteria; 3–4 levels usually operate best within the 2–10 range
- Replay historical labeled data before launch, check agreement between
scoreand human labels, then set thresholds - Classification feeds “publish/review/remove” — it isn’t the end of the pipeline; keep full logs (with confidence) for appeals
Further: the official hierarchical-classification cookbook runs parallel beam search over deep patent/retail/biomedical/source-code trees. Back to the overview, or on to Tool approval.