How to Use Jev for Social Listening: API Request, 7-Step Recipe and the Questions That Worked
In one line: To use Jev for social listening, send each forum post as structured state plus three narrow questions (relevance, sentiment, intent) to one endpoint, tune the thresholds on your own labelled sample, and use its confidence to decide which answers to trust. In our tests that cost about $4.50 per 100,000 posts at a median 0.36 seconds per post.
This is the setup guide behind our Jev vs Claude Haiku benchmark. Every step below comes with the measured evidence that made us adopt it.
What is Jev, in three sentences?
Jev is TypeSafe AI’s “System One” decision model: you give it structured state and a question, and it returns a typed judgment with probabilities. The question types we used are a Score (a graded scale), a Noul (a yes-probability) and a Choice (pick one option). It does not generate text, which makes it fast and cheap but means it only ever labels what you ask it about.
What does a Jev API request look like?
One POST per post, with three questions evaluated in parallel on the same state. The key below is a placeholder; keep real keys in an environment variable.
curl https://api.typesafe.ai/v1/systemone \
-H "Authorization: Bearer $TYPESAFE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "jev-1.13.0",
"state": {
"keyword": "Tesla",
"thread_title": "(thread title, for context)",
"opening_post": "(first message of the thread, 600 characters at most)",
"quoted_post": "(earlier message that post replies to, if any)",
"post": "(the text to judge)"
},
"questions": {
"sentiment": { "type": "score", "instructions": "...", "criteria": [ "5 level descriptions" ] },
"relevant": { "type": "noul", "instructions": "...", "criteria": { "true": "...", "false": "..." } },
"intent": { "type": "choice", "instructions": "...", "criteria": { "complaint": "...", "praise": "...", "question": "...", "other": "..." } }
}
}'The response gives you the answers and the token usage:
answers.sentiment.score, from 0 to 4, plusconfidenceanswers.relevant.noul, from 0 to 1answers.intent.choice, plusconfidenceusage.input_tokens, andmodel(the version that actually answered)
Subtract 2 from the sentiment score to get −2 to +2. The full wording of all three questions is in the copyable block below. See TypeSafe’s own docs at docs.typesafe.ai for the authoritative API reference.
The 7-step recipe, and the evidence behind each step
1. Ask narrow, separate questions in one request
Relevance, sentiment and intent are three questions, not one. They are evaluated in parallel on the same state, so asking three costs one round trip. A single vague “is this good for the brand?” question hides exactly the disagreements you want to see.
2. Send structured state, and judge only one field
Our state has five fields: keyword, thread_title, opening_post (context, up to 600 characters), quoted_post (context) and post (the only text to judge). Quoted replies need this structure. On HardwareZone, replies embed “X said: … Click to expand…”. Sending the quote and the reply as separate fields raised Jev’s sentiment accuracy from 62.5% to 70.0% and intent from 70% to 75%; relevance moved from 97.5% to 92.5%. Deleting the quote entirely was worse, because short replies lose their meaning.
Context helps with sarcasm too: adding the thread’s opening post fixed one sarcastic Cantonese reply that an earlier version read as positive.
3. Use a Score for sentiment, not a four-way Choice
Most disagreements are about mild tone. A 5-level Score lets you set your own neutral band afterwards, instead of letting the model decide where “slightly negative” ends.
4. Tune thresholds on your own labelled sample, with a held-out half
This is the single biggest lever. On 300 LIHKG posts, mapping the Jev score at a naive ±0.5 gave 64.0% sentiment accuracy; tuned thresholds gave 78.7% on held-out rows. Jev shades neutral text slightly negative, so the negative cut-off must be stricter. Our final values were sentiment at or below −1.25 is negative, at or above +0.75 is positive, and relevance yes-probability at or above 0.55. Tune on half your labelled rows and report on the other half, as we did. Your numbers will differ.
5. Write your definitions into the criteria
Jev reads literally. With our earlier intent definition it labelled criticism of a government minister as a product “complaint”. Adding “criticism of individual people, politicians or the government is other” lifted intent accuracy from 82.7% to 85.3% (titles 87% to 91%). Decide explicitly what counts as relevant, whether bad news is negative, and what a complaint is. Definitions moved our results more than the choice of model did.
6. Use confidence as a second axis
Over 300 LIHKG posts with the final questions:
| Label | Rule | Share of posts | Accuracy |
|---|---|---|---|
| Relevance | yes-probability at most 0.2 or at least 0.8 | 82% | 96.7% |
| Sentiment | confidence at least 0.7 | 60% | 92.2% |
| Sentiment | confidence at least 0.5 | 82% | 86.1% |
Auto-accept the confident answers and send the rest to a human or a second model. In the benchmark, a hybrid of Jev when confident and Claude Haiku otherwise matched Haiku on sentiment (82.7%) and relevance (95.3%) and scored 91.3% on intent against Haiku’s 90.0%, with Jev handling 51%, 83% and 77% of rows respectively.
7. Pin the model version
Use jev-1.13.0, not jev-latest, so answers do not move when the alias moves. The response echoes the version that answered, so you can log it. Re-run your held-out check whenever you upgrade.
A related check: Jev’s docs warn that it can favour the first Choice option. We reversed the sentiment option order on 105 posts and 98.1% of answers were unchanged, so the effect was small for us, but it is cheap to test on yours.
The full v3 question definitions
These are the exact questions behind the benchmark numbers, ready to copy. The same context sentence is appended to each question.
CONTEXT (appended to each question):
`post` is a forum message or a thread title. `opening_post`, when present, is the first message of the thread, and `quoted_post`, when present, is an earlier message that `post` replies to. Use them only as context; judge `post` only.
SENTIMENT (Score, 5 levels; value minus 2 gives -2..+2)
instructions: How negative or positive is `post`? Count both the feeling it expresses and the news it reports: a report of a bad event is negative even when written neutrally, and good news is positive.
1. Clearly negative: complains, criticises, mocks, is angry or disappointed, or reports serious bad news (deaths, injuries, crimes, scandals, big losses)
2. Slightly negative: mild criticism or doubt, or reports minor bad news (price rises, small problems, falling share prices)
3. Neutral: factual information with no good or bad side, a question, or no feeling
4. Slightly positive: mild approval, or reports minor good news (discounts, free rides, small gains)
5. Clearly positive: praises, recommends or is excited, or reports clearly good news
RELEVANCE (Noul)
instructions: Does `post` concern `keyword`: an opinion, information, news or a question about it or its products, prices, stores, stations, vehicles or staff, including incidents that happen in or involve them?
true: The post concerns `keyword`, even if `keyword` is only the place, vehicle or company where an incident happened.
false: `keyword` is not involved: the post is about another brand, a different topic, or other people in the conversation (for example a reply mocking a politician that says nothing about `keyword`).
INTENT (Choice)
complaint: Criticises or expresses dissatisfaction with a product, brand or company's service. Criticism of individual people, politicians or the government is `other`, not complaint.
praise: Praises or recommends a product or brand, or expresses satisfaction with it.
question: Mainly asks a question.
other: Anything else: news, information, prices, advice, jokes, remarks or criticism about people or the government, or a neutral view.How much does the Jev API cost in practice?
Jev’s list price is $0.042 per million input tokens, with output tokens free. Measured on our setup:
Jev (jev-1.13.0) | Claude Haiku 5.5 | |
|---|---|---|
| Tokens per post | 1,063 input (3 questions, output free) | 307 input + 151 output (batched 15 per request) |
| Cost per post | $0.000045 | $0.000106 |
| Cost per 100,000 posts | about $4.50 | about $10.60 |
| Latency | median 0.36 s per post (p90 0.41 s) | median about 15 s per 15-post batch |
The question text is sent with every request, so longer criteria cost more tokens. We did not test how much accuracy shorter criteria would give up.
What rate limits apply?
TypeSafe’s docs list 80 requests per second and 100K tokens per second per account. At about 1,063 tokens per request, the token limit allows roughly 94 requests per second, so the request limit binds first. Cap your concurrency below it, and add retry with backoff for 429 responses.
What pitfalls should you plan for?
- Literal reading. Jev did what the definition said, not what we meant. Test edge cases (politicians, brand as location) and write them into the criteria.
- Sarcasm. Both models miss some. 「加價900不跌反升」 praises the iPhone 17 for holding its value but reads like a complaint about a price rise. Opening-post context fixes some cases, not all.
- CJK and non-English text. TypeSafe’s docs say English is the primary training language and CJK is “handled but not equally well”. In our main test Claude Haiku 5.5 was 3 to 5 points ahead on Cantonese, and in a small Thai pilot Haiku also led. Measure on your own language before you commit.
- Default thresholds. Do not judge Jev on naive cut-offs; see step 4.
- Small samples. Our main result rests on 150 held-out rows (noise roughly ±6 points) from one forum.
Can I skip the setup and get labelled rows directly?
Yes. The same three labels are an optional “AI sentiment labels” switch on our LIHKG actor on Apify. It uses Jev as the engine and costs $0.002 per labelled row, with the first 20 rows free per run on the free plan, effective 2026-10-27. We chose Jev for cost, speed, calibrated confidence and no extra vendor account, given the small accuracy gap. This guide is independent: neither TypeSafe nor Anthropic endorses or sponsors it. For more on the forum side, see the Hong Kong forums hub.
From least effort to most freedom
FAQ
How do I call the Jev API? Send a POST request to https://api.typesafe.ai/v1/systemone with an Authorization Bearer header, a JSON body containing the model (we pin jev-1.13.0), a state object with your structured fields, and a questions object. Each question has a type (score, noul or choice), instructions and criteria. The response returns typed answers, such as answers.sentiment.score with a confidence, plus usage.input_tokens.
How much does the Jev API cost in practice? Jev is priced at $0.042 per million input tokens with output tokens free. In our setup, three questions per post used 1,063 input tokens, which is about $0.000045 per post or about $4.50 per 100,000 posts. Claude Haiku 5.5 on the same task cost about $0.000106 per post, roughly $10.60 per 100,000.
What are Jev’s rate limits? TypeSafe’s docs list 80 requests per second and 100K tokens per second per account. Cap your concurrency below that. In our tests the median latency was 0.36 s per post (p90 0.41 s), so each worker handles roughly three posts per second.
Should I pin the Jev model version? Yes. We pinned jev-1.13.0 rather than the jev-latest alias so that answers do not shift when the alias moves. The response also echoes which model version answered.
Can Jev write text? No. Jev is a System One model that returns typed answers and probabilities. It does not generate text, so use it for labels such as relevance, sentiment and intent, not for summaries.
Does Jev work for Chinese and Cantonese forums? It worked usefully in our tests on LIHKG, but TypeSafe’s docs say English is the primary training language and CJK is handled but not equally well. In our benchmark Claude Haiku 5.5 was ahead by 3 to 5 points on 150 held-out Cantonese posts, so measure it on your own sample.
Further reading
- Jev vs Claude Haiku: an independent sentiment benchmark: the full results, calibration and caveats behind the recipe above.
- Build a no-code forum sentiment pipeline with Apify + n8n + GPT: wire labels into a scheduled workflow.
- Monitor Taiwan’s Bahamut Gaming Forum in English with Claude Code + Apify MCP: the ask-in-chat alternative to a scheduled pipeline.
Chad runs 30+ published Apify actors, including the LIHKG scraper whose posts were used here. Numbers are from a run on 2026-10-12.