Identity
Description
Test any LLM or agent in Mandarin, Korean, Norwegian, German, Dutch, Spanish and Portuguese before you route real traffic to it. Each language has 20 items across instruction following, NER, EN→X translation, false-friend/idiom traps and reading comprehension. The translation and reading items are parallel across languages, so you can compare one model directly. Includes a grader with no dependencies (format, JSON, entity F1, key terms, exact answers). All items are original, with no contamination from public benchmarks.
Tags
evaluation, eval-suite, multilingual, benchmark, ner, translation, instruction-following, llm-testing, jsonl
Use cases
Test LLMs/agents in 7 languages before routing traffic, Regression-test multilingual quality after model upgrades
Technical
Prompt count
140
File size
12 KB