AI Benchmark Engineer | Native Language Specialist - German (Germany) - Remote

Germany (Remote). Posted 8 months ago

The role

You will design, build, and validate Terminal-Bench tasks to test large language models on multilingual software challenges. This involves creating task environments, prompting, and implementing verifier scripts to measure multilingual robustness.

What you will do

Evaluate coding agents through task engineering
Build realistic task environments using native language datasets and files
Find failure points in AI using native language prompting and translation
Support development of reference implementations
Write reliable, deterministic verifier scripts
Analyze execution logs and calibrate task difficulty
Participate in a 4-layer human quality control process

Required qualifications

1+ years of industry experience in software or prompt engineering
Proven track record at leading technology companies and/or graduation from top-tier engineering universities
Native or near-native fluency in German
High English proficiency
Strong proficiency in Python, standard shell scripting, and data processing
Extensive experience with Terminal/CLI-based development workflows
Working familiarity with coding agents
Deep technical understanding of multilingual text processing pitfalls including encoding/decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe s

Nice to have

Bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts

Benefits

Work when you want
No fixed hours, no check-ins, no micromanaging
Competitive rates
Prompt payments
Access to diverse, innovative projects

Before you apply

Not ideal as a full time job or primary income source
Work availability fluctuates with project demand
Requires reliable availability and commitment
Cannot engage contractors in regions subject to international embargo or sanctions
Submit an updated copy of your CV in English

About the company

LILT's mission is to make the world's information available to everyone, no matter the language they speak. They provide multilingual AI and human-verified services to Enterprises, Governments, and AI Developers.