AI Benchmark Engineer | Native Language Specialist - Dutch - Remote

Netherlands (Remote). Posted a month ago

The role

You will design, build, and validate Terminal-Bench tasks to test large language models on multilingual software challenges. This involves creating task environments, prompting, and implementing verifier scripts in Dutch.

What you will do

Evaluate coding agents through task engineering
Build realistic task environments using native language datasets and files
Find failure points in AI using native language prompting and translation
Support development of reference implementations
Write reliable, deterministic verifier scripts
Analyze execution logs and calibrate task difficulty
Participate in a 4-layer human quality control process

Required qualifications

1+ years of industry experience in software engineering
Proven track record at leading technology companies and/or graduation from top-tier engineering universities
Native or near-native fluency in Dutch with deep understanding of grammar, register, and phrasing rules
High English proficiency
Strong proficiency in Python, standard shell scripting, and data processing
Extensive experience with Terminal/CLI-based development workflows
Working familiarity with coding agents
Deep technical understanding of multilingual text processing pitfalls including encoding/decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe s

Benefits

Work when you want, as much or as little as you want
No fixed hours, no check-ins, no micromanaging
Competitive rates
Prompt payments
Access to diverse, innovative projects

Before you apply

Not ideal as a full time job or primary income source
Requires reliable availability and commitment
Cannot engage contractors in regions subject to international embargo or sanctions
Submit an updated copy of your CV in English
Complete a GenAI assessment

About the company

LILT provides multilingual AI and human-verified services to Enterprises, Governments, and AI Developers around the world. Their mission is to make the world's information available to everyone, no matter the language they speak.