AI Benchmark Engineer | Native Language Specialist - French (France) - Remote

The role

You will design, build, and validate Terminal-Bench tasks to test large language models on multilingual software challenges. This involves creating task environments, prompting, and implementing verifier scripts in your native language.

What you will do

• Evaluate coding agents through task engineering.
• Build realistic task environments using native language datasets and files.
• Find failure points in AI using native language prompting and translation.
• Support development of reference implementations.
• Write reliable, deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty.
• Participate in a 4-layer human quality control process.

Required qualifications

• 1+ years of industry experience in software engineering.
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
• Native or near-native fluency in French.
• Deep understanding of French grammar, register, and phrasing rules.
• High English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows.
• Working familiarity with coding agents.
• Deep technical understanding of multilingual text processing pitfalls including encoding/decoding robustness and Unicode normalization.
• Deep technical understanding of locale-dependent conventions.
• Deep technical understanding of Text I/O, toolchain interoperability, and safe string operations.

Benefits

• Work when you want, as much or as little as you want.
• No fixed hours, no check-ins, no micromanaging.
• Competitive rates.
• Prompt payments.
• Access to diverse, innovative projects.

Before you apply

• Not ideal as a full time job or primary income source.
• Requires reliable availability and commitment.
• Cannot engage contractors in regions subject to international embargo or sanctions.
• Submit an updated copy of your CV in English.
• Complete a GenAI assessment.

About the company

LILT's mission is to make the world's information available to everyone, no matter the language they speak. They provide multilingual AI and human-verified services to Enterprises, Governments, and AI Developers.