AI Benchmark Engineer | Native Language Specialist - Japanese - Remote

The role

You will design, build, and validate Terminal-Bench tasks to test large language models on multilingual software challenges. This involves creating task environments, prompting, and implementing verifier scripts in Japanese.

What you will do

• Evaluate coding agents through task engineering.
• Build realistic task environments using native language datasets and files.
• Find failure points in AI using native language prompting and translation.
• Support development of reference implementations.
• Write reliable, deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty.
• Participate in a 4-layer human quality control process.

Required qualifications

• 1+ years in software or prompt engineering.
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
• Native or near-native fluency in Japanese with deep understanding of grammar, register, and phrasing rules.
• High English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows.
• Working familiarity with coding agents.
• Deep technical understanding of multilingual text processing pitfalls including encoding/decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe s

Benefits

• Work when you want with no fixed hours, check-ins, or micromanaging.
• Competitive rates and prompt payments.
• Access to diverse, innovative projects.

Before you apply

• Not ideal as a full time job or primary income source.
• Requires reliable availability and commitment.
• Cannot engage contractors in regions subject to international embargo or sanctions.
• Submit an updated copy of your CV in English.
• Complete a GenAI assessment.

About the company

LILT provides multilingual AI and human-verified services to Enterprises, Governments, and AI Developers. Their mission is to make the world's information available to everyone, no matter the language they speak.