LLM Engineer (Data Generation)

The role

This person will design, generate, evaluate, and improve training data to enhance model performance from a Data-Centric AI perspective. They will build data generation pipelines and define data requirements by analyzing model performance bottlenecks and failure cases.

What you will do

• Design and generate data to improve model performance
• Collaborate with Research and Model Training teams to define data requirements
• Design, generate, and refine training data such as Instruction, Preference, Reasoning, and Domain-specific Data
• Analyze the impact of generated data on model performance and iteratively improve generation strategies
• Design and operate automated training data generation pipelines including Synthetic Data
• Build large-scale data generation workflows
• Optimize pipelines considering generation cost, processing speed, and quality
• Define quantitative and qualitative evaluation criteria for data quality and training effectiveness
• Verify data quality using LLM-as-a-Judge, Rule-based Validation, and Human Feedback
• Develop iterative generation and filtering strategies for data quality improvement

Required qualifications

• 3+ years of experience in LLM, Machine Learning, or Data Generation
• General understanding of deep learning, machine learning, and natural language processing
• Understanding of model training data composition, preprocessing, quality evaluation, and training application
• Python-based data processing and automation development capabilities
• Experience in processing, refining, filtering, and quality management of large-scale training datasets or equivalent capabilities
• Experience in Synthetic Data generation, Data Evaluation, or Data Curation using LLM
• Understanding or practical experience with LLM API and Prompting/Generation Strategy
• Excellent problem-solving, collaboration, and communication skills

Nice to have

• Experience in building large-scale LLM training data such as LLM Pre-training, Supervised Fine-tuning(SFT), Preference Optimization(DPO/RLHF/RLAIF)
• Experience in LLM Evaluation Framework (OpenAI Evals, LM Evaluation Harness, DeepEval, etc.) or building proprietary evaluation systems
• Experience in designing high-difficulty LLM data such as multi-turn conversation, Tool Calling, Agent, Reasoning, and Code Generation
• Experience in building Python-based data pipelines (Airflow, Ray, Spark, etc.) or large-scale data processing environments
• Contribution to Generative AI, LLM, NLP open-source projects or experience in paper/technical presentations

Before you apply

• Coding and assignment test
• 1st interview (video, approx. 1 hour)
• 2nd interview (in-person or video, approx. 3 hours)
• Reference check may be conducted after the interview process with candidate consent
• A 3-month probation period may apply

About the company

42dot is a company working on next-generation Generative AI model performance improvement.