Outersignal
Research Engineer, Applied AI Engineering
New York City, Remote. Posted 4 months ago
The role
This is a backend-focused software engineering role specializing in applying ML/AI to user-facing products. You will work on systems that power the AI/data pipeline including prompt chaining, eval design, research APIs, model routing, and prod orchestration.
What you will do
- Build and improve production AI/data pipelines across LLMs, APIs, databases, and workflow systems.
- Design evals and build Jupyter notebooks to measure model behavior, data quality, extraction accuracy, and customer impact.
- Build observability into AI workflows to understand cost, latency, reliability, and quality.
- Experiment with new models, retrieval strategies, structured-output techniques, prompt/program architectures, and model-routing approaches.
- Integrate with research and AI APIs and build robust abstractions around them.
- Help define the foundation for future inference serving, fine-tuning, dataset generation, and model evaluation infrastructure.
Required qualifications
- Strong engineer who can reason through distributed systems, data pipelines, and databases.
- Comfortable working across varying programming languages.
- Extremely AI-fluent and actively use modern AI tools to move faster.
- Strong first-principles thinking to turn ambiguous problems into hypotheses, experiments, and shipped systems.
- Good judgment and taste to simplify aggressively and care about maintainability.
- Care about measurement using evals, tests, traces, or data.
Nice to have
- Experience with PyTorch
- Experience with Hugging Face
- Experience with vLLM
- Experience with Ray
- Experience with MLflow
- Experience with RAG
- Experience with fine-tuning & RL
- Experience with inference serving
- Experience with model evaluation systems
Benefits
- Early seat to help shape the DNA of the Applied AI Engineering org
- Meaningful equity in a venture-backed company
About the company
OuterSignal is building the customer intelligence layer for commerce. They provide a platform that enriches orders in real time with professional and personal signals to build personas and power outreach.