Data Engineer

The role

You will design, construct, install, test, and maintain highly scalable data management systems. You will work with engineering and devops teams to build and optimize data pipelines for the Search API.

What you will do

• Design, build, and maintain efficient and reliable ETL/ELT pipelines for data warehousing.
• Develop and improve data infrastructure.
• Ensure data quality, integrity, and security across all data platforms.
• Optimize data systems for performance and scalability.
• Troubleshoot and resolve issues related to data pipelines and data infrastructure.
• Collaborate with cross-functional teams to understand data needs and deliver solutions.

Required qualifications

• Degree in Computer Science, Statistics, Engineering, or a related quantitative field.
• 3+ years of professional experience as a Data Engineer or in a similar role focused on data infrastructure.
• Proficiency in Python.
• Solid experience with relational databases (SQL) and NoSQL databases.
• Experience with AWS and their data services.
• Proven experience in building and optimizing data pipelines and data architectures.

Nice to have

• Experience with Mongo/Snowflake/Redis/S3 General/S3 Express.
• Experience with big data technologies
• Experience with Airflow.
• Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
• Experience working in a company focused on API services or search technology.
• Knowledge of data governance and data security best practices.

About the company

Tavily is a company focused on providing a Search API. They deliver search results to developers and businesses.