Data Consultant

The role

You will design, build, and maintain data pipelines and infrastructure. You will work on end-to-end data solutions and collaborate with cross-functional teams.

What you will do

• Design, develop, and maintain scalable ETL pipelines.
• Build and optimize data storage solutions like data lakes and data warehouses.
• Integrate structured and unstructured data from various systems.
• Ensure data accuracy, consistency, and completeness through validation and cleansing.
• Maintain documentation for data processes, tools, and systems.
• Collaborate with product managers and stakeholders to gather requirements.
• Participate in Agile development processes including sprint planning and daily stand-ups.
• Conduct testing and debugging to ensure reliability, security, and performance.
• Write unit and integration tests.
• Propose innovative solutions to improve application performance, security, and scalability.

Required qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• Over 3 years of experience as a Data Engineering.
• Proficiency in Python programming language.
• Experience with database technologies such as SQL (e.g., MySQL, PostgreSQL) and NoSQL (e.g., MongoDB) databases.
• Strong understanding of Programming Libraries/Frameworks and technologies such as Flask, API frameworks, datawarehousing/lakehouse, principles, database and ORM, data analysis databricks, panda's, Spa
• Knowledge of utilities and tools: logging, requests, subprocess, regex, pytest.
• Knowledge of ELK stack, Redis, distributed task queues.
• Strong understanding of data warehousing/lakehousing principles and concurrent/parallel processing concepts.
• Familiarity with at least one cloud data engineering stack (Azure, AWS, or GCP).
• Familiarity with version control systems like Git and collaborative development workflows.
• Competence in working on Linux OS and creating shell scripts.
• Solid understanding of software engineering principles, design patterns, and best practices.

Before you apply

• This is specifically for USA location only.

About the company

Particle41 is driven by core values of Empowering, Leadership, Innovation, Teamwork, and Excellence.