Data Engineer

The role

This person will design, develop, and manage large-scale data workflows across structured and unstructured datasets. The role focuses on building RAG pipelines, orchestrating ETL/ELT processes, and deploying scalable data systems in AWS.

What you will do

• Design, build, and maintain RAG pipelines, including document ingestion, indexing, embedding workflows, and model retrieval flows.
• Develop and manage structured and unstructured data pipelines supporting analytics, ML, and application workloads.
• Build and optimize ETL/ELT pipelines in AWS using services such as S3, Lambda, Step Functions, EMR, Glue, ECS/EKS, and IAM best practices.
• Implement and operate NiFi flows for high-throughput, low-latency data ingestion and transformation.
• Develop, orchestrate, and schedule workflows using Prefect.
• Implement indexing, search, and retrieval patterns using ElasticSearch.
• Collaborate closely with architecture, ML, and application teams to support scalable data solutions.
• Ensure data quality, lineage, governance, and security across all pipelines.
• Monitor system performance and troubleshoot issues across distributed data systems.

Required qualifications

• Solid understanding of ETL/ELT processes and data modeling best practices.
• Hands-on experience implementing workflows in Prefect.
• In-depth knowledge of ElasticSearch indexing, cluster management, and search optimization.
• Proficiency in Python.
• Familiarity with common data libraries (Pandas, PySpark, requests, etc.).
• An active TS/SCI federal security clearance is required.

Nice to have

• Strong experience with Apache NiFi for data flow management and real-time ingestion.
• Experience building or maintaining RAG pipelines (e.g., vector databases, embeddings, document chunking strategies, retrieval optimization).
• Proven ability to manage structured and unstructured data pipelines at scale.
• Experience with AWS cloud services for data engineering.
• Strong version control and CI/CD experience.
• Experience with vector databases (OpenSearch, Pinecone, Weaviate, etc.)
• Familiarity with containerized workflows (Docker, Kubernetes)
• Experience supporting LLM or generative AI production systems
• Background in distributed systems, streaming platforms, or data mesh architectures
• Prefect 2.0 preferred

Benefits

• hands-on experience
• certifications
• industry training

Before you apply

• An active TS/SCI federal security clearance is required.
• Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States.
• Candidates who are currently employed by a client of Accenture Federal Services or an affiliated Accenture business may not be eligible for consideration.

About the company

Accenture Federal Services is a technology company within global Accenture that helps the US federal government. Its 13,000+ people pursue technology and ingenuity for clients across defense, national security, public safety, civilian, and military health organizations.