Content Pipeline Engineer

The role

The Content Pipeline Engineer will design and implement reusable pipelines for extracting, structuring, indexing, and retrieving information from unstructured content. This includes building document-processing capabilities for PDFs, maps, and other documents to support downstream applications and AI services.

What you will do

• Design and implement reusable ingestion and processing pipelines for unstructured content.
• Implement OCR and multimodal document-processing workflows.
• Extract text, tables, metadata, and domain-specific attributes from source documents.
• Support extraction of information from plats and survey drawings.
• Develop document parsing, classification, normalization, chunking, and metadata-enrichment processes.
• Maintain linkage and provenance between extracted information and source documents.
• Build document and vector indexes using OpenSearch or comparable technologies.
• Implement embeddings, semantic search, and retrieval pipelines.
• Develop APIs and reusable interfaces for downstream applications and AI services.
• Implement agentic workflows for classification, extraction, and quality assurance.

Required qualifications

• Bachelor's degree in Computer Science, Software Engineering, Data Science, Information Systems, or a related technical field.
• 5+ years of software engineering, data engineering, search, document-processing, or related experience.
• Strong Python development experience and experience building production-grade data or content-processing pipelines.
• Hands-on experience processing PDFs, images, scanned documents, and semi-structured content using OCR and document-extraction technologies.
• Experience with document stores, search engines, or vector databases, including embeddings and semantic retrieval.
• Experience with document parsing, classification, chunking, metadata enrichment, and preservation of source provenance.
• Experience with LLM-enabled or agentic document-processing workflows, including extraction, validation, routing, or exception handling.
• Experience building robust pipelines with automated testing, logging, monitoring, error handling, and quality controls.
• Experience evaluating OCR and extraction quality using measurable criteria and confidence thresholds.
• Strong problem-solving skills for working with complex and inconsistent document formats.

Nice to have

• Experience with OpenSearch
• Experience with AWS document-processing, AI, or machine-learning services
• Experience with agentic frameworks such as LangChain, LangGraph, or similar orchestration frameworks
• Experience with multimodal models, vision-language models, or LLM-based document extraction
• Experience building RAG or enterprise semantic-search solutions
• Experience with maps, engineering drawings, plats, survey documents, or legal descriptions
• Experience extracting bearings, distances, coordinates, or other geospatial information from documents
• Experience working in the U.S. Federal Government space
• Experience with permitting, environmental, energy, land-management, or regulatory documents

Benefits

• medical, dental and vision coverage
• 401k plan with employer contribution
• paid holidays
• vacation
• tuition reimbursement

About the company

blackcanyonconsulting is a high performing, professional service and technology focused organization.