Data Engineer
We are looking for skilled Data Engineers at Associate and Senior Associate levels to build and manage scalable data pipelines in a modern Azure-based data platform. This role involves working on data integration, transformation, and ensuring high data quality in regulated environments.
Key Responsibilities: Design, develop, maintain, and optimize ETL/data pipelines using Python and SQL Server for data ingestion, transformation, and integration from diverse sources such as databases, structured files, and REST APIs. Write clean, efficient, and scalable Python code for data processing, automation, and integrations, following best practices and coding standards. Develop, manage, and optimize SQL queries, views, and stored procedures to support business logic, reporting, and high-performance data processing. Build and manage data workflows using Azure Data Factory, Azure SQL, Blob Storage, and related Azure services, ensuring reliability and scalability. Monitor and troubleshoot data pipelines, performance issues, and data quality problems, ensuring minimal downtime and high system efficiency. Ensure data quality, integrity, lineage, and compliance with organizational and regulatory standards (e.g., HIPAA, SOC2). Collaborate with cross-functional teams including data analysts, software engineers, and DevOps to understand requirements and align data solutions with business needs. Work on API integrations and upstream/downstream system connectivity, including exposure to application ecosystems (e.g., .NET-based services). Participate in and contribute to code reviews, design discussions, and best practice implementation to improve overall engineering quality. Drive continuous improvement, automation, and performance optimization across data pipelines and workflows. Support and contribute to data governance, documentation, and standardization of processes and architectures. (For experienced candidates) Provide technical guidance, mentorship, and leadership to junior team members and contribute to architectural decisions.
Preferred Skills: Strong proficiency in Python and SQL (T-SQL, SQL Server) Experience in ETL pipeline development and data integration Hands-on exposure to Azure Data Services (ADF, Azure SQL, Blob Storage) Experience with REST APIs / external data sources Knowledge of data quality, validation, and pipeline monitoring Familiarity with Git/version control Strong analytical and problem-solving skills Experience with Azure Synapse, Azure DevOps, Azure Functions Knowledge of data modeling and warehousing concepts Exposure to Power BI or data visualization tools Understanding of data security and compliance standards Familiarity with .NET / API-based integrations
