Data Engineer

Ampcus Inc. is a certified global provider of a broad range of Technology and Business consulting services. We are in search of a highly motivated candidate to join our talented Team.

Job Title: Data Engineer

Location(s): Dunwoody, GA

• Design, develop, and maintain robust and scalable data pipelines using Azure Data Factory (ADF).
• Build and optimize data warehousing solutions on Snowflake, ensuring performance, scalability, and data quality.
• Leverage Databricks for data processing, transformation, and advanced analytics using Spark and Python/Scala.
• Implement and manage data orchestration and scheduling using Control-M.
• Collaborate with data scientists and analysts to understand their data requirements and provide efficient data solutions.
• Monitor and troubleshoot data pipelines and data warehouse performance, ensuring data integrity and reliability.
• Implement data quality checks and validation processes to ensure accuracy and consistency of data.
• Develop and maintain data models and schemas for optimal data storage and retrieval.
• Implement and maintain data security and governance policies.
• Stay up-to-date with the latest advancements in Azure cloud technologies and data engineering best practices.
• Document data pipelines, data models, and ETL processes.
• Participate in code reviews and contribute to the development of data engineering standards and best practices.
• Troubleshoot and resolve data-related issues in a timely manner.

• Azure Cloud: Deep understanding of Azure cloud services, particularly in the data and analytics domain.
• Azure Data Factory (ADF): Proven experience in designing, building, deploying, and managing complex data pipelines using ADF, including data flows and mapping data flows.
• Databricks: Hands-on experience with Databricks, including Spark programming (Python or Scala), Delta Lake, and optimizing Spark jobs for performance.
• Snowflake: Extensive experience in designing, developing, and administering data warehouses on Snowflake, including data modeling, SQL development, performance tuning, and security features.
• Control-M: Experience in using Control-M for workflow orchestration, scheduling, monitoring, and managing batch processes.
• SQL: Strong proficiency in SQL for data querying, manipulation, and analysis across different database systems.
• Data Modeling: Solid understanding of different data modeling techniques (e.g., relational, dimensional).
• ETL/ELT Concepts: Comprehensive understanding of ETL and ELT principles and best practices.
• Scripting: Proficiency in at least one scripting language such as Python for automation and data manipulation.
• Version Control: Experience with Git and related version control workflows.
• Data Quality: Understanding of data quality principles and experience implementing data quality checks and processes.

• Experience with other Azure data services such as Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure Event Hubs, and Azure Functions.
• Knowledge of data governance frameworks and tools.
• Experience with CI/CD pipelines for data engineering deployments.
• Familiarity with agile development methodologies.
• Experience with data visualization tools (e.g., Power BI, Tableau).
• Knowledge of NoSQL databases.

• Microsoft Certified: Azure Data Engineer Associate.
• Snowflake SnowPro Core Certification.
• Databricks Certified Associate Developer for Apache Spark.
• Control-M certification.