Epicareer Might not Working Properly
Learn More

Data Engineering Manager

Salary undisclosed

Checking job availability...

Original
Simplified

Job Summary

  • We are seeking a Data Engineering Manager to lead the development of scalable data pipelines, ETL workflows, and analytics solutions.
  • Drive the migration of on-premises Microsoft SQL Server databases to Azure.
  • Responsible for ensuring efficient data ingestion, transformation, and storage.
  • Leverage Apache Kafka for real-time data streaming and event-driven architectures.
  • The ideal candidate has strong expertise in Azure cloud data services, Microsoft SQL Server, and data engineering best practices.
  • Experience in leading high-performing teams and managing cloud migration projects.

Key ResponsibilitiesData Engineering & Pipeline Development

  • Design, build, and maintain scalable ETL and data pipelines using Azure Data Factory and Databricks.
  • Develop real-time and batch data processing workflows for analytics and reporting.
  • Ensure data integrity, security, and governance across all pipelines.
  • Implement event-driven architectures with Apache Kafka and Azure Event Hubs.

On-Premises to Azure Migration

  • Lead the migration of Microsoft SQL Server databases to Azure SQL Database and Synapse Analytics.
  • Define and implement Azure-based data architecture for performance, scalability, and cost efficiency.
  • Optimize data storage, indexing, and partitioning strategies in Azure.
  • Work closely with DBA, DevOps, and security teams to ensure a seamless migration with minimal downtime.

Data Architecture & Infrastructure

  • Oversee the development of a modern Azure data platform (data lake, data warehouse, and real-time processing).
  • Manage and optimize Kafka-based streaming architectures using Azure Event Hubs.
  • Implement data governance, security policies, and compliance frameworks (BNM RMiT, GDPR).
  • Monitor and enhance data performance, availability, and reliability.

Data Analytics & Business Intelligence Enablement

  • Work with data analysts and scientists to develop self-service analytics and reporting.
  • Design efficient data models to support business intelligence (BI) using Azure Synapse Analytics and Power BI.
  • Enable seamless integration between operational and analytical data sources.

Team Leadership & Collaboration

  • Lead and mentor a team of data engineers, fostering best practices in Azure data engineering.
  • Promote Agile methodologies, DevOps, and CI/CD pipelines for data workflows.
  • Collaborate with engineering, product, and business teams to define data objectives.

Performance Monitoring & Continuous Improvement

  • Monitor and troubleshoot Azure data pipelines, ETL processes, and Kafka streams.
  • Identify opportunities to automate and improve efficiency.
  • Stay up to date with Azure cloud, big data, and streaming technologies.

Qualifications & SkillsTechnical Skills

  • Strong experience with Microsoft SQL Server, including query optimization and indexing.
  • Expertise in Azure Data Services (Azure SQL Database, Synapse Analytics, Data Factory, Data Lake).
  • Hands-on experience with Apache Kafka & Azure Event Hubs for real-time streaming.
  • Proficiency in Python, SQL, or Scala for data transformation and processing.
  • Experience with ETL tools (Azure Data Factory, dbt, Apache Airflow).
  • Strong understanding of data security, governance, and compliance (BNM RMiT, GDPR).
  • Familiarity with CI/CD and DevOps practices for data engineering in Azure.
  • Experience with big data frameworks (Databricks, Apache Spark).

Leadership & Soft Skills

  • Proven experience leading data engineering teams and projects.
  • Strong collaboration skills with engineering, product, and business teams.
  • Excellent problem-solving and decision-making skills.
  • Strong communication skills for both technical and non-technical stakeholders.

Preferred Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
  • 5+ years of experience in data engineering, analytics, or cloud migration, with at least 2+ years in a managerial role.
  • Experience in enterprise database migrations from on-prem SQL Server to Azure.
  • Previous experience in high-scale, real-time data streaming environments (e.g., fintech, payments, remittance).
  • Lead a cutting-edge Azure cloud data migration initiative.
  • Work in a fast-growing, data-driven fintech company.
  • Career growth opportunities.
  • Drive high-impact data projects and shape the company’s data strategy.