Technical Lead-Data Engineering

Your Role Our clients need expert advise and guidance in implementing solutions for managing and leveraging their data for improving their business. Your primary goal will be to help clients implement data solutions that drive business improvement. You will play a crucial role in understanding client requirements, proposing suitable solutions, and delivering them successfully. As the Lead Engineer, you will oversee end-to-end ETL processes, manage a team, and collaborate with various stakeholders to ensure effective data extraction, transformation, and loading into a Kimball data warehouse. You'll employ technologies such as Python, SQL (MSSQL, MySQL, Hive, PostgreSQL), Apache Spark (PySpark), and orchestration tools like Apache Airflow and Kubeflow. Your role also involves containerization using Kubernetes, Helm, and Kubedirector to maintain data quality and compliance for data- driven decision-making. Your Responsibilities • Take ownership of the proposed data solutions and drive their successful implementation. Work closely with cross-functional teams, including data engineers, developers, and project managers, to ensure timely and high-quality delivery. • Establish strong relationships with clients and serve as their trusted advisor. Understand their pain points and objectives, and provide strategic guidance on how data solutions can address their business challenges. • Lead and manage ETL development activities, ensuring adherence to project timelines, budgets, and quality standards. Proactively identify and mitigate risks to ensure smooth project execution. Requirements Able to lead ETL development team, plan the tasks and work collaboratively with the team. • Proficiency in programming languages such as Python, and SQL (MSSQL, MySQL, Hive, PostgreSQL) • Proficiency in data technologies (Apache Spark/PySpark, Apache Airflow, KubeFlow and S3) • Solid understanding of Kimball Data Warehouse design (dimension table, fact table, SCD tables) • Strong understanding of data management, data integration, and data governance principles. • Hands on experience with deployment on major platforms (eg: Kubernetes, Linux, Docker, Helm, Kubedirector) Please send an email with your resume to [email protected] stating the role you are applying for. Then, as per requirement, you need to answer pre-screening test.