Epicareer Might not Working Properly
Learn More

Data Engineer Team Lead

Salary undisclosed

Apply on


Original
Simplified

About Deriv

Deriv, a leading player in the online trading industry, is on a mission to make trading accessible to anyone, anywhere. For 25 years, our innovative products and services have attracted a dedicated customer base of more than 2.5 million people worldwide.

A career at Deriv will give you the opportunity to join a Platinum-accredited Investors in People (IIP) organisation and a certified Great Place to Work (GPTW). You’ll be part of a global team of more than 1,400 exceptional professionals across 25 offices. Join us on our mission to shape the future of online trading and be part of a dynamic team driving innovation and excellence in the financial markets.

To find out more about life at Deriv, check out our careers page here.

Your role

As the Data Engineer Team Lead at Deriv, you will play a crucial role in overseeing the design, development, and optimization of our data pipelines, ensuring seamless integration and availability of data across all systems. You will lead a talented team of data engineers and spearhead the execution of data strategies that are integral to supporting critical business decisions and operations. A significant responsibility is to develop a unified data pipeline that effectively serves both internal systems and external applications, ensuring secure and real-time data flow across all platforms.

Your key responsibilities

  • Team Leadership: Lead and mentor a team of data engineers to design, build, and optimize scalable data pipelines for analytics, reporting, and operational needs.

  • Unified Data Pipeline Management: Create and manage a unified data pipeline that integrates multiple data sources, facilitating efficient access for both internal and external systems, including third-party tools and client-facing platforms.

  • Utilization of Low-Code/No-Code Tools: Leverage low-code/no-code integration tools such as Fivetran or RudderStack to accelerate data workflows and improve integration efficiency.

  • Cross-Functional Collaboration: Work closely with various teams to identify data needs and ensure smooth data flows between systems, enhancing high availability and integration.

  • Data Governance and Quality Assurance: Implement and manage data governance and quality standards, ensuring data consistency and reliability throughout the organization.

  • Real-Time and Batch Data Architecture: Architect and oversee the implementation of real-time and batch data pipelines using data streaming tools like Kafka or Pub/Sub, ensuring low-latency, high-performance data processing.

  • Technical Guidance and Support: Provide technical guidance to your team, aiding in troubleshooting complex data issues and optimizing performance and cost-effectiveness while adhering to best practices.

  • Continuous Improvement: Stay informed on the latest in data engineering technologies and best practices, especially concerning real-time processing and low-code/no-code integrations, to enhance our data infrastructure continually.

  • Stakeholder Coordination: Communicate with stakeholders to ensure timely delivery of data solutions that effectively meet business requirements.

Requirements

What is needed

Education:

Bachelor's degree in Computer Science, Data Engineering, or a related field.

Experience:

  • 10+ years in data engineering, with at least 3+ years in a management or leadership role.

  • Expertise in designing, building, and maintaining data pipelines and data warehouses.

Technical Skills:

  • Strong knowledge of cloud platforms (Google Cloud or AWS) and familiarity with tools like BigQuery, PostgreSQL, and Airflow.

  • Proficient in programming languages such as Python, with experience in ETL/ELT processes.

  • Hands-on experience with low-code/no-code data integration tools.

  • Deep understanding of real-time data processing and streaming technologies such as Kafka or Pub/Sub.

  • Understanding of Data Frameworks: Experience in building unified data pipelines for secure and consistent data flows, working with both real-time and batch processing systems.

  • Governance Knowledge: Strong understanding of data governance, security, and compliance best practices.

  • Communication Skills: Excellent communication abilities with a talent for explaining complex technical concepts to non-technical audiences and mentoring junior team members.

  • Bachelor's degree in computer science, data engineering, or a related field.

Why Join Us

At Deriv, we foster a culture of innovation and continuous improvement. As our Data Engineer Team Lead, you will have the opportunity to drive significant projects, lead a talented team, and make a real impact on our data infrastructure and business outcomes. If you're passionate about data engineering and are looking for a stimulating environment to grow and lead, we would love to hear from you!