Apply on
Job Scope / Position Summary
We are looking for a skilled Data Engineer with expertise in data modelling and data pipeline development. As a Data Engineer/Analyst, you will work with the big data team, adopting best practices to develop and optimize data pipelines, reporting and dashboarding to meet business requirements. Proficiency in serverless cloud services like AWS Glue, Aurora, Lambda, RDS and Redshift. Handle Management Information Systems (MIS) to generate reports and develop dashboards using business intelligence tools.
Main Responsibility
Data Pipeline Development
Discuss with Solution Architect on designing data pipelines
Develop data pipelines with proposed solution
Manage and maintain data pipelines
Design and optimize of the ETL/ELT architectures and performances
Develop efficient and scalable data processing workflows
Monitor data pipelines, detect issue and fix
Data Modelling
Create and maintain data models, including entity-relationship diagrams, to represent data structures and relationships
Design database schemas that optimize data storage, retrieval and query performance
Improve query performances
Optimize on database storage
Reporting and Visualization
Timely Reporting
Develop reports, dashboards and visualizations that effectively communicate data-driven insights
Ensure the reports are delivered on schedule and in a timely manner
BAU Tasks
Understanding and translating business requirements into technical requirements
Develop/Enhance SAP reports based on business requirements
Develop/Enhance Dashboards based on business requirements
Fix bug whenever detected
Deploy changes into GitHub
Deliver reports to business team.
Ensure data accuracy and consistency
Qualification and Experience Requirement
Education : Bachelor’s Degree in Computer Science, Engineering, or related field; Master’s degree preferred
Experience : 3 years of hands-on experience of data integration and analytics in a corporate or consulting setting
Skills :
- Design, develop and maintain efficient data pipelines to support data processing, transformation and integrations needs.
- Collaborate with cross-functional teams to understand data requirements and implement solutions that are needed for business needs.
- Work with various data sources and formats, ensuring data quality and consistency.
- Monitor and troubleshoot data pipelines to ensure consistency of data flow.
- Handle Management Information Systems (MIS) including data collection, analysis and reporting for decision-making purpose.
- Able to build report by tools (SAP, PowerBI or any reputable BI tools)
- Hands on experience in AWS services (AWS Redshift, AWS Glue, AWS S3, AWS Lambda, AWS RDS, AWS AuroraDB, AWS API Gateway)
- Coding language: Python, PySpark, SQL, Linux
- Other skills: ETL/ELT skills, Architecture modelling, Data Modelling, Dashboarding, Story Telling, Documentation