Apply on
PURPOSE
A role to be responsible for system design, development and implementation of regional frontend systems. Provide maintenance and support on production systems.
KEY ACCOUNTABILITIES
•Design, develop, document and implement end-to-end data pipelines and data integration processes, both batch and real-time. This include data analysis, data profiling, data cleansing, data lineage, data mapping, data transformation, developing ETL / ELT jobs and workflows, and deployment of data solutions.
•Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness, and to optimize and fine-tune ETL / ELT processes.
•Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of ETL / ELT processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards.
•Prepare test data, assist to create and execute test plans, test cases and test scripts.
•Collaborate with Data Architect, Data Modeler, IT team members, SMEs, vendors and internal business stakeholders, to understand data needs, gather requirements and implement data solutions to deliver business goals.
•BAU support for any data issues and change requests, document all investigations, findings, recommendations and resolutions.
QUALIFICATIONS / EXPERIENCE
•Bachelor in IT, Computer Science or Engineering.
•At least 3-5 years of using Big Data technologies like Azure and AWS Big Data Solution, Hadoop, Hive, HBase, Spark, Sqoop, Kafka and Spark Streaming.
•Minimum 5 years of professional experience in data warehouse, operational data store, and large scale data architecture implementations in Unix or/and Windows environment.
•At least 3+ years of solid hands-on development experience with ETL development to transform complex data structure in multiple data sources environment.
•At least 5 years data model (relational and/or data warehouse), data mart design and implementation.
•Minimum 3-5 years ETL programming in any of these languages including Python, Scala, Java or R
•Experience on Azure Databricks for ETL/ELT development and big data analytics programming in Python
•Strong Experience with various of ETL/ELT frameworks, data warehousing concepts, data management framework and data lifecycle processes.
