
Senior Data Engineer
RM 6,000 - RM 7,999 / month
Checking job availability...
Original
Simplified
Job Responsibilities: Data pipeline construction: Design, build and maintain data pipelines for data collection, storage, processing and analysis. Ensure data pipelines are efficient, scalable, and secure. Data warehouse and database management: Design and build data warehouses to support complex data extraction, transformation, and loading (ETL) processes.(from DA or user’s requirement) Manage and optimize SQL and NoSQL database systems. Data integration: Integrate data from different data sources to ensure data consistency and quality. Work with data scientists and analysts to provide data support and optimize data query performance. Automation and monitoring: Automate data processing tasks. Monitor the performance of data pipelines and promptly address issues in the data flow. Qualifications: Bachelor's degree in Computer Science, Data Engineering, or a related field. Advanced degree preferred. Proven experience as a data engineer with expertise in data pipeline development, ETL processes, and data architecture. Proficiency in programming languages such as Python. Database management: Familiar with SQL and NoSQL databases, such as PostgreSQL/ClickHouse and MongoDB . Big data technology preferred: Hadoop, Spark, Flink, etc. Data modeling and warehouse technology preferred: Familiar with the design and implementation of data warehouses, such as using Snowflake, Redshift, etc. Devops experience preferred:Docker, Kubernetes. Cloud platform: Proficient in using cloud services such as AWS, Azure, and Google Cloud. Workflow management: Apache Airflow, etc Excellent problem-solving skills and the ability to work collaboratively in a team. Passion for gaming and a deep understanding of gaming data is a plus.