Assistant Vice President, Big Data Engineer
Salary undisclosed
Checking job availability...
Original
Simplified
WHY JOIN US?
Job Responsibilities
WHAT YOU WILL DO:
WHO YOU ARE:
- We practice a vibrant & energetic office culture.
- We provide opportunities for career advancement within the company.
- Good performance is always rewarded accordingly.
Job Responsibilities
WHAT YOU WILL DO:
- Drive the technical design and implementation of large scale data platforms, utilizing modern and open source technologies in a cloud environment.
- Optimize data and data pipeline, as well as optimizing data flow and collection.
- Ability to provide cost effective, secure and scalable solution which is operationally efficient and
- Be able to work on complex data engineering efforts and lead technical teams through the solution design process
- Research, evaluate and formally recommend third party software and technology package.
- Stay abreast of emerging technologies and projects in the modern Data Engineering/Machine Learning space.
- Gather, analyze and process raw data at scale.
- Design and develop data applications using selected tools and frameworks as required and requested.
- Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
- Collaborate with data stakeholders and stewards on the verification and the accuracy of the information collected.
- Provide technical lead to Data Owners and Stewards on data definition, data lineage changes by supporting intake process, performing impact analysis, and conducting domain specific profiling.
- Develop custom templates to assure end to end data lineage form Source system of records to reporting data layer.
- Engage in data and metric standardization with data stakeholders and steward and solicit formal approval for adoption into data dictionary and glossary to enable end-users improved comprehension of which metrics to use and when to use them.Work closely with the engineering team to integrate your work into our production systems.
- Process unstructured data into a form suitable for analysis and analyze processed data.
- Monitoring data performance and modifying infrastructure as needed.
- Define data retention policies.
WHO YOU ARE:
- BA/BS/MS degree or equivalent experience; Computer Science or Math background preferred.
- 5+ years’ experience of Big Data platform implementation, including 3+ years of hands-on experience in implementation and performance tuning EMR/Hadoop/Spark implementations.
- Experience Architecting Big Data platforms using Apache Hadoop, Cloudera, Hortonworks and MapR distributions
- Fluency in python; familiarity with functional languages as well.
- Proficient in SQL; experience with Hive
- Strong knowledge of AWS data processing architectures and data services such as Redshift, Glue, Kinesis, EMR, Sagemaker, etc.
- AWS Solution Architect & Big Data Speciality certification will be an added advantage.
- 7+ years of experience of IT platform implementation in a highly technical and analytical role.
- Knowledge of serverless and micro services will be added advantage.
- Strategic, good analytical and problem-solving skills.
- Able to formulate technology roadmap and plan out implementation strategy in a systematic and methodological way.
- A good level of business knowledge.
- Ability to work independently with minimal supervision.
- Ability to work with all levels of personnel from business teams to senior management.
- Ability to communicate well technically, both orally and in written form.
- Ability to deal with ambiguity.
WHY JOIN US?
Job Responsibilities
WHAT YOU WILL DO:
WHO YOU ARE:
- We practice a vibrant & energetic office culture.
- We provide opportunities for career advancement within the company.
- Good performance is always rewarded accordingly.
Job Responsibilities
WHAT YOU WILL DO:
- Drive the technical design and implementation of large scale data platforms, utilizing modern and open source technologies in a cloud environment.
- Optimize data and data pipeline, as well as optimizing data flow and collection.
- Ability to provide cost effective, secure and scalable solution which is operationally efficient and
- Be able to work on complex data engineering efforts and lead technical teams through the solution design process
- Research, evaluate and formally recommend third party software and technology package.
- Stay abreast of emerging technologies and projects in the modern Data Engineering/Machine Learning space.
- Gather, analyze and process raw data at scale.
- Design and develop data applications using selected tools and frameworks as required and requested.
- Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
- Collaborate with data stakeholders and stewards on the verification and the accuracy of the information collected.
- Provide technical lead to Data Owners and Stewards on data definition, data lineage changes by supporting intake process, performing impact analysis, and conducting domain specific profiling.
- Develop custom templates to assure end to end data lineage form Source system of records to reporting data layer.
- Engage in data and metric standardization with data stakeholders and steward and solicit formal approval for adoption into data dictionary and glossary to enable end-users improved comprehension of which metrics to use and when to use them.Work closely with the engineering team to integrate your work into our production systems.
- Process unstructured data into a form suitable for analysis and analyze processed data.
- Monitoring data performance and modifying infrastructure as needed.
- Define data retention policies.
WHO YOU ARE:
- BA/BS/MS degree or equivalent experience; Computer Science or Math background preferred.
- 5+ years’ experience of Big Data platform implementation, including 3+ years of hands-on experience in implementation and performance tuning EMR/Hadoop/Spark implementations.
- Experience Architecting Big Data platforms using Apache Hadoop, Cloudera, Hortonworks and MapR distributions
- Fluency in python; familiarity with functional languages as well.
- Proficient in SQL; experience with Hive
- Strong knowledge of AWS data processing architectures and data services such as Redshift, Glue, Kinesis, EMR, Sagemaker, etc.
- AWS Solution Architect & Big Data Speciality certification will be an added advantage.
- 7+ years of experience of IT platform implementation in a highly technical and analytical role.
- Knowledge of serverless and micro services will be added advantage.
- Strategic, good analytical and problem-solving skills.
- Able to formulate technology roadmap and plan out implementation strategy in a systematic and methodological way.
- A good level of business knowledge.
- Ability to work independently with minimal supervision.
- Ability to work with all levels of personnel from business teams to senior management.
- Ability to communicate well technically, both orally and in written form.
- Ability to deal with ambiguity.