Epicareer Might not Working Properly
Learn More

Python Data Engineer

Salary undisclosed

Apply on


Original
Simplified
Job Description:
We are seeking a highly skilled Web Crawler Data Engineer to join our growing team. The successful candidate will be responsible for designing, developing, and maintaining web scraping systems to collect, process, and manage large datasets from various online sources.

Responsibilities:
- Design and implement web crawlers to extract data from diverse websites efficiently.
- Develop and maintain data pipelines to process and integrate scraped data into our data warehouse.
- Write clean, scalable, and maintainable code to ensure robust data extraction.
- Analyze website structures and adapt web crawlers to handle dynamic and complex web pages.
- Ensure data accuracy and integrity through validation and quality checks.
- Troubleshoot and resolve issues related to web scraping and data processing.
- Stay current with the latest web scraping technologies, tools, and methodologies.
- Ensure compliance with legal and ethical standards for web scraping activities.