Software - Data Extraction Engineering Intern (Python)

Job description
????Role Description:

We are looking for a skilled Python Developer who specializes in web scraping and data extraction to help us build a system that retrieves product data from various e-commerce websites.

Your role will involve developing and maintaining scripts that pull real-time data from online retailers (like the price of items, product data, etc.), structuring the data into a usable format, and optimizing the process for efficiency and accuracy.

For updates and to be considered, follow: https://www.linkedin.com/company/snapwrite

What You’ll Be Doing
• Developing a Python-based web scraper (could use Node.js as well) to extract product prices from multiple retailers.
• Ensuring accurate data retrieval from websites (for example: Costco, Staples, Best Buy, Amazon, Visions, London Drugs, Canadian Tire, Dufresne Furniture, Tanguay, TepperMan's, LG, Samsung, etc.)
• Structuring and formatting the extracted data into JSON format.
• Implementing measures to handle CAPTCHAs, rate limits, and anti-bot protections.
• Maintaining and optimizing web scraping scripts for speed and reliability.
• :building_construction: Tech Stack
Languages: Python
Scraping Frameworks: Scrapy, BeautifulSoup, Selenium, Playwright
APIs: Knowledge of RESTful APIs to integrate the code into the existing codebaseAutomating data collection and monitoring fluctuations over time (for example: fluctuations in prices).

Tech Stack

Languages: Python

Scraping Frameworks: Scrapy, BeautifulSoup, Selenium, Playwright

APIs: Knowledge of RESTful APIs to integrate the code into the existing codebase

Core Requirements
• Experience in web scraping and data extraction.
• Proficiency in Python, with experience using Scrapy, Selenium, BeautifulSoup, or anything that gets the job done
• Understanding of HTML, CSS, and JavaScript (for navigating dynamic websites).
• Experience with handling CAPTCHAs, proxies, and bot detection mechanisms.
• Knowledge of REST APIs for structuring and delivering extracted data.
• Ability to work independently and solve complex challenges.

Nice to Have
• Knowledge of Node.js
• Experience with AI/ML for data processing.
• Knowledge of Docker for containerizing scrapers.
• Experience with data visualization tools to track price trends.