Freelance Data Scraping Engineer (Python)

Role overview

Mindrift is looking for a Freelance Python Data Scraping Engineer for the Tendem project. In this role (called an AI Pilot at Mindrift), you will build and quality-check specialized web data scraping workflows within a hybrid AI + human system.

Responsibilities

• Own end-to-end data extraction workflows across complex websites to deliver complete, accurate, structured datasets.
• Use internal tools such as Apify and OpenRouter, and combine them with custom workflows for collection, validation, and execution.
• Extract data reliably from dynamic/interactive sites, including JavaScript-rendered content and changing site behavior.
• Enforce data quality standards using validation checks, cross-source consistency controls, formatting/spec adherence, and systematic verification.
• Scale scraping for large datasets via batching/parallelization, monitoring failures and maintaining stability against minor site changes.

Requirements

• 3+ years of relevant experience in data engineering, web scraping, automation, or software development.
• Strong Python web scraping experience (e.g., BeautifulSoup, Selenium) including dynamic content (JS/AJAX/infinite scroll) and API-based scraping (via proxies).
• Ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML).
• Background in data cleaning, normalization, and validation; producing structured outputs (e.g., CSV/JSON/Google Sheets).
• Hands-on experience using LLMs/AI frameworks to enhance automation and problem-solving.
• Strong attention to detail and commitment to data accuracy.
• Self-directed troubleshooting ability.
• English proficiency (B2 or above).

Nice to have

• GitHub link.
• Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields.

Compensation

• Earn up to $37/hour equivalent depending on contribution level and pace; other platform projects may vary based on scope and expertise.

Work setup

• Fully remote, part-time, on your own schedule.