Freelance Data Scraping Engineer (Python)

Role overview

Mindrift is looking for Freelance Python Data Scraping Engineers to support the Tendem project within its hybrid AI + human system. In this role (called an AI Pilot at Mindrift), you will collaborate with Tendem Agents that handle repetitive tasks, while you contribute critical thinking, domain expertise, and quality control to produce accurate, actionable results.

Key responsibilities

• Own end-to-end data extraction workflows across complex websites, ensuring coverage, accuracy, and reliable delivery of structured datasets.
• Use internal tools such as Apify and OpenRouter alongside custom workflows to accelerate collection, validation, and execution.
• Ensure robust extraction from dynamic/interactive web sources, including JavaScript-rendered content.
• Enforce data quality with validation checks, cross-source consistency, formatting compliance, and systematic verification before delivery.
• Scale scraping for large datasets using batching/parallelization, monitoring failures and maintaining stability against minor site changes.

Requirements

• At least 3 years of relevant experience in data engineering, web scraping, automation, or software development.
• Strong Python web scraping experience (e.g., BeautifulSoup, Selenium or similar), including JS/AJAX/infinite scroll patterns and API usage via proxies.
• Ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML).
• Background in data cleaning, normalization, and validation, producing structured outputs (e.g., CSV, JSON, Google Sheets).
• Hands-on experience with LLMs/AI frameworks to improve automation and problem-solving.
• Strong attention to detail and commitment to data accuracy.
• Self-directed troubleshooting and independent execution.
• English proficiency (B2+).

Nice to have

• A GitHub link.

Compensation & work setup

• Freelance, part-time remote. Earn up to $37/hour equivalent depending on level and pace of contribution.
• Performance-based bonus programs may be available for high-quality, consistent delivery.