Freelance Data Scraping Engineer (Python)

Role Overview

Mindrift is seeking a Freelance Python Data Scraping Engineer to work on the Tendem project as an AI Pilot. In this hybrid AI + human workflow, you’ll collaborate with Tendem Agents while applying critical thinking, domain expertise, and strong quality control to deliver accurate, actionable structured datasets.

Responsibilities

• Own end-to-end data extraction workflows across complex websites, ensuring coverage, accuracy, and reliable delivery of structured datasets.
• Use provided tools like Apify and OpenRouter, along with custom workflows, to accelerate data collection, validation, and execution.
• Extract reliably from dynamic/interactive sources (including JavaScript-rendered content).
• Enforce data quality standards using validation checks, cross-source consistency controls, adherence to formatting specs, and systematic verification.
• Scale scraping for large datasets via batching/parallelization; monitor failures and maintain stability against minor site structure changes.

Requirements

• At least 3 years of relevant experience in data engineering, web scraping, automation, or software development.
• Strong Python web scraping experience (e.g., BeautifulSoup, Selenium), including dynamic content (JS/AJAX/infinite scroll) and API-based scraping via proxies.
• Ability to extract from complex structures (hierarchies, archived pages, inconsistent HTML).
• Background in data cleaning, normalization, and validation; produce structured datasets (e.g., CSV/JSON/Google Sheets).
• Hands-on experience with LLMs/AI frameworks to improve automation and problem-solving.
• High attention to detail and commitment to data accuracy.
• Self-directed troubleshooting and independent work style.
• English proficiency: B2 or above.

Nice to Have

• Degree in Engineering/Applied Mathematics/Computer Science (or related).
• GitHub link.

Compensation

• Up to $37/hour equivalent, depending on level and pace; varies by project scope, complexity, and expertise.

Remote / Schedule

• Fully remote, part-time, on your own schedule.