Freelance Data Scraping Engineer (Python)

Role overview

Mindrift is looking for a Freelance Data Scraping Engineer (Python) for the Tendem project. In this role (called an AI Pilot at Mindrift), you collaborate with Tendem Agents that automate repetitive tasks while you apply critical thinking, domain expertise, and quality control to produce accurate, actionable structured datasets.

Key responsibilities

• Own end-to-end data extraction workflows across complex websites, ensuring coverage, accuracy, and reliable delivery of structured datasets.
• Use internal tools (provided Apify and OpenRouter) plus custom workflows to accelerate scraping, validation, and execution.
• Extract reliably from dynamic/interactive web sources, handling JavaScript-rendered content, changing site behavior, and proxy/API-based access.
• Enforce data quality using validation checks, cross-source consistency, formatting adherence, and systematic verification.
• Scale scraping for large datasets using batching/parallelization; monitor failures and maintain stability as site structures shift.

Requirements

• 3+ years of relevant experience in data engineering, web scraping, automation, or software development.
• Strong Python web scraping skills (e.g., BeautifulSoup, Selenium), including dynamic content patterns (JS, AJAX, infinite scroll) and API access via proxies.
• Proven ability to extract from complex/inconsistent structures (hierarchies, archived pages, inconsistent HTML).
• Background in data cleaning, normalization, and validation; ability to deliver structured datasets (e.g., CSV, JSON, Google Sheets).
• Hands-on experience working with LLMs/AI frameworks to improve automation and problem-solving.
• High attention to detail and commitment to data accuracy.
• Self-directed troubleshooting and independent work.
• English proficiency (B2+).

Nice to have

• Bachelor’s or Master’s degree in a technical field (Engineering, Applied Mathematics, Computer Science, or related).
• GitHub link.

Contract & compensation

• Freelance / part-time remote contribution model.
• Up to $37/hour equivalent, depending on level and pace; additional bonus programs for high-quality, consistent delivery.

Remote policy

• Fully remote, on your own schedule.