Software Engineer (Web Scraping)
$200,000–$400,000 year
On-site · New York City, New York, United States
Job Summary
Design and maintain scalable data collection pipelines for dynamic JavaScript-rich web applications using browser automation and HTTP extraction; own data integrity end-to-end including validation, anomaly detection, and monitoring; collaborate with quantitative researchers to scope dataset requirements and deliver well-documented, reusable deliverables; stay current with emerging tools and frameworks, championing targeted improvements that demonstrably enhance reliability, data quality, and long-term maintainability.
Required Qualifications
- Proven track record of developing and maintaining production-grade web scrapers and crawlers using tools such as Playwright, Selenium, and/or direct HTTP extraction and parsing techniques.
- Excellent Python experience.
- Hands-on experience designing robust data pipelines, including orchestration, monitoring and alerting, and automated workflows for quality assurance and failure recovery.
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.