Competitive Intelligence Engineer
On-sitePune, Maharashtra, India
Job Summary
Develop scalable Python-based systems to extract, transform, and normalize data from complex digital assets including web interfaces, PDFs, and documents. Automate interaction with dynamic content rendered in browsers using tools like Selenium and parse content using PDF libraries and OCR where necessary. Design fault-tolerant extraction logic with logging, error handling, and retry mechanisms to ensure high accuracy. Build pipelines to clean, structure, and deliver data downstream in JSON, CSV, or database-ready formats while collaborating with analysts to optimize workflows. Evaluate and prototype AI/LLM-based solutions for advanced parsing. Requires 5+ years of experience in data engineering, proficiency in Python, Selenium, and PDF parsing libraries, with a focus on building robust, automated data solutions.
Required Qualifications
- 5+ Years
- Python-based automation or data extraction roles
- 3+ years of experience in Python-based automation or data extraction roles
- requests
- BeautifulSoup or similar parsing libraries
- Selenium or other browser automation tools
- PDF parsing libraries (e.g., PyMuPDF, PDFMiner, pdfplumber)
- OCR tools (e.g., Tesseract, EasyOCR)
- Strong understanding of HTML/XML structure, selectors, and dynamic content loading
- Solid debugging, logging, and optimization practices
- Ability to handle authentication flows, session management, and anti-bot scenarios
- Familiarity with version control (Git) and basic CI/CD workflows
Desired Qualifications
- Experience with Large Language Models (LLMs) for text analysis, entity extraction, or classification tasks
- Exposure to cloud-based storage and processing (GCP)
- Basic knowledge schedulers (e.g., cronjob, Airflow)
- Experience working in agile, cross-functional teams with fast iteration cycles
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.