Software Engineer - ETL Developer with Python
HybridGurugram, Haryana, India
EXPIREDGurugram, Haryana, IndiaHybridFull TimeMasters DegreeEnterprise
Full TimeMasters DegreeEnterprise
Job Summary
Design, develop, and maintain scalable ETL systems and data pipelines using Python, SQLAlchemy, and AWS services. Transform and process large structured and unstructured datasets while optimizing SQL queries for relational databases like MSSQL, PostgreSQL, and Oracle. Architect agentic AI solutions to automate data workflows and integrate real-time streaming platforms such as Kafka and Kinesis. Collaborate with cross-functional teams to implement data models supporting business intelligence and machine learning initiatives. Stay current with emerging data engineering trends to recommend process improvements.
Required Qualifications
- Advanced technical expertise in Python programming
- Relational databases
- ETL processes
- Strong foundation in data modeling
- Data transformation
- Processing of both structured and unstructured data
- Design, develop, and maintain scalable and robust data pipelines and ETL systems using Python and SQLAlchemy
- Transform and process large datasets from diverse sources, ensuring data integrity, quality, and consistency
- Optimize and build efficient SQL queries for data retrieval and manipulation
- Manage and interact with relational databases (MSSQL, PostgreSQL, Oracle, etc.)
- Collaborate with cross-functional teams to understand data requirements and deliver innovative solutions
- Implement data models and structures to support business intelligence, analytics, and machine learning initiatives
- Leverage advanced Python programming techniques and libraries (e.g., Pandas, Numpy) to solve complex data challenges
- Architect and implement agentic AI solutions to automate data processes, enhance pipeline intelligence, and support next-generation analytics
- Work with AWS cloud services related to data engineering ( AWS Batch, S3, Lambda, etc.)
- Stay abreast of emerging trends in data engineering and AI and proactively recommend improvements
- Integrate real-time data streaming solutions (e.g., Kafka, Kinesis) into data pipelines
- Multi-threading
- Multi-processing
- Regular Expressions
- Exception Handling
- Generators
- Decorators
- Context Managers
- Asyncio
- Type Hinting
- Best practices for scalable code
- ETL
- Processing structured and unstructured data
- Data quality
- Data integrity
- Query Optimization
- Query Building
- Experience with ORMs like SQLAlchemy
- Exposure to databases such as MSSQL, PostgreSQL, Oracle, etc.
- Functional and Object-Oriented Programming (OOPS)
- Strong analytical and problem-solving skills for feature development and automation
- Bachelor's or master's degree in computer science, Engineering, or a related field
- Proven experience as a Data Engineer or in a similar role
- Excellent communication and teamwork abilities
- Ability to work in a fast-paced, agile environment
Desired Qualifications
- Exposure to or interest in agentic AI frameworks and tools (e.g., LangChain, OpenAI APIs, or similar)
- Experience integrating AI/ML models into data pipelines
- Experience with AWS data engineering services (Athena, AWS Batch, S3, Lambda, etc.)
- Experience with real-time data streaming platforms (Kafka, Kinesis, etc.)
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.