Data Engineer
RemoteColombia
ColombiaRemoteFull TimeMedium
Full TimeMedium
Job Summary
Design and maintain scalable data pipelines using Databricks and Apache Spark to ingest, transform, and load data from diverse sources. Implement Change Data Capture solutions for real-time synchronization and write efficient Python code for processing and automation. Collaborate with data architects to translate business requirements into robust pipeline designs while monitoring performance, reliability, and cost efficiency. Ensure data quality and integrity across all layers, participate in code reviews, and document architecture for team visibility. Support cross-functional teams on analytics, reporting, and downstream AI/ML use cases.
Required Qualifications
- 3+ years of experience in data engineering or a related field
- Strong hands-on experience with Databricks for data processing and analytics workloads
- Proficiency with Apache Spark for large-scale distributed data processing
- Experience implementing CDC solutions and tools (e.g., Debezium or similar)
- Strong programming skills in Python for data engineering tasks
- Solid understanding of SQL and relational database concepts
- Experience building and maintaining ETL/ELT pipelines in production environments
- Understanding of data modeling principles and data warehousing concepts
- Strong analytical and problem-solving skills
- Good communication skills, with the ability to work across technical and business teams
Desired Qualifications
- Experience with cloud platforms (AWS, Azure) and their data services
- Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows)
- Knowledge of data governance and cataloging practices
- Experience with streaming data technologies (e.g., Kafka)
- Experience working in Agile/Scrum environments
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.