Data Engineer (Fintech)
RemoteNigeria
Job Summary
Design, develop, and maintain robust data pipelines using ETL and ELT methodologies to process and integrate data from various sources into a data lake, warehouse, and operational data stores. Implement real-time streaming solutions with Kafka and Debezium, while building and scheduling custom workflows using Apache Airflow. Manage infrastructure as code with Terraform, Kubernetes, and Helm, and develop continuous integration and deployment pipelines. Conduct unit and integration testing to ensure code quality and data integrity, and engage with cross-functional teams to understand data needs. Monitor system performance as part of an on-call rotation. This permanent, hybrid role in Lagos, Nigeria requires a Data Engineer with strong Python skills and 3+ years of experience in building data pipelines.
Required Qualifications
- Bachelor's degree in Computer Science, Engineering or a related field
- Proficiency in Python
- Minimum of 3 years of experience in data engineering roles
- Minimum of 2 years experience in software and/or application development roles
- Hands-on experience with Kafka, Debezium, and Kafka Connect
- Proficiency in a data pipeline orchestration tool or suitable workflow orchestration tool like Apache Airflow, Databricks, Dagster or Airbyte
- Strong understanding and hands-on experience working with various database technologies, including MySQL, PostgreSQL, MongoDB and Redshift
- Experience with Terraform, Kubernetes, and Helm for infrastructure management
- Solid knowledge of cloud computing concepts
- Ability to write complex SQL queries across different dialects
- Familiarity with unit and integration testing methodologies
- Experience in setting up and maintaining CI/CD pipelines
- Good verbal and written communication skills
- Demonstrated ability to work collaboratively within a team and across departments
- Comfortable working in a fast-paced environment with changing priorities, technologies and tooling
- Strong analytical and problem-solving skills
Desired Qualifications
- Knowledge of JavaScript and Scala
- BigQuery and SingleStore
- AWS services
- Exposure to analytical systems and basic data science tooling
- Familiarity with basic machine learning and analytical modelling concepts
- Exposure to self-service reporting tools like Tableau, Looker and DOMO
- Life-long learners
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.