Data Engineer
RemoteIndia
Job Summary
Maintain existing automation of dataflows in Dataiku and Dagster while building new flows to replace manual processes using SQL and Python scripts. Optimize PostgreSQL databases, structure data according to business priorities, and enhance monitoring and alerting for databases and dataflows. Formalize the conceptual data model, manage functional dependencies, and execute scripts in Bash, Python, and SQL within Kubernetes environments. Develop new indicators linked to business divisions and utilize APIs and web services for R&D. Work with significant geospatial data experience, including GDAL/OGR libraries, to support AI functions like market data and land registry. Collaborate with developers and product owners to evolve data models while adapting to French office hours for the Paris-based team.
Required Qualifications
- 5+ Years of experience
- 5 Years of relevant experience
- SQL
- Python scripts
- Postgres
- Significant geospatial data experience
- Significant experience in dataiku
- Experience in optimizing database (Postgress)
- Data structuring according to business priorities
- Maintaining existing automation of dataflows (mostly in Dataiku, but some also in Dagster)
- Building new dataflows and automating in Dataiku (SQL and Python scripts)
- Enhancing monitoring and alerting of databases and dataflows
- Formalization and updating of the conceptual data model
- Graph of functional dependencies
- Regular interactions with developers and product owners
- Improvement/automation of current business scripts
- Construction and industrial production of new indicators
- Use of APIs and web services
- Significant experience in designing and managing complex databases
- Excellent technical level in PostgreSQL DBMS and SQL spatial queries
- Knowledge of GDAL/OGR libraries
- Significant experience working with Dataïku software
- Must have significant experience with ETLs
- Very good knowledge of Python
- Execution of scripts in Bash shell
- Python
- SQL
- Network concepts
- API/web applications
- Adapt to working during French office hours
Desired Qualifications
- Experience in Kubernetes environment
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.