Senior Data Engineer
RemoteUnited States or India
United States or IndiaRemotePart TimeSenior LevelBachelors DegreeCloud ServicesMedium
Part TimeSenior LevelBachelors DegreeMediumCloud Services
Job Summary
Develop end-to-end technical solutions into production, ensuring performance, security, and scalability for data integration. Leverage expertise in Java, Python, and Scala to build on core cloud platforms like Snowflake, AWS, Azure, and Databricks. Optimize SQL queries and design detailed documentation including POCS and roadmaps. Manage automated data transformation pipelines using dbt, Spark, and orchestration tools such as Airflow and NiFi. Handle client-facing communication to deliver presentations and translate complex data challenges into robust architectural designs.
Required Qualifications
- At least 4+ years experience as a Software Engineer, Data Engineer or Data Analyst
- Ability to develop end-to-end technical solutions into production — and to help ensure performance, security, scalability, and robust data integration
- Programming expertise in Java, Python and/or Scala
- Core cloud data platforms including Snowflake, AWS, Azure, Databricks and GCP
- SQL and the ability to write, debug, and optimize SQL queries
- Client-facing English written and verbal communication skills and experience
- Create and deliver detailed presentations
- Detailed solution documentation (e.g. including POCS and roadmaps, sequence diagrams, class hierarchies, logical system views, etc.)
- 4-year Bachelor's degree in Computer Science or a related field
Desired Qualifications
- Production experience in core data platforms: Snowflake, AWS, Azure, GCP, Hadoop, Databricks
- Cloud and Distributed Data Storage: S3, ADLS, HDFS, GCS, Kudu, ElasticSearch/Solr, Cassandra or other NoSQL storage systems
- Data integration technologies: Spark, Kafka, event/streaming, Streamsets, Matillion, Fivetran, NiFi, AWS Data Migration Services, Azure DataFactory, Informatica Intelligent Cloud Services (IICS), Google DataProc or other data integration technologies
- Multiple data sources (e.g. queues, relational databases, files, search, API)
- Complete software development lifecycle experience including design, documentation, implementation, testing, and deployment
- Automated data transformation and data curation: dbt, Spark, Spark streaming, automated pipelines
- Workflow Management and Orchestration: Airflow, AWS Managed Airflow, Luigi, NiFi
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.