SENIOR DATA ENGINEER - BEES DATA
On-siteCampinas, São Paulo, Brazil
Job Summary
Implement and maintain data platform components including ingestion jobs, dbt models, and Spark transformations while ensuring outputs match expected schemas and downstream expectations. Fix defects in entity resolution logic and improve component performance, data quality checks, or reliability when gaps or incidents arise. Apply security and compliance expectations to handle sensitive data according to classification rules and use approved identity and secrets patterns. Submit well-structured pull requests with clear descriptions, testing evidence, and documentation of data purpose and flows for regulated changes. Review peers' code daily, address feedback promptly, and escalate validation or scope uncertainties without merging unstable code. Follow existing ETL and MDM standards, security baselines, and team patterns rather than inventing parallel approaches.
Required Qualifications
- Bachelor's degree in Computer Science, Computer Engineering, Information Systems, Systems Analysis and Development, or similar
- Intermediate English
- Code quality: write clear, readable, modular code; follow team naming and formatting conventions; avoid unnecessary duplication in your own changes; prefer changes that can be understood without a verbal walkthrough
- Verification: add required unit or transformation-level tests; validate schema assumptions and basic data quality conditions; ensure changes do not break existing behavior
- Delivery: submit well-structured pull requests that include a clear description of the change, context, and expected impact, and evidence of testing
- Stack (typical): Python, SQL, and data processing with PySpark and/or Scala as used in the team's pipelines
- Pipelines: practical experience building or maintaining batch/stream components with orchestration (for example, Apache Airflow, Databricks Workflows, or similar) and version control (Git)
- Data work: comfortable with transformation, cleansing, aggregation, and basic performance tuning for SQL and Spark workloads, given volume and complexity
- Cloud: familiarity with services on a major provider (AWS, Azure, or Google Cloud) in the way the team deploys and runs jobs
- Security baseline for data engineering: follow least-privilege IAM and service principals for pipelines; prefer encryption in transit and at rest where the platform provides it; keep dependencies and images within approved channels and address high-severity findings from scanners or security tooling when they affect your components
- Compliance-aware delivery: When a change touches regulated data, new integrations, or new exports, document data purpose, flows, and safeguards in the PR or linked ticket so risk and compliance partners can assess impact without guesswork
Desired Qualifications
- Hands-on with transformation tooling and data contracts in a shared warehouse
- APIs or event interfaces used for data exchange between systems
- Infrastructure-as-code or CI/CD (for example, Azure DevOps, Terraform, GitHub Actions) for job deployment
- Familiarity with data governance tooling (catalog, quality, policy tags) or vulnerability / secret scanning in CI for data repos and pipelines
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.