Software Engineer II - Streaming
$105,100–$131,400 year
RemoteUnited States or Palo Alto, California, United States
Job Summary
Build and maintain scalable real-time streaming services that ingest, process, transform, and deliver vehicle, cloud, and operational data at scale. Design and operate Apache Flink applications using Apache Kafka or Redpanda to power analytics, machine learning, and AI-driven operational decision-making. Develop production-grade streaming architectures in Java, Scala, Python, or Go, optimizing for low latency, throughput, and resiliency while implementing robust failure recovery and observability. Partner with platform, data, and product teams to deliver end-to-end real-time solutions and contribute reusable SDKs and self-service capabilities. Leverage AI-assisted development tools to accelerate code quality and engineering productivity while maintaining high standards for distributed systems reliability.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field, or equivalent practical experience
- 3+ years of professional software engineering experience building distributed systems and real-time streaming platforms
- Strong hands-on experience with Apache Kafka or Redpanda
- Topics and partitions
- Consumer groups
- Replication
- Delivery semantics
- Producer and consumer tuning
- Performance optimization
- Production operations
- Strong experience building production applications using Apache Flink
- Stateful stream processing
- Event time processing
- Watermarks
- Windowing
- Checkpointing
- Savepoints
- Parallelism
- Backpressure
- Fault recovery
- Strong understanding of event-driven architectures and real-time streaming systems
- Solid understanding of distributed systems including scalability, fault tolerance, consistency models, replication, and performance optimization
- Proficiency in one or more programming languages such as Java, Scala, Python, or Go
- Strong software engineering fundamentals including data structures, algorithms, object-oriented design, and system design
- Experience building cloud-native applications using Docker and Kubernetes
- Experience working with public cloud platforms such as AWS, Azure, or GCP
- Experience debugging production issues across distributed applications and streaming infrastructure
- Familiarity with REST APIs, gRPC, or event-driven service integration
- Strong communication and collaboration skills with the ability to work effectively across cross-functional teams
Desired Qualifications
- Experience operating large-scale Apache Kafka and Apache Flink deployments in production
- Experience with Kafka Connect, Debezium, or Change Data Capture (CDC) architectures
- Experience designing schema-based event contracts using Avro, Protobuf, JSON Schema, and Schema Registry
- Experience integrating streaming platforms with Apache Iceberg, Delta Lake, ClickHouse, Apache Pinot, or similar analytical storage systems
- Familiarity with Spark Structured Streaming or complementary stream processing technologies
- Experience building real-time feature engineering pipelines supporting machine learning and AI applications
- Experience with OpenTelemetry, Prometheus, Grafana, distributed tracing, and modern observability platforms
- Experience using Terraform, Helm, or Infrastructure as Code practices
- Experience building edge-to-cloud or vehicle-to-cloud streaming architectures
- Experience participating in production incident response, postmortems, and operational excellence initiatives
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.