Backend Engineer_AI Gateway
$126,840–$126,840 year
On-siteEnglewood Cliffs, New Jersey, United States
Job Summary
Design and operate a high-performance AI Gateway that routes, inspects, and governs traffic between enterprise clients and LLM providers using async Python, network programming, and cloud infrastructure. Manage massive concurrent streaming connections with low latency, implementing chunked transfer handling, back-pressure, and zero-copy buffering. Build production-grade REST/Streaming APIs with FastAPI, configure Kubernetes deployments with service-mesh integration, and orchestrate event-driven task queues via Redis Streams or Kafka. Tune PostgreSQL databases, manage dynamic secrets in HashiCorp Vault, and secure L4/L7 load balancing with TLS termination and mTLS. Automate infrastructure with Terraform and CI/CD pipelines featuring container scanning and blue-green deployments. Integrate enterprise security gateways, configure SSO flows with Azure AD or Okta, and monitor observability stacks for proactive cost-optimization and incident response.
Required Qualifications
- 1 yr Contract
- Full-Time, On Site
- Pay Rate: ~$10,570/mo DOE
- Async Python & Network Programming (3 + yrs)
- Streaming & Large‐Payload Handling
- Protocol Knowledge
- FastAPI‐Based Backend (3 + yrs)
- Containerization & Kubernetes
- Async Messaging & Event‐Driven Design
- PostgreSQL & Vault
- Cloud Networking & Security
- IaC & CI/CD
- Enterprise Security Gateways
- IAM & SSO
- AI/LLM Pipelines
- API‐Gateway Operations
- Observability & FinOps
Desired Qualifications
- Korean Bilingual Highly Preferred
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.