AI Platform Engineer (m/f/x)
HybridMunchen, Bavaria, Federal Republic of Germany
Munchen, Bavaria, Federal Republic of GermanyHybridFull TimeMasters DegreeFINTECHMedium
Full TimeMasters DegreeMediumFINTECH
Job Summary
Design and maintain an internal AI provisioning platform to streamline access to various LLMs and MCP Servers via LiteLLM. Implement comprehensive LLM observability, logging, and alerting systems using Datadog and Langfuse while enforcing model governance for security, data privacy, and cost management. Collaborate with embedded AI Software Engineers to support cloud-native architectures and agentic workflow integrations, ensuring production AI systems remain reliable and performant.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field
- Several years of professional experience in Platform Engineering, DevOps, SRE, or backend software development
- Strong experience and confidence working with cloud infrastructure (AWS preferred) and Infrastructure as Code (Terraform)
- Proficiency in at least one programming language (e.g., Python, Go) and experience with CI/CD pipelines and containerization (e.g., Docker)
- Solid understanding of API cost management, rate limiting, security best practices, and data privacy for AI models
- Experience with, or a strong willingness to learn, LLM gateways, observability platforms, and telemetry (e.g., LiteLLM, Langfuse, Datadog)
Desired Qualifications
- Strong problem-solving skills, a sense of humor, and the ability to thrive in an ego-free environment as an embedded platform expert within a cross-functional Data Science and AI team
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.