Senior Applied AI Engineer (Observability) · Komodo Health

Fri, 24 Apr 2026 16:03:34 GMT
Engineering

Komodo Health · San Francisco, CA · $230,000-$270,000 USD · hybrid

We're hiring a Senior Applied AI Engineer to join our AI Platform/Observability team. You will be responsible for building the foundation that makes AI systems trustworthy at scale. This is a backend-heavy, systems-focused role in a greenfield environment.

Your starting point will be AI observability, evaluation, and production reliability. From there, your scope expands into broader platform ownership: agent systems, orchestration layers, and shared infrastructure. Observability is the entry point,not the end state.

In this role, you will:

Build the observability and reliability foundation for AI systems across Komodo (logging, tracing, evaluation pipelines, feedback loops)

Define how the organization measures LLM performance and quality in production (hallucinations, drift, latency, failure modes)

Ship production-grade AI systems that improve platform reliability, scalability, and performance

Lead design and architecture for complex applied AI systems (multi-agent workflows, tool-calling systems, model pipelines)

Establish evaluation frameworks and experimentation practices (A/B testing, offline + online evaluation)

You will drive experimentation across the organization, set best practices, and integrate new AI techniques into Komodo’s broader engineering ecosystem.

Additional responsibilities include:

Contributing to reusable infrastructure, patterns, and standards adopted across teams

Partnering with product, data, and platform teams to shape AI-driven solutions

We're looking for someone with strong expertise in LLMs and prompt systems, agent orchestration, and tool/function calling. You should have hands-on experience with AI observability, evaluation, or monitoring systems, debugging and improving production AI behavior, and strong backend engineering skills.

As a Senior Applied AI Engineer, you will be part of a team that is shaping the future of healthcare intelligence. You will have the opportunity to work on cutting-edge projects, collaborate with experts in the field, and make a meaningful impact on the company's mission.