Your starting point will be AI observability, evaluation, and production reliability. From there, your scope expands into broader platform ownership: agent systems, orchestration layers, and shared infrastructure. Observability is the entry point,not the end state.

In this role, you will:

• Build the observability and reliability foundation for AI systems across Komodo (logging, tracing, evaluation pipelines, feedback loops)
• Define how the organization measures LLM performance and quality in production (hallucinations, drift, latency, failure modes)
• Ship production-grade AI systems that improve platform reliability, scalability, and performance
• Lead design and architecture for complex applied AI systems (multi-agent workflows, tool-calling systems, model pipelines)
• Establish evaluation frameworks and experimentation practices (A/B testing, offline + online evaluation)

You will drive experimentation across the organization, set best practices, and integrate new AI techniques into Komodo’s broader engineering ecosystem.

Additional responsibilities include:

• Contributing to reusable infrastructure, patterns, and standards adopted across teams
• Partnering with product, data, and platform teams to shape AI-driven solutions

We're looking for someone with strong expertise in LLMs and prompt systems, agent orchestration, and tool/function calling. You should have hands-on experience with AI observability, evaluation, or monitoring systems, debugging and improving production AI behavior, and strong backend engineering skills.

As a Senior Applied AI Engineer, you will be part of a team that is shaping the future of healthcare intelligence. You will have the opportunity to work on cutting-edge projects, collaborate with experts in the field, and make a meaningful impact on the company's mission.

XML job scraping automation by YubHub