As a Backend Engineer - API at xAI, you will play a crucial role in building the xAI API that serves our models to developers worldwide. You will own the end-to-end system responsible for high-throughput inference, handling billions of tokens per minute with low latency and high availability, including model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling.

Responsibilities

• Build the xAI API that serves our models to developers worldwide
• Own the end-to-end system responsible for high-throughput inference, handling billions of tokens per minute with low latency and high availability, including model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling

Basic Qualifications

• Expert knowledge of either Rust or C++
• Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems
• Knowledge of service observability and reliability best practices
• Experience in operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDB

Preferred Skills and Experience

• Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)
• Experience designing or building with agent SDKs and agent orchestration frameworks
• Experience with Docker, Kubernetes, and containerized applications
• Expert knowledge of gRPC (unary, response streaming, bi-directional streaming, REST mapping)

Compensation and Benefits

$180,000 - $440,000 USD

Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

XML job scraping automation by YubHub

You will have expert knowledge of either Rust or C++ and experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems. You will also have knowledge of service observability and reliability best practices, as well as experience in operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDB.

Preferred skills and experience include experience with LLM inference engines and serving frameworks, agent SDKs and agent orchestration frameworks, Docker, Kubernetes, and containerized applications, and expert knowledge of gRPC.

In addition to a competitive base salary of $180,000 - $440,000 USD, you will receive equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

XML job scraping automation by YubHub