As a Backend Engineer - API at xAI, you will play a crucial role in building the xAI API that serves our models to developers worldwide. You will own the end-to-end system responsible for high-throughput inference, handling billions of tokens per minute with low latency and high availability, including model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling.

Responsibilities

• Build the xAI API that serves our models to developers worldwide
• Own the end-to-end system responsible for high-throughput inference, handling billions of tokens per minute with low latency and high availability, including model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling

Basic Qualifications

• Expert knowledge of either Rust or C++
• Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems
• Knowledge of service observability and reliability best practices
• Experience in operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDB

Preferred Skills and Experience

• Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)
• Experience designing or building with agent SDKs and agent orchestration frameworks
• Experience with Docker, Kubernetes, and containerized applications
• Expert knowledge of gRPC (unary, response streaming, bi-directional streaming, REST mapping)

Compensation and Benefits

$180,000 - $440,000 USD

Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

XML job scraping automation by YubHub

You will have expert knowledge of either Rust or C++ and experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems. You will also have knowledge of service observability and reliability best practices, as well as experience in operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDB.

Preferred skills and experience include experience with LLM inference engines and serving frameworks, agent SDKs and agent orchestration frameworks, Docker, Kubernetes, and containerized applications, and expert knowledge of gRPC.

In addition to a competitive base salary of $180,000 - $440,000 USD, you will receive equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

XML job scraping automation by YubHub

We build foundational AI infrastructure that enables large-scale training and inference across diverse workloads and rapidly evolving hardware generations. Our work directly shapes how AI systems are designed, deployed, and scaled today and into the future. Engineers on this team operate with end-to-end ownership, deep technical rigor, and a strong bias toward real-world impact.

The MAIST is a startup-like team inside Microsoft AI, created to push the boundaries of AI toward Humanist Superintelligence,ultra-capable systems that remain controllable, safety-aligned, and anchored to human values. Our mission is to create AI that amplifies human potential while ensuring humanity remains firmly in control.

As a Senior AI Systems Engineer, you will own the software integration layer between our custom AI chip's proprietary SDK and SGLang, a state-of-the-art serving framework for Large Language Models (LLMs) and Vision-Language Models. You will be responsible for ensuring that our silicon can seamlessly run SGLang inference workloads at peak performance, bypassing the traditional CUDA ecosystem entirely.

Responsibilities:

• Framework Integration: Architect and develop the backend integration to make our custom AI chip a first-class citizen in SGLang.
• Custom Operator Development: Write custom C++ / PyTorch extensions that map SGLang's primitive operations (e.g., RadixAttention, FlashAttention, matrix multiplications) to our custom chip's proprietary software layer.
• Performance Optimization: Profile and optimize end-to-end LLM inference latency, throughput, and memory utilization (Paged Attention) on our hardware.
• Cross-Functional Collaboration: Work closely with our hardware architecture and compiler teams to provide feedback on our custom software stack and silicon design based on framework-level bottlenecks.
• Testing & Deployment: Build robust testing pipelines to validate model accuracy and performance parity against standard GPU baselines.

Qualifications:

• Must-Have Qualifications: BS, MS, or PhD in Computer Science, Computer Engineering, or a related field. Software engineering experience focusing on systems programming, ML infrastructure, or AI compilers. Expertise in Python: Deep understanding of memory management, concurrent programming. Experience with LLM Inference Engines: Hands-on experience modifying or extending frameworks like SGLang, vLLM, DeepSpeed-FastGen, or TensorRT-LLM. PyTorch Internals: Strong experience writing PyTorch C++ extensions and custom operators. Hardware Interfacing: Proven track record of integrating machine learning workloads with hardware accelerators (GPUs, TPUs, NPUs) using custom SDKs, APIs, or low-level drivers.

• Nice-to-Have Qualifications: Prior experience working on non-CUDA software ecosystems (e.g., AMD ROCm, AWS Neuron, Google XLA). Familiarity with AI compilers and intermediate representations (MLIR, Apache TVM, OpenAI Triton). Strong understanding of underlying LLM architectures (Transformers, MoE) and state-of-the-art attention algorithms (FlashAttention v2/v3). Previous experience at an AI silicon startup or working on custom accelerators (e.g., Google TPU, AWS Trainium).

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

XML job scraping automation by YubHub

Our work directly shapes how AI systems are designed, deployed, and scaled today and into the future. Engineers on this team operate with end-to-end ownership, deep technical rigor, and a strong bias toward real-world impact.

As a Senior AI Systems Engineer, you will own the software integration layer between our custom AI chip's proprietary SDK and SGLang, a state-of-the-art serving framework for Large Language Models (LLMs) and Vision-Language Models.

Responsibilities:

• Architect and develop the backend integration to make our custom AI chip a first-class citizen in SGLang.
• Write custom C++ / PyTorch extensions that map SGLang's primitive operations (e.g., RadixAttention, FlashAttention, matrix multiplications) to our custom chip's proprietary software layer.
• Profile and optimize end-to-end LLM inference latency, throughput, and memory utilization (Paged Attention) on our hardware.
• Work closely with our hardware architecture and compiler teams to provide feedback on our custom software stack and silicon design based on framework-level bottlenecks.
• Build robust testing pipelines to validate model accuracy and performance parity against standard GPU baselines.

Qualifications:

• BS, MS, or PhD in Computer Science, Computer Engineering, or a related field.
• Software engineering experience focusing on systems programming, ML infrastructure, or AI compilers.
• Expertise in Python: Deep understanding of memory management, concurrent programming.
• Experience with LLM Inference Engines: Hands-on experience modifying or extending frameworks like SGLang, vLLM, DeepSpeed-FastGen, or TensorRT-LLM.
• PyTorch Internals: Strong experience writing PyTorch C++ extensions and custom operators.
• Hardware Interfacing: Proven track record of integrating machine learning workloads with hardware accelerators (GPUs, TPUs, NPUs) using custom SDKs, APIs, or low-level drivers.

XML job scraping automation by YubHub

Our work directly shapes how AI systems are designed, deployed, and scaled today and into the future. Engineers on this team operate with end-to-end ownership, deep technical rigor, and a strong bias toward real-world impact.

As a Senior AI Systems Engineer, you will own the software integration layer between our custom AI chip's proprietary SDK and SGLang, a state-of-the-art serving framework for Large Language Models (LLMs) and Vision-Language Models.

Responsibilities:

• Architect and develop the backend integration to make our custom AI chip a first-class citizen in SGLang.
• Write custom C++ / PyTorch extensions that map SGLang's primitive operations (e.g., RadixAttention, FlashAttention, matrix multiplications) to our custom chip's proprietary software layer.
• Profile and optimize end-to-end LLM inference latency, throughput, and memory utilization (Paged Attention) on our hardware.
• Work closely with our hardware architecture and compiler teams to provide feedback on our custom software stack and silicon design based on framework-level bottlenecks.
• Build robust testing pipelines to validate model accuracy and performance parity against standard GPU baselines.

Qualifications:

• BS, MS, or PhD in Computer Science, Computer Engineering, or a related field.
• Software engineering experience focusing on systems programming, ML infrastructure, or AI compilers.
• Expertise in Python: Deep understanding of memory management, concurrent programming.
• Experience with LLM Inference Engines: Hands-on experience modifying or extending frameworks like SGLang, vLLM, DeepSpeed-FastGen, or TensorRT-LLM.
• PyTorch Internals: Strong experience writing PyTorch C++ extensions and custom operators.
• Hardware Interfacing: Proven track record of integrating machine learning workloads with hardware accelerators (GPUs, TPUs, NPUs) using custom SDKs, APIs, or low-level drivers.

XML job scraping automation by YubHub