The ideal candidate combines strong ML fundamentals with deep expertise in backend system design. You’ll work in a highly collaborative environment, bridging research and engineering to deliver seamless experiences to our customers and accelerate innovation across the company.

Responsibilities:

• Build and maintain fault-tolerant, high-performance systems for serving LLM workloads at scale.
• Build an internal platform to empower LLM capability discovery.
• Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
• Conduct architecture and design reviews to uphold best practices in system design and scalability.
• Develop monitoring and observability solutions to ensure system health and performance.
• Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Ideally you’d have:

• 5+ years of experience building large-scale, high-performance backend systems.
• Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
• Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.).
• Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc.
• Experience with containers and orchestration tools (e.g., Docker, Kubernetes).
• Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform).
• Proven ability to solve complex problems and work independently in fast-moving environments.

Nice to haves:

• Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference.

Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position, determined by work location and additional factors, including job-related skills, experience, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be eligible for equity grant. You’ll also receive benefits including, but not limited to: Comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. Additionally, this role may be eligible for additional benefits such as a commuter stipend.

XML job scraping automation by YubHub

We're looking for engineers who have owned high scale operational sensitive systems like customer facing APIs, Edge Gateways, ML Inference, or similar services and have an interest in getting deep building LLM APIs and runtimes at scale. As a Staff Engineer, you'll play a critical role in shaping both the product experience and core infrastructure.

The impact you will have:

• Design and implement core systems and APIs that power Databricks Foundation Model Serving, ensuring scalability, reliability, and operational excellence.
• Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
• Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads.
• Contribute directly to key components across the serving infrastructure , from working in systems like vLLM and SGLang to creating token based rate limiters and optimizers , ensuring smooth and efficient operations at scale.
• Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
• Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.
• Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy.

What we look for:

• 10+ years of experience building and operating large-scale distributed systems.
• Experience leading high-scale operationally sensitive backend systems.
• A track record of up-leveling teams engineering excellence.
• Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
• Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
• Strong communication skills and ability to collaborate across teams in fast-moving environments.
• Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
• Passion for mentoring, growing engineers, and fostering technical excellence.

XML job scraping automation by YubHub

Owns the science roadmap for grounding—including retrieval, re-ranking, attribution, and reasoning—driving initiatives from problem framing to production impact. Designs and evolves state-of-the-art retrieval and RAG orchestration across documents, tables, code, and images. Builds citation and provenance systems (e.g., passage highlighting, quote-level alignment, confidence scoring) to reduce hallucinations and increase user trust. Leads experimentation and evaluation using A/B testing, interleaving, NDCG, MRR, precision/recall, and calibration curves to guide measurable trade-offs. Advances tool-augmented grounding through schema-aware retrieval, function calling, knowledge graph joins, and real-time connectors to databases, cloud object stores, search indexes, and the web. Partners with platform engineering to productionize models with scalable inference, embedding services, feature stores, caching, and privacy-compliant multi-tenant systems. Nurtures collaborative relationships with product and business leaders across Microsoft, influencing strategic decisions and driving business impact through technology. Authors white papers, contributes to internal tools and services, and may publish research to generate intellectual property. Bridges the gap between researchers (e.g., Microsoft Research) and development teams, applying long-term research to solve immediate product needs. Leads high-stakes negotiations to ensure cutting-edge technologies are applied practically and effectively. Identifies and solves significant business problems using novel, scalable, and data-driven solutions. Shapes the direction of Microsoft and the broader industry through pioneering product and tooling work. Mentors applied scientists and data scientists, establishing best practices in experimentation, error analysis, and incident review. Collaborates cross-functionally with PMs, research, infrastructure, and security teams to align on milestones, SLAs, and safety protocols. Communicates clearly through design documentation, progress updates, and presentations to executives and customers. Contributes to ethics and privacy policies, identifies bias in product development, and proposes mitigation strategies.

XML job scraping automation by YubHub

Microsoft AI are looking for a talented Member of Technical Staff, Multimodal Infrastructure to help build the next wave of capabilities of our personalized AI assistant, Copilot. We're looking for someone who will bring an abundance of positive energy, empathy, and kindness to the team every day, in addition to being highly effective.

About the Role

We are seeking a highly skilled and experienced engineer to join our team as a Member of Technical Staff, Multimodal Infrastructure. The successful candidate will be responsible for designing, developing, and maintaining large-scale multimodal data processing pipelines, model pretraining and post-training frameworks, and model inference and serving frameworks. They will work closely with research scientists and product engineers to solve infra-related problems and find a path to get things done despite roadblocks to get their work into the hands of users quickly and iteratively.

Accountabilities

• Design, develop, and maintain large-scale multimodal data processing pipelines.
• Design, develop, and maintain large-scale multimodal model pretraining and post-training frameworks.
• Design, develop, and maintain large-scale multimodal model inference and serving frameworks.

The Candidate we're looking for

Experience:

• Bachelor's Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.

Technical skills:

• Strong proficiency in distributed data processing infra (resource utilization management, fault tolerance, ray & spark) and CPU/GPU batch processing optimizations.
• Experience with state-of-art model inference and serving frameworks.
• Experience with image/video/audio data processing.
• Experience with common data formats for efficient I/O.

Personal attributes:

• Enjoy working in a fast-paced, design-driven, product development cycle.
• Embody our Culture and Values.

Benefits

• Competitive salary and benefits package.
• Opportunities for professional growth and development.
• Collaborative and dynamic work environment.
• Access to cutting-edge technology and tools.
• Flexible work arrangements.

XML job scraping automation by YubHub