The ideal candidate combines strong ML fundamentals with deep expertise in backend system design. You’ll work in a highly collaborative environment, bridging research and engineering to deliver seamless experiences to our customers and accelerate innovation across the company.

Responsibilities:

• Build and maintain fault-tolerant, high-performance systems for serving LLM workloads at scale.
• Build an internal platform to empower LLM capability discovery.
• Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
• Conduct architecture and design reviews to uphold best practices in system design and scalability.
• Develop monitoring and observability solutions to ensure system health and performance.
• Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Ideally you’d have:

• 5+ years of experience building large-scale, high-performance backend systems.
• Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
• Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.).
• Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc.
• Experience with containers and orchestration tools (e.g., Docker, Kubernetes).
• Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform).
• Proven ability to solve complex problems and work independently in fast-moving environments.

Nice to haves:

• Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference.

Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position, determined by work location and additional factors, including job-related skills, experience, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be eligible for equity grant. You’ll also receive benefits including, but not limited to: Comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. Additionally, this role may be eligible for additional benefits such as a commuter stipend.

XML job scraping automation by YubHub

You'll shape the future of fal's inference engine and ensure our generative models achieve best-in-class performance. Your work directly impacts our ability to rapidly deliver cutting-edge creative solutions to users, from individual creators to global brands.

Day-to-day, you'll set technical direction, guide your team to build high-performance inference solutions, and personally contribute to critical inference performance enhancements and optimizations. You'll collaborate closely with research & applied ML teams, influence model inference strategies and deployment techniques, and drive advanced performance optimizations.

As a leader, you'll mentor and scale your team, coach and expand your team of performance-focused engineers, and help them innovate, solve complex performance challenges, and level up their skills.

To succeed in this role, you'll need to be deeply experienced in ML performance optimization, understand the full ML performance stack, and know inference inside-out. You'll also need to thrive in cross-functional collaboration and have excellent leadership skills.

If you're ready to lead the future of inference performance at a fast-paced, high-growth frontier, apply now!

XML job scraping automation by YubHub

The role involves architecting and optimizing the inference for high-volume, low-latency, and high-availability environments, leading the acquisition and automation of benchmarks, collaborating with cross-functional teams, and innovating solutions to enhance our AI-powered applications.

Key responsibilities include:

• Architecting and optimizing the inference for high-volume, low-latency, and high-availability environments
• Leading the acquisition and automation of benchmarks at both micro and macro scales
• Introducing new techniques and tools to improve performance, latency, throughput, and efficiency in our model inference stack
• Building tools to identify bottlenecks and sources of instability, and designing solutions to address them
• Collaborating with machine learning researchers, engineers, and product managers to bring cutting-edge technologies into production
• Optimizing code and infrastructure to maximize hardware utilization and efficiency
• Mentoring and guiding team members, fostering a culture of collaboration, innovation, and continuous learning

Requirements include:

• Extensive experience in C++ and Python, with a strong focus on backend development and performance optimization
• Deep understanding of modern ML architectures and experience with performance optimization for inference
• Proven track record with large-scale distributed systems, particularly performance-critical ones
• Familiarity with PyTorch, TensorRT, CUDA, NCCL
• Strong grasp of infrastructure, continuous integration, and continuous development principles
• Ability to lead and mentor team members, driving projects from concept to implementation
• Results-oriented mindset with a bias towards flexibility and impact
• Passion for staying ahead of emerging technologies and applying them to AI-driven solutions
• Humble attitude, eagerness to help colleagues, and a desire to see the team succeed

Our Culture

We're driven to build a strong company culture and are looking for individuals with solid alignment with the following:

• Reason with rigor
• Are you audacious enough?
• Make our customers succeed
• Ship early and accelerate
• Leave your ego aside

XML job scraping automation by YubHub

As a Solutions Architect, you will engage directly with developers, researchers, and data scientists with some of NVIDIA’s most strategic technology customers as well as work directly with business and engineering teams on product strategy.

Key Responsibilities:

• Help cloud customers craft, deploy, and maintain scalable, GPU-accelerated inference pipelines on cloud ML services and Kubernetes for large language models (LLMs) and generative AI workloads.
• Enhance performance tuning using TensorRT/TensorRT-LLM, vLLM, Dynamo, and Triton Inference Server to improve GPU utilization and model efficiency.
• Collaborate with multi-functional teams (engineering, product) and offer technical mentorship to cloud customers implementing AI inference at scale.
• Build custom PoCs for solution that address customer’s critical business needs applying NVIDIA hardware and software technology
• Partner with Sales Account Managers or Developer Relations Managers to identify and secure new business opportunities for NVIDIA products and solutions for ML/DL and other software solutions
• Prepare and deliver technical content to customers including presentations about purpose-built solutions, workshops about NVIDIA products and solutions, etc.
• Conduct regular technical customer meetings for project/product roadmap, feature discussions, and intro to new technologies. Establish close technical ties to the customer to facilitate rapid resolution of customer issues

Requirements:

• BS/MS/PhD in Electrical/Computer Engineering, Computer Science, Statistics, Physics, or other Engineering fields or equivalent experience.
• 3+ Years in Solutions Architecture with a proven track record of moving AI inference from POC to production in cloud computing environments including AWS, GCP, or Azure
• 3+ years of hands-on experience with Deep Learning frameworks such as PyTorch and TensorFlow
• Excellent knowledge of the theory and practice of LLM and DL inference
• Strong fundamentals in programming, optimizations, and software design, especially in Python
• Experience with containerization and orchestration technologies like Docker and Kubernetes, monitoring, and observability solutions for AI deployments
• Knowledge of Inference technologies - NVIDIA NIM, TensorRT-LLM, Dynamo, Triton Inference Server, vLLM, etc
• Proficiency in problem-solving and debugging skills in GPU environments
• Excellent presentation, communication and collaboration skills

Nice to Have:

• AWS, GCP or Azure Professional Solution Architect Certification.
• Experience optimizing and deploying large MoE LLMs at scale
• Active contributions to open-source AI inference projects (e.g., vLLM, TensorRT-LLM Dynamo, SGLang, Triton or similar)
• Experience with Multi-GPU Multi-node Inference technologies like Tensor Parallelism/Expert Parallelism, Disaggregated Serving, LWS, MPI, EFA/Infiniband, NVLink/PCIe, etc
• Experience in developing and integrating monitoring and alerting solutions using Prometheus, Grafana, and NVIDIA DCGM and GPU performance Analysis and tools like NVIDIA Nsight Systems

XML job scraping automation by YubHub

We are looking for an engineering leader who is hands-on with deep learning—comfortable reading/modeling code, not just running it. You will lead, encourage, and develop world-class engineering and data teams distributed across Europe, Asia and the United States.

Key Responsibilities:

• Architect and operationalize NVIDIA’s end-to-end data Inference Acceleration strategy, powering Inferencing and continuous performance improvements.
• Drive Strategic Implementations of TensorRT, VLLM and other accelerated frameworks for inference solutions for Edge and Enterprise devices: Lead Accelerated Computing efforts and solutions for key Metropolis verticals. Set up Proofs of Readiness (PORs) and guide their implementations.
• Leading customer solutions: Collaborate with major Metropolis OEMs and Partners to architect highly accelerated and optimized custom deep learning models and inference pipelines for their specific requirements. Offer direct customer support, including debugging, technical education, and handling customer inquiries for our Metropolis partner and customers. Responsible for drafting and finalizing SOWs with internal customers and partners.
• Performance Benchmarking: Orchestrate efforts to achieve leading performance results on industry benchmarks like MLPerf on various edge and Enterprise devices.
• Technical Leadership & Influence: Function as a technical leader for deep learning across multiple teams, giving oversight and build support. Apply customer insights to influence the composition and structure of upcoming SOC / GPU deep learning hardware.
• Scaling the team: Strategically hiring to meet new demands while also mentoring and adjusting existing teams to new deep learning challenges.
• Representing Nvidia Deep learning solutions in webinars, conferences and partner events

Requirements:

• Masters in Computer Science/Electrical Engineering or equivalent experience.
• A minimum of 8 years of meaningful involvement in machine learning/deep learning research or practical experience, coupled with 7+ years of leadership background and overall 15+ years of industry experience.
• Over 10 years of validated expertise in the embedded software sector, holding technical leadership positions accountable for delivering outstanding production software within a multifaceted setting.
• Deep Knowledge of GPU, CPU and dedicated deep learning architecture fundamentals and low-level performance optimizations using heterogeneous computing.
• Hands-on experience with VLMs, LLMs, or multimodal AI systems applied to perception, data triage, or automated labeling.
• Strong expertise in large-scale data processing, systems build, or machine learning pipelines.
• Strong communication, careful planning, and technical leadership capabilities.

Benefits:

• Competitive salary package and benefits
• Eligible for equity

How to Apply:

Applications for this job will be accepted at least until March 13, 2026.

XML job scraping automation by YubHub

Location

San Francisco

Employment Type

Full time

Department

Scaling

Compensation

• $295K – $555K • Offers Equity

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the Team

OpenAI’s Inference team powers the deployment of our most advanced models - including our GPT models, 4o Image Generation, and Whisper - across a variety of platforms. Our work ensures these models are available, performant, and scalable in production, and we partner closely with Research to bring the next generation of models into the world. We're a small, fast-moving team of engineers focused on delivering a world-class developer experience while pushing the boundaries of what AI can do.

We’re expanding into multimodal inference, building the infrastructure needed to serve models that handle image, audio, and other non-text modalities. These workloads are inherently more heterogeneous and experimental, involving diverse model sizes and interactions, more complex input/output formats, and tighter coordination with product and research.

About the Role

We’re looking for a software engineer to help us serve OpenAI’s multimodal models at scale. You’ll be part of a small team responsible for building reliable, high-performance infrastructure for serving real-time audio, image, and other MM workloads in production.

This work is inherently cross-functional: you’ll collaborate directly with researchers training these models and with product teams defining new modalities of interaction. You'll build and optimize the systems that let users generate speech, understand images, and interact with models in ways far beyond text.

In this role, you will:

• Design and implement inference infrastructure for large-scale multimodal models.

• Optimize systems for high-throughput, low-latency delivery of image and audio inputs and outputs.

• Enable experimental research workflows to transition into reliable production services.

• Collaborate closely with researchers, infra teams, and product engineers to deploy state-of-the-art capabilities.

• Contribute to system-level improvements including GPU utilization, tensor parallelism, and hardware abstraction layers.

You might thrive in this role if you:

• Have experience building and scaling inference systems for LLMs or multimodal models.

• Have worked with GPU-based ML workloads and understand the performance dynamics of large models, especially with complex data like images or audio.

• Enjoy experimental, fast-evolving work and collaborating closely with research.

• Are comfortable dealing with systems that span networking, distributed compute, and high-throughput data handling.

• Have familiarity with inference tooling like vLLM, TensorRT-LLM, or custom model parallel systems.

• Own problems end-to-end and are excited to operate in ambiguous, fast-moving spaces.

Nice to Have:

• Experience working with image generation or audio synthesis models in production.

• Exposure to distributed ML training or system-efficient model design.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

XML job scraping automation by YubHub

Microsoft AI are looking for a talented Member of Technical Staff, Multimodal Infrastructure to help build the next wave of capabilities of our personalized AI assistant, Copilot. We're looking for someone who will bring an abundance of positive energy, empathy, and kindness to the team every day, in addition to being highly effective.

About the Role

We are seeking a highly skilled and experienced engineer to join our team as a Member of Technical Staff, Multimodal Infrastructure. The successful candidate will be responsible for designing, developing, and maintaining large-scale multimodal data processing pipelines, model pretraining and post-training frameworks, and model inference and serving frameworks. They will work closely with research scientists and product engineers to solve infra-related problems and find a path to get things done despite roadblocks to get their work into the hands of users quickly and iteratively.

Accountabilities

• Design, develop, and maintain large-scale multimodal data processing pipelines.
• Design, develop, and maintain large-scale multimodal model pretraining and post-training frameworks.
• Design, develop, and maintain large-scale multimodal model inference and serving frameworks.

The Candidate we're looking for

Experience:

• Bachelor's Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.

Technical skills:

• Strong proficiency in distributed data processing infra (resource utilization management, fault tolerance, ray & spark) and CPU/GPU batch processing optimizations.
• Experience with state-of-art model inference and serving frameworks.
• Experience with image/video/audio data processing.
• Experience with common data formats for efficient I/O.

Personal attributes:

• Enjoy working in a fast-paced, design-driven, product development cycle.
• Embody our Culture and Values.

Benefits

• Competitive salary and benefits package.
• Opportunities for professional growth and development.
• Collaborative and dynamic work environment.
• Access to cutting-edge technology and tools.
• Flexible work arrangements.

XML job scraping automation by YubHub

Microsoft AI are looking for a talented Principal Software Engineer at their Mountain View office. This role sits at the heart of strategic decision-making, driving innovations in AI infrastructure. You'll work directly with key partners to understand, design, and implement complex inferencing capabilities for state-of-the-art deep learning models.

About the Role

As a Principal Software Engineer, you will be responsible for engaging directly with key partners to understand, design, and implement complex inferencing capabilities for state-of-the-art deep learning models. You will work with cutting-edge hardware and software stacks to deliver best-in-class inference performance while optimizing for cost, leveraging open-source projects to advance deep learning applications. You will collaborate with external and internal teams to identify new areas for improvement and contribute to innovations that enhance model performance and deployment.

Accountabilities

• Engage directly with key partners to understand, design, and implement complex inferencing capabilities for state-of-the-art deep learning models.
• Work with cutting-edge hardware and software stacks to deliver best-in-class inference performance while optimizing for cost.

The Candidate we're looking for

Experience:

• 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.

Technical skills:

• Experience with model compression (quantization, distillation, SVD, low-rank methods).
• Experience in building high-throughput inference serving stacks (continuous batching, KV-cache optimizations, routing).

Personal attributes:

• Solid experience in GPU inference optimization (CUDA, TensorRT, Triton, or custom GPU kernels).
• Proficiency in profiling tools (Nsight, TensorBoard, PyTorch profiler) and ability to identify CPU/GPU bottlenecks.

Benefits

• Competitive salary range of USD $139,900 – $274,800 per year.
• Comprehensive benefits package, including health insurance, retirement plan, and paid time off.
• Opportunities for professional growth and development.
• Collaborative and dynamic work environment.

XML job scraping automation by YubHub

Microsoft AI are looking for a talented Principal Software Engineer at their Redmond office. This role sits at the heart of strategic decision-making, turning market data into actionable insights for a company that's revolutionising AI technology. You'll work directly with leadership to shape the company's direction in the AI market.

About the Role

As a Principal Software Engineer, you will be responsible for designing and implementing complex software systems that drive innovation in AI infrastructure. You will work with cutting-edge hardware and software stacks to deliver best-in-class inference performance while optimizing for cost, leveraging open-source projects to advance deep learning applications. You will collaborate with external and internal teams to identify new areas for improvement and contribute to innovations that enhance model performance and deployment.

Accountabilities

• Engage directly with key partners to understand, design, and implement complex inferencing capabilities for state-of-the-art deep learning models, driving innovations in AI infrastructure.
• Work with cutting-edge hardware and software stacks to deliver best-in-class inference performance while optimizing for cost, leveraging open-source projects to advance deep learning applications.

The Candidate we're looking for

Experience:

• Bachelor’s Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Technical skills:

• Experience with model compression (quantization, distillation, SVD, low-rank methods).
• Experience in building high-throughput inference serving stacks (continuous batching, KV-cache optimizations, routing).

Personal attributes:

• Solid experience in GPU inference optimization (CUDA, TensorRT, Triton, or custom GPU kernels).
• Proficiency in profiling tools (Nsight, TensorBoard, PyTorch profiler) and ability to identify CPU/GPU bottlenecks.

Benefits

• Competitive salary
• Comprehensive benefits package
• Opportunities for professional growth and development
• Collaborative and dynamic work environment

XML job scraping automation by YubHub