Key responsibilities include:

• Designing and developing engaging learning experiences for internal teams and external customers
• Translating complex system capabilities, procedures, and workflows into clear instructional content
• Applying instructional design methodologies and learning principles to maximize learner engagement and retention
• Developing interactive learning activities, simulations, and scenario-based content aligned to real operational use cases

The ideal candidate will have a Bachelor's degree in instructional design, education, or a related field, and proven experience designing and delivering training in technical environments. Proficiency with eLearning authoring tools and experience designing and delivering instructor-led training are also essential.

In addition to a competitive salary, Saronic offers a comprehensive benefits package, including medical insurance, dental and vision insurance, time off, parental leave, and retirement plan.

This role requires access to export-controlled information or items that require 'U.S. Person' status.

XML job scraping automation by YubHub

Responsibilities: Research on post-training (e.g., RL and SFT) for information-seeking scenarios in Gemini Research on novel evaluation methods for improving model quality, grounding, and factuality Research on orchestration of tool calls, and improved retrieval methods, for information-seeking scenarios

About You: To succeed in this role, you should have a PhD in a relevant area, or an equivalent research/publication record. You should also have strong software-engineering skills in addition to a research background. Experience in reinforcement learning, post-training methods, and LLMs for information-seeking scenarios is an advantage.

XML job scraping automation by YubHub

Location

San Francisco

Employment Type

Full time

Location Type

Hybrid

Department

Applied AI

Compensation

• $230K – $325K • Offers Equity

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the team

Applied Evals defines what good looks like for safe, advanced AI systems. We turn complex, high-value workflows into clear, reproducible signals that guide model training and product quality. Our work bridges frontier customers and models, ensuring improvements show up where users experience them. We combine hands-on, unscalable efforts with systems that others can extend, creating a compounding loop of model improvement.

About the Role

We’re hiring product-minded engineers to design and build evals and harnesses that capture real-world quality for advanced AI systems. You’ll own the loop from prototyping with users to building reliable pipelines and integrating signals into training stacks. This role sits at the center of model improvement. The systems you design will directly shape how models behave, accelerate their reliability, and raise the standard for what customers expect.

You’ll collaborate closely with research and product teams and work across the stack, from backend pipelines to user-facing interfaces. The work includes evaluating multi-turn and tool-using systems, designing agent harnesses, and applying reinforcement learning and related methods in production settings. Engineers who succeed in this role bring both a builder’s mindset and the judgment to create reusable systems that others can build on. Many thrive here by operating like founders or founding engineers, taking initiative, moving quickly, and creating structure where none exists.

This role is based in our San Francisco HQ. We use a hybrid work model of 3 days in the office per week and offer relocation assistance.

In this role, you will:

• Define the core evaluation signals that drive model improvement at OpenAI, turning vague product gaps into crisp, defensible measures of quality

• Design agents, harnesses, and eval pipelines that are reliable, reproducible, and extendable

• Prototype solutions with real workflows and convert them into scalable feedback loops

• Connect evaluation signals directly to research and training systems so product improvements show up in what users experience

• Shape model interaction paradigms by partnering with engineering, research, and product teams on how models are deployed and measured

• Build reusable systems and tools that enable contributions from across the company and steadily raise the quality bar

You’ll thrive in this role if you:

• Bring 4+ years of experience in software engineering with strong fundamentals and a track record of shipping production systems end-to-end

• Have experience building AI agents or applications, including designing evals and improving performance through prompting or scaffolding

• Are familiar with evaluation methods for LLMs and have worked with patterns like multi-agent workflows, tool use, or long context.

• Are familiar with deep learning concepts or have prior exposure to training models.

• Communicate clearly across technical and non-technical audiences across levels

• Are motivated by high-impact collaboration with research and product teams and thrive in ambiguity

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

XML job scraping automation by YubHub

San Francisco; Seattle

Employment Type

Full time

Department

Applied AI

Compensation

• $230K – $385K • Offers Equity

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the Team

The Support Automation team at OpenAI scales the organization by applying cutting-edge AI models to real-world challenges, automating and enhancing work across the organization. From customer operations to engineering, we develop an ecosystem of automation products that empower our colleagues and drive impact. We're passionate about crafting products that serve those around us, blending rapid prototyping with a focus on long-term quality and reliability. By creating reusable solutions, we create patterns that can be applied across diverse domains within OpenAI.

TLDR: this team leverages OpenAI technology to improve OpenAI, and you’ll have the opportunity to leverage the full extent of our tech (both public and pre-released) to accomplish this mission.

About the Role

We’re looking for a Backend Software Engineer with experience working in ML/LLM-heavy domains to help to design and build an evals infrastructure that measures the quality of OpenAI’s support automation. This is a deeply technical and highly cross-functional role where you’ll build robust systems and backend services that serve as the foundation for how knowledge is created, accessed, and applied across OpenAI. The role will especially focus on working closely with Data Science and Research partners to design and build evals at scale.

In this role, you will:

• Design eval pipelines that are reliable, reproducible, and extendable

• Build the infrastructure for continuous eval monitoring frameworks (regression/drift monitoring, building robust golden datasets) along with feedback loops that ultimately strengthen support automation

• Design, build, and maintain backend services and APIs to support intelligent automation and knowledge systems

• Integrate and structure data across internal platforms, transforming it into formats optimized for use by downstream systems and AI workflows.

• Collaborate closely with data, research, and engineering teams to integrate OpenAI models into high-leverage workflows

• Own the full development lifecycle of new backend systems and internal platform capabilities

• Build with scale and maintainability in mind, while rapidly iterating on new ideas

You might be a great fit if you have:

• 4+ years of backend engineering experience at product-driven companies (excluding internships)

• Proficiency in backend technologies. Our tech stack includes Python, FastAPI, and Postgres

• Experience designing and scaling distributed systems, APIs, or data processing pipelines

• Have experience building AI agents or applications, including designing evals and improving performance through prompting or scaffolding

• Are familiar with evaluation methods for LLMs and have worked with patterns like multi-agent workflows, tool use, or long context.

• Experience creating production evals and/or measuring performance of ML/LLM models at scale

• A pragmatic mindset. You’re comfortable shipping iteratively while building toward a long-term vision

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

XML job scraping automation by YubHub

San Mateo, CA, United States

Early Career

ID: 5825

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators.

At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device.

We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there.

A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone.

WHY DATA SCIENCE & ANALYTICS?

The Data Science & Analytics organization's mission is to increase our speed, frequency, and acumen in making decisions at scale by instilling a data-influenced approach to building products. We cover a wide area of the data spectrum, including analytical data engineering, product analytics, experimentation, causal inference, statistical modeling, and machine learning. Aligned and partnered with product verticals, we use this extensive tool belt to discover new opportunities and unmet use cases, influence and craft the product roadmap, and prioritize, build data products, and measure impact on our community of players and developers.

WHY GENERATIVE AI?

Our team’s mission is to enable Roblox Creators to bring GenAI capabilities to millions of users. We drive this innovation with a core commitment to safety, responsibility, and quality.

As a Data Scientist, you will play a critical role in evaluation and optimization for user-facing GenAI systems (such as text, image, video, 3D, 4D). You will define how we measure safety, responsibility, quality, and efficiency. You will combine annotation analysis, design of experiments, causal inference, model-based evaluation methods (such as LLM-as-a-judge), optimization algorithm, and AI models to drive product decisions and model improvements.

You Will:

• Develop Evaluation Frameworks: Design and operationalize rigorous evaluation systems for either GenAI features (text, image, video, 3D, 4D). This includes eval experiment design, dataset design, label reliability analysis, and implementing and finetuning LLM-as-judge methods.

• Run Rigorous Experiments: Conduct online experiments (A/B tests) and causal inference to quantify the impact of GenAI features. You will identify opportunities, measure lift, and ensure statistical rigor.

• Define Success Metrics: Partner with cross-functional teams to define leading/lagging indicators for GenAI feature user satisfaction, business success, and safety.

• Build Automated Systems: Research and apply state-of-the-art methodologies to build reproducible evaluation tooling that lift rigor and efficiency across the company.

• Conduct Applied Research at the Frontier: Maintain an active pulse on the intersection of Gen AI and Data Science. You will innovate on methodology and techniques to solve unique business challenges while contributing to the broader field in the technical community.

You Have:

• Possess or pursuing a PhD or equivalent in Statistics, Economics, Computer Science, Applied Math, Physics, Engineering, or a related quantitative field.

• Technical Proficiency: Strong proficiency in SQL (Hive/Spark) for manipulating large datasets and scripting languages (Python or R) for analysis and modeling.

• Experimentation and Causal Inference: A solid grounding in experimentation, causal inference, and statistical analysis, including test design and metric design for feature impact.

• Problem Solving: A demonstrated track record of framing ambiguous problems, designing analytical approaches, and solving open-ended data science problems that drive business impact.

• Learning Agility: Ability to effectively and responsibly use AI tools to enhance productivity and a passion for continuously improving methods in a fast-evolving field.

• GenAI Familiarity: Familiarity with GenAI models and safety/quality evaluation methods. Expertise in the model training lifecycle is a plus (e.g., fine-tuning, RLHF, or synthetic data generation).

• Applied Research Background: A track record of applied research or publications in relevant technical fields is highly valued.

You may redact age, date of birth, and dates of attendance/graduation from your resume if you prefer.

For roles that are based at our headquarters in San Mateo, CA: The starting base pay for this position is as shown below. The actual base pay is dependent upon a variety of job-related factors such as professional background, training, work experience, location, business needs and market demand. Therefore, in some circumstances, the actual salary could fall outside of this expected range. This pay range is subject to change and may be modified in the future. All full-time employees are also eligible for equity compensation and for benefits as described on this page.

Annual Salary Range

$185,860—$221,380 USD

Roles that are based in an office are onsite Tuesday, Wednesday, and Thursday, with optional presence on Monday and Friday (unless otherwise noted).

XML job scraping automation by YubHub