As a hands-on contributor to the core systems that ensure the safety, reliability, and continuous improvement of LLM-powered workflows and agents for the enterprise, you will partner with Scale's Operations team and enterprise customers to translate ambiguity into structured evaluation data. This involves guiding the creation and maintenance of gold-standard human-rated datasets and expert rubrics that anchor AI evaluation systems.

Your responsibilities will also include analysing feedback and collected data to identify patterns, refine evaluation frameworks, and establish iterative improvement loops that enhance the quality and relevance of human-curated assessments. You will design, research, and develop LLM-as-a-Judge autorater frameworks and AI-assisted evaluation systems, including creating models that critique, grade, and explain agent outputs.

To succeed in this role, you will need a strong foundational knowledge of large language models, a passion for tackling complex evaluation challenges, and the ability to thrive in a dynamic, fast-paced research environment. You should be able to think outside the box, stay current with the latest literature in AI evaluation, and be passionate about integrating novel research ideas into our workflows to build best-in-class evaluation systems.

In addition to your technical expertise, you will need excellent communication and collaboration skills, as you will work closely with cross-functional teams to drive project success.

If you are a motivated and detail-oriented individual with a passion for AI research and evaluation, we encourage you to apply for this exciting opportunity.

XML job scraping automation by YubHub

Researcher, Frontier Biological and Chemical Risks

Location

San Francisco

Employment Type

Full time

Department

Safety Systems

Compensation

• Estimated Base Salary $295K – $445K

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the Team

The Preparedness team is an important part of the Safety Systems org at OpenAI, and is guided by OpenAI’s Preparedness Framework.

Frontier AI models have the potential to benefit all of humanity, but also pose increasingly severe risks. To ensure that AI promotes positive change, the Preparedness team helps us prepare for the development of increasingly capable frontier AI models. This team is tasked with identifying, tracking, and preparing for catastrophic risks related to frontier AI models.

The mission of the Preparedness team is to:

1. Closely monitor and predict the evolving capabilities of frontier AI systems, with an eye towards misuse risks whose impact could be catastrophic to our society

1. Ensure we have concrete procedures, infrastructure and partnerships to mitigate these risks and to safely handle the development of powerful AI systems

Preparedness tightly connects capability assessment, evaluations, and internal red teaming, and mitigations for frontier models, as well as overall coordination on AGI preparedness. This is fast paced, exciting work that has far reaching importance for the company and for society.

About the Role

We are looking to hire exceptional research engineers that can push the boundaries of our frontier models. Specifically, we are looking for those that will help us shape our empirical grasp of the whole spectrum of AI safety concerns and will own individual threads within this endeavor end-to-end.

You will own the scientific validity of our frontier preparedness capability evaluations—designing new evals grounded in real threat models (including high-consequence domains like CBRN as well as cyber and other frontier-risk areas), and maintaining existing evals so they don’t stale or silently regress. You’ll define datasets, graders, rubrics, and threshold guidance, and produce auditable artifacts (evaluation cards, capability reports, system-card inputs) that leadership can trust during high-stakes launches.

In this role, you'll:

• Work on identifying emerging AI safety risks and new methodologies for exploring the impact of these risks

• Build (and then continuously refine) evaluations of frontier AI models that assess the extent of identified risks

• Design and build scalable systems and processes that can support these kinds of evaluations

• Contribute to the refinement of risk management and the overall development of “best practice” guidelines for AI safety evaluations

You might thrive in this role if you:

• Are passionate and knowledgeable about short-term and long-term AI safety risks

• Demonstrate the ability to think outside the box and have a robust “red-teaming mindset”

• Have experience in ML research engineering, ML observability and monitoring, creating large language model-enabled applications, and/or another technical domain applicable to AI risk

• Are able to operate effectively in a dynamic and extremely fast-paced research environment as well as scope and deliver projects end-to-end

It would be great if you also have:

• First-hand experience in red-teaming systems—be it computer systems or otherwise

• A good understanding of the (nuances of) societal aspects of AI deployment

• Excellent communication skills and the ability to work cross-functionally

_This role may require access to technology or technical data controlled under the U.S. Export Administration Regulations or International Traffic in Arms Regulations. Therefore, this role is restricted to individuals described in paragraph (a)(1) of the definition of “U.S. person” in the U.S. Export Administration Regulations, 15 C.F.R. § 772.1, and in the International Traffic in Arms Regulations, 22 C.F.R. § 120.62. U.S. persons are U.S. citizens, U.S. legal permanent residents, individuals granted asylum status in the United States, and individuals who are not U.S. citizens but are lawfully admitted for permanent residence in the United States._

XML job scraping automation by YubHub

Research Engineer, Frontier Evals & Environments

Location

San Francisco

Employment Type

Full time

Department

Research

Compensation

• $205K – $380K • Offers Equity

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the team

The Frontier Evals & Environments team builds north star model environments to drive progress towards safe AGI/ASI. This team builds ambitious environments to measure and steer our models, and creates self-improvement loops to steer our training, safety, and launch decisions. Some of the team's open-sourced evaluations include GDPval, SWE-bench Verified, MLE-bench, PaperBench, and SWE-Lancer, and the team built and ran frontier evaluations for GPT4o, o1, o3, GPT 4.5, ChatGPT Agent, and GPT5. If you are interested in feeling firsthand the fast progress of our models, and steering them towards good, this is the team for you.

About you

We seek exceptional research engineers that can push the boundaries of our frontier models. Specifically, we are looking for those that will help us shape our empirical grasp of the whole spectrum of AI capabilities measurement and will own individual threads within this endeavor end-to-end.

In this role, you'll:

• Create ambitious RL environments to push our models to their limits

• Work on measuring frontier model capabilities, skills, and behaviors

• Develop new methodologies for automatically exploring the behavior of these models

• Help steer training for our largest training runs, and see the future first

• Design scalable systems and processes to support continuous evaluation

• Build self-improvement loops to automate model understanding

We expect you to be:

• Passionate and knowledgeable about AGI/ASI measurement

• Strong engineering and statistical analysis skills

• Able to think outside the box and have a robust “red-teaming mindset”

• Experienced in ML research engineering, stochastic systems, observability and monitoring, LLM-enabled applications, and/or another technical domain applicable to AI evaluations

• Able to operate effectively in a dynamic and extremely fast-paced research environment as well as scope and deliver projects end-to-end

It would be great if you also have:

• First-hand experience in red-teaming systems—be it computer systems or otherwise

• An ability to work cross-functionally

• Excellent communication skills

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

XML job scraping automation by YubHub