You will help define best practices for developers building on claude for deployment to users across different developmental stages, design age-assurance policies that protect minors from inappropriate content and interactions, and establish clear boundaries for adult content and experiences. In addition, you will advise teams on opportunities for age-appropriate helpfulness, including advising cross-functional teams on beneficial use cases for younger users where appropriate.

Safety is core to our mission and you’ll help shape policy creation and development so that our users can safely interact with and build on top of our products in a harmless, helpful and honest way.

Responsibilities:

• Serve as an internal subject matter expert, leveraging deep expertise in child safety, adult content, youth development, and age-appropriate design to:

• Draft new policies that help govern the responsible use of our models for emerging capabilities and use cases

• Design evaluation frameworks for testing model performance in areas of expertise

• Conduct regular reviews and testing of existing policies to identify and address gaps and ambiguities

• Review flagged content to drive enforcement and policy improvements

• Update our usage policies based on feedback collected from external experts, our enforcement team, and edge cases that you will review

• Work with safeguards product teams to identify and mitigate concerns, and collaborate on designing appropriate interventions for users across different age groups

• Advise on age assurance approaches and content classification frameworks in partnership with Enforcement, Product, Engineering, and Legal teams

• Educate and align internal stakeholders around our policies and our approach to safety in your focus area(s)

• Keep up to date with new and existing AI policy norms, regulatory requirements (e.g., age-appropriate design codes), and industry standards, and use these to inform our decision-making on policy areas

Requirements:

• As a researcher, subject matter expert, or trust & safety professional working in one or more of the following focus areas: child safety, youth online safety, age assurance, developmental science, content classification and rating systems, or adult content policy.

• Drafting or updating product and / or user policies, with the ability to effectively bridge technical and policy discussions

• Designing or implementing age-appropriate experiences, age assurance mechanisms, or content classification / labeling systems

• Working with generative AI products, including writing effective prompts for policy evaluations and classifier development

• Aligning product policy decisions between diverse sets of stakeholders, such as Product, Engineering, Public Policy, and Legal teams

• Understanding the challenges that exist in developing and implementing product policies at scale, including in the content moderation space

• Thinking creatively about the risks and benefits of new technologies, and leveraging data and research to inform policy recommendations

• Navigating and prioritizing work efforts amidst ambiguity

Salary:

$245,000-$285,000 USD

XML job scraping automation by YubHub

You will be a member of the user well-being team, with an initial focus on standing up detection, review, and escalation workflows for this domain , from tuning classifiers and curating evaluation datasets through to managing external partnerships and real-world harm escalation pathways.

This position may later expand to include broader areas of user well-being enforcement. Safety is core to our mission, and you'll help shape policy enforcement so that our users can interact with and build on top of our products across all surfaces in a harmless, helpful, and honest way.

In this position, you may be exposed to and engage with explicit content spanning a range of topics, including those of a sexual, violent, or psychologically disturbing nature. There is also an on-call responsibility across the Policy and Enforcement teams.

Responsibilities:

• Design and architect automated enforcement systems and review workflows for human exploitation and abuse, ensuring they scale effectively while maintaining high accuracy

• Partner with Product, Engineering, and Data Science teams to build and tune detection signals for human trafficking, sextortion, and image-based sexual abuse, and to develop custom mitigations for these sensitive policy areas

• Curate policy violation examples, maintain golden evaluation datasets, and track enforcement actions across both consumer and API surfaces

• Conduct deep-dive investigations into suspected exploitation activity , using SQL and other data analysis tools to surface threat patterns and bad-actor behavior in large datasets , then produce clear, well-sourced intelligence reports that inform detection strategy and surface policy gaps to the Safeguards policy design team

• Study trends internally and in the broader ecosystem , including evolving trafficking and sextortion tactics , to anticipate how AI systems could be misused for exploitation as capabilities advance

• Review and investigate flagged content to drive enforcement decisions and policy improvements, exercising careful judgment on the line between permitted adult content and exploitative material

• Build and maintain relationships with external intelligence partners , including hotlines, NGOs, and industry hash-sharing consortia , to inform our approach and enable appropriate real-world escalation

You may be a good fit if you have:

• 3+ years of experience in trust and safety, content moderation, counter-exploitation work, or a related field

• Subject matter expertise in one or more of: human trafficking, human exploitation and abuse, sextortion, image-based sexual abuse / non-consensual intimate imagery, or commercial sexual exploitation

• Experience building or operating detection and review workflows for sensitive content, at a platform, NGO, hotline, or similar organization

• Ability to use SQL, Python, and/or other data analysis tools to interact with large datasets and derive insights that support key decisions and recommendations

• Demonstrated ability to analyze complex situations and make well-reasoned decisions under pressure

• Sound judgment in distinguishing permitted content from exploitative content, and comfort working in areas where these lines require careful reasoning

• Strong attention to detail and ability to maintain accurate documentation

• Ability to collaborate with team members while navigating rapidly evolving priorities and workstreams

Preferred:

• Familiarity with the NGO and industry ecosystem working on these harms (for example, Polaris Project, Thorn, NCMEC, IWF, StopNCII, or industry hash-sharing initiatives)

• Experience conducting open-source investigations or threat actor profiling in a trust & safety, intelligence, or law enforcement context

• Experience working with generative AI products, including writing effective prompts for content review and enforcement

• A deep interest in AI safety and responsible technology development

• Experience standing up real-world harm escalation pathways or working with law enforcement referral processes

The annual compensation range for this role is listed below.

For sales roles, the range provided is the role’s On Target Earnings (“OTE”) range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.

Annual Salary: $245,000-$285,000 USD

XML job scraping automation by YubHub

You will be a member of the user well-being team, and your initial focus will be on standing up detection, review, and escalation workflows for this domain , from tuning classifiers and curating evaluation datasets through to managing external partnerships and real-world harm escalation pathways.

This position may later expand to include broader areas of user well-being enforcement. Safety is core to our mission, and you'll help shape policy enforcement so that our users can interact with and build on top of our products across all surfaces in a harmless, helpful, and honest way.

In this position, you may be exposed to and engage with explicit content spanning a range of topics, including those of a sexual, violent, or psychologically disturbing nature. There is also an on-call responsibility across the Policy and Enforcement teams.

Responsibilities:

• Design and architect automated enforcement systems and review workflows for human exploitation and abuse, ensuring they scale effectively while maintaining high accuracy

• Partner with Product, Engineering, and Data Science teams to build and tune detection signals for human trafficking, sextortion, and image-based sexual abuse, and to develop custom mitigations for these sensitive policy areas

• Curate policy violation examples, maintain golden evaluation datasets, and track enforcement actions across both consumer and API surfaces

• Conduct deep-dive investigations into suspected exploitation activity , using SQL and other data analysis tools to surface threat patterns and bad-actor behavior in large datasets , then produce clear, well-sourced intelligence reports that inform detection strategy and surface policy gaps to the Safeguards policy design team

• Study trends internally and in the broader ecosystem , including evolving trafficking and sextortion tactics , to anticipate how AI systems could be misused for exploitation as capabilities advance

• Review and investigate flagged content to drive enforcement decisions and policy improvements, exercising careful judgment on the line between permitted adult content and exploitative material

• Build and maintain relationships with external intelligence partners , including hotlines, NGOs, and industry hash-sharing consortia , to inform our approach and enable appropriate real-world escalation

Requirements:

• 3+ years of experience in trust and safety, content moderation, counter-exploitation work, or a related field

• Subject matter expertise in one or more of: human trafficking, human exploitation and abuse, sextortion, image-based sexual abuse / non-consensual intimate imagery, or commercial sexual exploitation

• Experience building or operating detection and review workflows for sensitive content, at a platform, NGO, hotline, or similar organization

• Ability to use SQL, Python, and/or other data analysis tools to interact with large datasets and derive insights that support key decisions and recommendations

• Demonstrated ability to analyze complex situations and make well-reasoned decisions under pressure

• Sound judgment in distinguishing permitted content from exploitative content, and comfort working in areas where these lines require careful reasoning

• Strong attention to detail and ability to maintain accurate documentation

• Ability to collaborate with team members while navigating rapidly evolving priorities and workstreams

Preferred Qualifications:

• Familiarity with the NGO and industry ecosystem working on these harms (for example, Polaris Project, Thorn, NCMEC, IWF, StopNCII, or industry hash-sharing initiatives)

• Experience conducting open-source investigations or threat actor profiling in a trust & safety, intelligence, or law enforcement context

• Experience working with generative AI products, including writing effective prompts for content review and enforcement

• A deep interest in AI safety and responsible technology development

• Experience standing up real-world harm escalation pathways or working with law enforcement referral processes

Compensation:

The annual compensation range for this role is $245,000-$285,000 USD.

XML job scraping automation by YubHub

You will help define best practices for developers building on claude for deployment to users across different developmental stages, design age-assurance policies that protect minors from inappropriate content and interactions, and establish clear boundaries for adult content and experiences. In addition, you will advise teams on opportunities for age-appropriate helpfulness, including advising cross-functional teams on beneficial use cases for younger users where appropriate.

Safety is core to our mission and you’ll help shape policy creation and development so that our users can safely interact with and build on top of our products in a harmless, helpful and honest way.

You may be a good fit if you have experience:

As a researcher, subject matter expert, or trust & safety professional working in one or more of the following focus areas: child safety, youth online safety, age assurance, developmental science, content classification and rating systems, or adult content policy.

Drafting or updating product and / or user policies, with the ability to effectively bridge technical and policy discussions.

Designing or implementing age-appropriate experiences, age assurance mechanisms, or content classification / labeling systems.

Working with generative AI products, including writing effective prompts for policy evaluations and classifier development.

Aligning product policy decisions between diverse sets of stakeholders, such as Product, Engineering, Public Policy, and Legal teams.

Understanding the challenges that exist in developing and implementing product policies at scale, including in the content moderation space.

Thinking creatively about the risks and benefits of new technologies, and leveraging data and research to inform policy recommendations.

Navigating and prioritizing work efforts amidst ambiguity.

The annual compensation range for this role is $245,000-$285,000 USD.

XML job scraping automation by YubHub

To be successful in this role, you will need expertise in improving Large Language Models (LLMs), specifically related to CSE, to maximize efficiencies in enforcement and support. You will also need to have a proven expertise in identifying, mitigating, and preventing Child Sexual Abuse Material (CSAM) and Child Sexual Exploitation (CSE), including grooming behaviors and risks in AI-generated content, with strong knowledge of relevant legal obligations (such as NCMEC reporting) and industry standards for protecting minors.

You will also have experience in online safety and reducing harm to protect our users and preserve Free Speech in the global public square. You will be able to interpret and apply xAI safety policies effectively, and have strong skills in ethical reasoning and risk assessment. You will also have a strong ability to utilize resources, guidelines, and frameworks for accurate safety-focused actions and escalations.

In addition, you will have strong communication, interpersonal, analytical, and ethical decision-making skills. You will be committed to continuous improvement of processes to prioritize safety and risk mitigation. You will also have expertise in data analysis to identify emerging abuse vectors, uncover opportunities for operational efficiencies, and design automations that strengthen enforcement effectiveness and platform safety.

Preferred qualifications include experience working in a Trust and Safety for a social media company, leveraging AI or other automation tools. You will also have experience collaborating with child safety organizations (such as NCMEC) and utilizing specialized detection tools or developing classifiers for CSAM/CSE in social media or generative AI platforms. Additionally, you will have expertise in red-teaming and adversarial testing of Large Language Models to proactively identify novel abuse vectors, jailbreaks, and safety failure modes, with a proven ability to translate findings into concrete improvements for enforcement systems and platform robustness.

This role may involve exposure to sensitive or graphic content, including vulgar language, violent threats, pornography, and other graphic images.

XML job scraping automation by YubHub

xAI is seeking a Senior Analyst - Safety Operations (CSE) to join our team. As a Senior Analyst, you will play a critical role in ensuring the safety and integrity of our AI systems.

Responsibilities

• Process appeals, audit automations, and properly label use cases in the system.
• Provide labels, annotations, and inputs on projects involving safety protocols, risk scenarios, and policy compliance.
• Support the delivery of high-quality curated data that reinforces xAI's rules and ethical alignment.
• Collaborate with team members to provide feedback on tasks that improve AI's defenses to detect illegal and unethical behavior, as well as align Grok with our rules enforcement.

Basic Qualifications

• Expertise in improving Large Language Models (LLMs), specifically related to CSE, to maximize efficiencies in enforcement and support and ability to propose solutions to increase security and safety of our platform.
• Proven expertise in identifying, mitigating, and preventing Child Sexual Abuse Material (CSAM) and Child Sexual Exploitation (CSE), including grooming behaviors and risks in AI-generated content, with strong knowledge of relevant legal obligations (such as NCMEC reporting) and industry standards for protecting minors.
• Proven experience in online safety and reducing harm to protect our users and preserve Free Speech in the global public square.
• Ability to interpret and apply xAI safety policies effectively.
• Proficiency in analyzing complex scenarios, with strong skills in ethical reasoning and risk assessment.
• Strong ability to utilize resources, guidelines, and frameworks for accurate safety-focused actions and escalations.
• Strong communication, interpersonal, analytical, and ethical decision-making skills.
• Commitment to continuous improvement of processes to prioritize safety and risk mitigation.
• Expertise in data analysis to identify emerging abuse vectors, uncover opportunities for operational efficiencies, and design automations that strengthen enforcement effectiveness and platform safety.

Preferred Skills and Experience

• Experience working in a Trust and Safety for a social media company, leveraging AI or other automation tools.
• Experience collaborating with child safety organizations (such as NCMEC) and utilizing specialized detection tools or developing classifiers for CSAM/CSE in social media or generative AI platforms.
• Expertise in red-teaming and adversarial testing of Large Language Models to proactively identify novel abuse vectors, jailbreaks, and safety failure modes, with a proven ability to translate findings into concrete improvements for enforcement systems and platform robustness.

This role may involve exposure to sensitive or graphic content, including vulgar language, violent threats, pornography, and other graphic images.

XML job scraping automation by YubHub

Trust & Safety Operations Analyst

Location

San Francisco

Employment Type

Full time

Department

Compensation

• $189K – $280K • Offers Equity

The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.

• Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts

• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)

• 401(k) retirement plan with employer match

• Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)

• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees

• 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)

• Mental health and wellness support

• Employer-paid basic life and disability coverage

• Annual learning and development stipend to fuel your professional growth

• Daily meals in our offices, and meal delivery credits as eligible

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.

More details about our benefits are available to candidates during the hiring process.

This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.

About the Team

At OpenAI, our Trust, Safety & Risk Operations teams safeguard our products, users, and the company from abuse, fraud, scams, regulatory non-compliance, and other emerging risks. We operate at the intersection of operations, compliance, user trust, and safety working closely with Legal, Policy, Engineering, Product, Go-To-Market, and external partners to ensure our platforms are safe, compliant, and trusted by a diverse, global user base.

We support users across ChatGPT, our API, enterprise offerings, and developer tools handling sensitive inbound cases, building detection and enforcement systems, and scaling operational processes to meet the demands of a fast-moving, high-stakes environment.

About the Role

We are seeking experienced, senior-level analysts who specialize in one or more of the following areas:

• Content Integrity & Scaled Enforcement – Detecting, reviewing, and acting on policy violations, harmful content, and emerging abuse patterns at scale.

• Emerging Risk Operations – Identifying, triaging, and mitigating new and complex safety, policy, or integrity challenges in a rapidly evolving AI landscape.

In this role, you will own high-sensitivity workflows, act as an incident manager for complex cases, and build scalable operational systems; including tooling, automation, and vendor processes that reinforce user safety and trust while meeting our legal, ethical, and product obligations.

We use a hybrid work model of 3 days in the San Francisco office per week and offer relocation assistance to new employees.

Please note: This role may involve exposure to sensitive content, including material that is sexual, violent, or otherwise disturbing.

In This Role, You Will:

• Handle and resolve high-priority cases in your area of specialization (scaled content enforcement, fraud/scams, privacy/regulatory, or emerging risks).

• Perform in-depth risk evaluations and investigations using internal tools, product signals, and third-party data.

• Act as incident manager for escalations requiring nuanced policy, legal, or regulatory interpretation.

• Partner with cross-functional teams to design and implement world-class operational workflows, decision trees, and automation strategies.

• Build feedback loops from casework to inform product, engineering, and policy improvements.

• Develop and maintain playbooks, SOPs, macros, and knowledge resources for internal teams and vendors.

• Lead or contribute to cross-functional projects, from zero-to-one process builds to global operational scale-ups.

• Monitor operational health through case quality audits, SLA adherence, escalation accuracy, and user satisfaction metrics.

• Train and support vendor teams, ensuring consistent quality and alignment with OpenAI’s trust and safety standards.

You Might Thrive in This Role If You:

• Have 5+ years of experience in one or more of: trust & safety, fraud prevention, scam investigation, privacy/legal operations, compliance, or other risk/integrity domains ideally in a global or high-growth tech environment.

• Leverage OpenAI technology to enhance workflows, improve decision-making, and scale operational impact.

• Bring deep domain expertise in your specialization area and familiarity with relevant legal, policy, and technical frameworks.

• Have a track record of scaling operations, building processes, and working cross-functionally to improve performance and safety outcomes.

• Possess exceptional analytical skills able to detect patterns, assess risk, and recommend policy or product changes based on evidence.

• Communicate with clarity, empathy, and precision especially in sensitive user-facing contexts.

• Thrive in ambiguous, high-autonomy environments and balance speed with diligence.

Are comfortable with frequent context switching, managing multiple projects, and prioritizing impact.

What We Offer

• Competitive salary and equity package

• Comprehensive benefits, including medical, dental, and vision insurance

• 401(k) retirement plan with employer match

• Paid parental leave and medical and caregiver leave

• Flexible PTO and paid company holidays

• Mental health and wellness support

• Annual learning and development stipend

• Daily meals in our offices and meal delivery credits

• Relocation support for eligible employees

• Additional taxable fringe benefits, such as charitable donation matching and wellness stipends

How to Apply

If you are a motivated and experienced professional looking to join a dynamic team, please submit your application, including your resume and a cover letter, to [insert contact information]. We look forward to hearing from you!

XML job scraping automation by YubHub