We're seeking a Tech Lead Manager to lead a talented team of research scientists and research engineers focused on developing and implementing novel evaluation methodologies, metrics, and benchmarks to assess the capabilities and limitations of our cutting-edge LLMs.

Key responsibilities:

• Lead a team of highly effective research scientists and research engineers on LLM evals.
• Conduct research on the effectiveness and limitations of existing LLM evaluation techniques.
• Design and develop novel evaluation benchmarks for large language models, covering areas such as instruction following, factuality, robustness, and fairness.
• Communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects.
• Collaborate with internal teams and external partners to refine metrics and create standardized evaluation protocols.
• Implement scalable and reproducible evaluation pipelines using modern ML frameworks.
• Publish research findings in top-tier AI conferences and contribute to open-source benchmarking initiatives.

Ideal candidate has 5+ years of hands-on experience in large language model, NLP, and Transformer modeling, in the setting of both research and engineering development. Experience supporting and leading a team of research scientists and research engineers is also required.

XML job scraping automation by YubHub

Join the team as Twilio's next Software Engineer L3. This position is that of a Senior Software Engineer to join Twilio's Messaging Compliance Onboarding team. Programmable Messaging is Twilio's biggest product. To keep pace with the evolving messaging compliance ecosystem, we need strong engineers that can create innovative solutions to ensure compliance with Twilio partners.

In this role, you'll build and maintain multiple compliance program workflows, carrier/ecosystem integrations and customer interactions in the Compliance domain. You will design and develop elegant and scalable solutions across a wide variety of compliance program types including frontend UI experiences and backend APIs, that are highly available and responsive.

You will work through ambiguity, deliver quickly and with a high quality. Build towards achieving the next generation of architecture vision that empowers expansion of Compliance programs. Interact cross functionally across engineering teams within Twilio to align and build the product and architecture vision.

Twilio values diverse experiences from all kinds of industries, and we encourage everyone who meets the required qualifications to apply. If your career is just starting or hasn't followed a traditional path, don't let that stop you from considering Twilio.

We are always looking for people who will bring something new to the table!

*Required:

• 5+ years of experience and a strong fundamental knowledge of software development using JVM languages.

• Experience building web services incorporating best practices for external systems integration, including defensive and hardened approaches to mitigate downstream issues.

• Experience working with highly scalable APIs, high volume data pipelines and large distributed systems.

• Maintaining and operating cloud services.

• An unwillingness to settle for 'good enough', instead staying focused on longevity through well-tested code and continuous improvement.

• Demonstrated commitment to seeking diverse viewpoints and acting with intention to create an inclusive team environment.

• Excellent written and verbal communication skills. Ability to write down and present designs and decisions throughout the development lifecycle, collaborating with engineering and non-engineering roles.

Desired:

• 5+ years of Engineering experience, developing and maintaining high traffic services.

• Familiarity with DynamoDB, SQS, and data integration services like AWS glue

• Familiarity with LLMs, prompt optimizations to improve model accuracy, setting up evaluation pipelines

• Familiarity with Kubernetes, Temporal or similar workflow orchestration

• Experience working with frontend libraries like React or similar.

Compensation:

*Please note this role is open to candidates outside of California, Colorado, Hawaii, Illinois, Maryland, Massachusetts, Minnesota, New Jersey, New York, Vermont, Washington D.C., and Washington State.

The estimated pay ranges for this role are as follows:

• Based in Colorado, Hawaii, Illinois, Maryland, Massachusetts, Minnesota, Vermont or Washington D.C.: $138,700 - $173,400

• Based in New York, New Jersey, Washington State, or California (outside of the San Francisco Bay area): $146,800 - $183,600

• Based in the San Francisco Bay area, California: $163,100 - $203,900.

This role may be eligible to participate in Twilio's equity plan and corporate bonus plan. All roles are generally eligible for the following benefits: health care insurance, 401(k) retirement account, paid sick time, paid personal time off, paid parental leave.

Applications for this role are intended to be accepted until 4/10/2026, but may change based on business needs.

Twilio thinks big. Do you? We like to solve problems, take initiative, pitch in when needed, and are always up for trying new things. That's why we seek out colleagues who embody our values , something we call Twilio Magic. Additionally, we empower employees to build positive change in their communities by supporting their volunteering and donation efforts. So, if you're ready to unleash your full potential, do your best work, and be the best version of yourself, apply now!

If this role isn't what you're looking for, please consider other open positions.

Twilio is proud to be an equal opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Qualified applicants with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Additionally, Twilio participates in the E-Verify program in certain locations, as required by law.

XML job scraping automation by YubHub

You will collaborate closely with technical teams to support the development and refinement of new AI capabilities, with a primary focus on quantitative financial risk management domains. Your expertise will drive the selection and rigorous resolution of complex risk-related problems, including market risk modeling, credit and counterparty risk, liquidity and funding risk, operational and model risk, stress testing & scenario analysis, Value at Risk (VaR)/Expected Shortfall (ES), risk attribution, capital allocation (economic/regulatory), and enterprise-wide risk frameworks under regulatory regimes (Basel, Dodd-Frank, IFRS 9, etc.).

This role requires exceptional quantitative rigor, rapid adaptation to evolving guidelines, and the ability to deliver precise, technically sound critiques, derivations, and solutions in a fast-paced environment. As a Finance Risk Expert, you will directly support xAI's mission by helping train and refine frontier AI models. You will teach the models how risk professionals quantify uncertainties, model tail events, assess portfolio vulnerabilities, ensure regulatory compliance, perform stress testing, and make data-driven decisions to protect capital and maintain financial stability.

Your tasks may include recording audio walkthroughs of risk models, participating in video-based scenario reasoning, or producing detailed quantitative risk analysis traces. All outputs are considered work-for-hire and owned by xAI.

Responsibilities:

• Use proprietary annotation and evaluation software to deliver accurate labels, rankings, critiques, and comprehensive solutions on assigned projects
• Consistently produce high-quality, curated data that adheres to strict quantitative and regulatory standards
• Collaborate with engineers and researchers to develop and iterate on new training tasks, risk-specific benchmarks, and evaluation frameworks
• Provide constructive feedback to improve the efficiency, precision, and usability of annotation and data-collection tools
• Select and solve challenging problems from financial risk domains where you have deep expertise

Basic Qualifications:

• Master’s or PhD in a quantitative discipline: Quantitative Finance, Financial Engineering, Financial Mathematics, Statistics, Applied Mathematics, Econometrics, Risk Management, Operations Research, Physics, Computer Science (with risk/finance focus), or closely related field or equivalent professional experience as a quantitative risk analyst, risk modeler, or risk quant
• Excellent written and verbal English communication (technical reports, regulatory documentation, explanatory breakdowns)
• Strong familiarity with financial risk data sources and platforms (Bloomberg, Refinitiv, Moody’s Analytics, S&P Capital IQ, RiskMetrics, internal bank risk systems, regulatory filings, Basel/FRB datasets, etc.)
• Exceptional analytical reasoning, attention to detail, and ability to exercise sound judgment with incomplete or ambiguous data

Preferred Skills and Experience:

• Professional experience in quantitative risk management, model development/validation, or risk analytics at a bank, hedge fund, asset manager, insurance company, regulator, or consulting firm
• Track record of publication(s) or contributions in refereed journals/conferences on risk, econometrics, statistics, or quantitative finance
• Prior teaching, mentoring, or training experience (university, industry workshops, regulatory training)
• Proficiency in Python/R for risk modeling (pandas, NumPy, SciPy, statsmodels, QuantLib, PyTorch/TensorFlow for ML risk models, etc.) and familiarity with risk systems (Murex, Calypso, Numerix, etc.)
• Experience with Monte Carlo simulation, copula models, stochastic processes, time-series analysis, extreme value theory, or machine learning for risk (anomaly detection, credit scoring, etc.)
• Knowledge of regulatory capital frameworks (Basel III/IV, FRB CCAR, SR 11-7 model risk guidance, IFRS 9/CECL, Solvency II)
• CFA, FRM, PRM, CQF, or similar risk-focused certifications
• Previous exposure to large language models, AI safety, or quantitative evaluation pipelines

Location and Other Expectations:

• Tutor roles may be offered as full-time, part-time, or contractor positions, depending on role needs and candidate fit
• For contractor positions, hours will vary widely based on project scope and contractor availability, with no fixed commitments required
• Tutor roles may be performed remotely from any location worldwide, subject to legal eligibility, time-zone compatibility, and role specific needs
• For US based candidates, please note we are unable to hire in the states of Wyoming and Illinois at this time
• We are unable to provide visa sponsorship
• For those who will be working from a personal device, your computer must meet xAI’s minimum hardware requirements

XML job scraping automation by YubHub

As a Model Behavior Architect at Mistral AI, you will be at the forefront of defining and measuring Large Language Model (LLM) behavior. You will work closely with our Science team to define what 'good' looks like for various tasks, including Reasoning, Audio, Alignment, Tools, and Frontier bets.

Responsibilities

• Interact with models to identify areas for improvement in model behavior
• Gather internal and external feedback on model behavior to scope areas for improvement
• Design and implement evaluation pipelines, data guidelines, data generation, and synthetic testing environments
• Identify and fix edge case behaviors through rigorous testing
• Develop robust evaluation pipelines for model candidates
• Collaborate with AI Scientists

About you

• You have a deep understanding of linguistics, language, and translation, engineering and code behavior, or LLM agents at work, including reasoning and tool use
• You have prior knowledge in training and optimizing model behavior
• You are an expert at building robust evaluations
• You thrive in dynamic and technically complex environments
• You have a track record of delivering innovative, out-of-the-box solutions to address real-world constraints

XML job scraping automation by YubHub

About the Team

OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all of humanity. A key part of achieving that mission is training models that deeply understand and reflect human preferences — the Human Data team is at the heart of that effort.

The Human Data engineering team creates the systems that enable scalable, high-quality human feedback. These systems are essential to how OpenAI trains and improves its most advanced models. Engineers on this team collaborate closely with world-class researchers to bring alignment techniques to life — from experimental ideas to production-ready feedback loops.

About the Role

We’re looking for software engineers to join the Human Data team and build the platforms, prototypes, tools, and infrastructure that power how our AI models are trained, aligned, and evaluated. You’ll partner with researchers and cross-functional teams to bring alignment ideas to life, influence future model training, and shape how models interact with the real world.

We’re looking for people who are excited by technical ownership, enjoy working across the stack, and are eager to solve ambiguous problems in a high-impact, fast-paced environment.

This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.

In this role, you will:

• Build and maintain robust full-stack systems for feedback collection, data labeling, and evaluation pipelines, while maintaining high levels of security.

• Translate experimental alignment research into scalable production infrastructure, including inference and model training stacks.

• Design and iterate on user-facing tools and backend services to support high-quality data workflows

• Partner with researchers, engineers, and program leads to shape feedback loops and model interaction paradigms

• Drive infrastructure improvements that enable faster iteration and scaling across OpenAI’s frontier models, from internal research tooling all the way to production ChatGPT.

You might thrive in this role if you:

• Have strong software engineering fundamentals and experience building production systems at scale

• Enjoy full-stack development with end-to-end ownership — from backend pipelines to user interfaces

• Are motivated by high-impact collaboration with research teams and solving novel, ambiguous problems

• Are excited to shape how AI systems learn from human preferences and reflect a broad range of human values

• Care deeply about inclusive tooling and building systems that enhance model safety, reliability, and usefulness

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

XML job scraping automation by YubHub