We're seeking a Tech Lead Manager to lead a talented team of research scientists and research engineers focused on developing and implementing novel evaluation methodologies, metrics, and benchmarks to assess the capabilities and limitations of our cutting-edge LLMs.

Key responsibilities:

• Lead a team of highly effective research scientists and research engineers on LLM evals.
• Conduct research on the effectiveness and limitations of existing LLM evaluation techniques.
• Design and develop novel evaluation benchmarks for large language models, covering areas such as instruction following, factuality, robustness, and fairness.
• Communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects.
• Collaborate with internal teams and external partners to refine metrics and create standardized evaluation protocols.
• Implement scalable and reproducible evaluation pipelines using modern ML frameworks.
• Publish research findings in top-tier AI conferences and contribute to open-source benchmarking initiatives.

Ideal candidate has 5+ years of hands-on experience in large language model, NLP, and Transformer modeling, in the setting of both research and engineering development. Experience supporting and leading a team of research scientists and research engineers is also required.

XML job scraping automation by YubHub