{"version":"0.1","company":{"name":"YubHub","url":"https://yubhub.co","jobsUrl":"https://yubhub.co/jobs/skill/ml-and-ai-development-flows"},"x-facet":{"type":"skill","slug":"ml-and-ai-development-flows","display":"Ml And Ai Development Flows","count":1},"x-feed-size-limit":100,"x-feed-sort":"enriched_at desc","x-feed-notice":"This feed contains at most 100 jobs (the most recently enriched). For the full corpus, use the paginated /stats/by-facet endpoint or /search.","x-generator":"yubhub-xml-generator","x-rights":"Free to redistribute with attribution: \"Data by YubHub (https://yubhub.co)\"","x-schema":"Each entry in `jobs` follows https://schema.org/JobPosting. YubHub-native raw fields carry `x-` prefix.","jobs":[{"@context":"https://schema.org","@type":"JobPosting","identifier":{"@type":"PropertyValue","name":"YubHub","value":"job_aa4023a9-ca3"},"title":"Product Manager","description":"<p>We are looking for a Product Manager to join the ML/AI Platform team. The team helps various groups build, deliver, and run ML and AI-enabled experiences at scale. We want a Product Manager to oversee AI Observability and Evaluation. These platform tools assist teams in tracking LLM behaviour during production, quickly resolving issues, measuring performance, and consistently refining our AI products.</p>\n<p>This is an internal platform role with highly technical customers (ML engineers, backend engineers, data scientists). You will partner closely with engineering leaders to define the paved road for instrumenting, monitoring, and operating AI workloads end-to-end.</p>\n<p><strong>Responsibilities</strong></p>\n<ul>\n<li>Own the product roadmap for Spotify&#39;s AI observability capabilities across instrumentation, data contracts, and debugging workflows.</li>\n<li>Collaborate with stakeholders to develop the roadmap for Spotify&#39;s AI/Agentic evaluation capabilities via LLM-as-judges and other approaches.</li>\n<li>Build golden path instrumentation defaults (SDKs/libraries/templates) that make LLM workloads observable by default.</li>\n<li>Partner with engineering to deliver root-cause workflows across LLM chains/agents (what changed, where time/cost is spent, which step fails, which prompt/template/model version regressed).</li>\n<li>Drive adoption with internal teams through discovery, pilots, documentation, and ongoing enablement.</li>\n<li>Define and track success metrics (coverage, time-to-first-trace, regressions caught pre-prod, MTTR, cost anomalies detected early).</li>\n<li>Partner with Privacy/Security to establish safe logging and retention guidance that supports both compliance and debuggability.</li>\n</ul>\n<p><strong>Requirements</strong></p>\n<ul>\n<li>2+ years of experience as a Product Manager, Technical Product Manager, or equivalent product ownership role.</li>\n<li>Familiar with ML and AI development flows, and comfortable reasoning about LLM application patterns (prompting, RAG/agents, evaluation basics).</li>\n<li>Aware of LLM evaluation approaches (golden datasets, regression testing, human-in-the-loop review, LLM-as-a-judge).</li>\n<li>Comfortable partnering with engineers on technical tradeoffs and system design, even if you are not writing production code.</li>\n<li>Strong product fundamentals: discovery, prioritization, roadmap communication, and stakeholder alignment.</li>\n<li>Metrics-oriented and pragmatic; you can define outcomes and iterate based on evidence.</li>\n</ul>\n<p><strong>Benefits</strong></p>\n<ul>\n<li>Competitive salary range: 98,155 - 140,222 CAD plus equity.</li>\n<li>Flexible work arrangement with some in-person meetings.</li>\n</ul>\n<p style=\"margin-top:24px;font-size:13px;color:#666;\">XML job scraping automation by <a href=\"https://yubhub.co\">YubHub</a></p>","url":"https://yubhub.co/jobs/job_aa4023a9-ca3","directApply":true,"hiringOrganization":{"@type":"Organization","name":"Spotify","sameAs":"https://www.spotify.com/","logo":"https://logos.yubhub.co/spotify.com.png"},"x-apply-url":"https://jobs.lever.co/spotify/92d1ba03-770d-4831-9227-e0db487567b9","x-work-arrangement":"hybrid","x-experience-level":"mid","x-job-type":"full-time","x-salary-range":"98,155 - 140,222 CAD plus equity","x-skills-required":["Product Management","ML and AI development flows","LLM application patterns","LLM evaluation approaches","Technical tradeoffs and system design","Product fundamentals","Metrics-oriented and pragmatic"],"x-skills-preferred":[],"datePosted":"2026-03-31T18:15:28.916Z","jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Toronto"}},"employmentType":"FULL_TIME","occupationalCategory":"Engineering","industry":"Technology","skills":"Product Management, ML and AI development flows, LLM application patterns, LLM evaluation approaches, Technical tradeoffs and system design, Product fundamentals, Metrics-oriented and pragmatic","baseSalary":{"@type":"MonetaryAmount","currency":"CAD","value":{"@type":"QuantitativeValue","minValue":98155,"maxValue":140222,"unitText":"YEAR"}}}]}