{"version":"0.1","company":{"name":"YubHub","url":"https://yubhub.co","jobsUrl":"https://yubhub.co/jobs/skill/content-moderation-systems"},"x-facet":{"type":"skill","slug":"content-moderation-systems","display":"Content Moderation Systems","count":1},"x-feed-size-limit":100,"x-feed-sort":"enriched_at desc","x-feed-notice":"This feed contains at most 100 jobs (the most recently enriched). For the full corpus, use the paginated /stats/by-facet endpoint or /search.","x-generator":"yubhub-xml-generator","x-rights":"Free to redistribute with attribution: \"Data by YubHub (https://yubhub.co)\"","x-schema":"Each entry in `jobs` follows https://schema.org/JobPosting. YubHub-native raw fields carry `x-` prefix.","jobs":[{"@context":"https://schema.org","@type":"JobPosting","identifier":{"@type":"PropertyValue","name":"YubHub","value":"job_f2e93c37-5a0"},"title":"Staff Software Engineer, Anti-Abuse & Security","description":"<p>The Anti-Abuse team is the front line defending Replit&#39;s platform from exploitation. We detect and shut down phishing deployments, prevent cryptomining on free-tier infrastructure, stop LLM token farming, and keep bad actors from weaponizing the platform against our users.</p>\n<p>This is adversarial work: attackers adapt constantly, and we build the detection systems, heuristics, and automated responses that stay ahead of them.</p>\n<p>What makes this role unique is the AI-native nature of Replit&#39;s platform. You&#39;ll work on problems that barely exist elsewhere: building guardrails for AI-generated code, detecting prompt injection attacks at scale, and using LLMs as a defensive tool against abuse.</p>\n<p>If you want hands-on experience applying AI to security problems, this is one of the few places you can do it in production with real attackers. You&#39;ll own problems end-to-end, from identifying emerging abuse patterns to shipping the systems that stop them at scale.</p>\n<p><strong>Responsibilities</strong></p>\n<ul>\n<li>Design and implement LLM guardrails that detect abuse scenarios in AI-generated code and agent interactions</li>\n<li>Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response decisions</li>\n<li>Build and operate abuse detection systems that identify phishing, cryptomining, account takeover, and financial fraud across millions of daily user actions</li>\n<li>Design automated response mechanisms that enforce platform policies without manual intervention</li>\n<li>Own the full abuse response lifecycle: detection, investigation, enforcement, and handling appeals alongside Support and Legal</li>\n<li>Analyze attack patterns using BigQuery and Hex, turning investigation findings into new detection rules</li>\n<li>Maintain and extend internal detection tools (Slurper, Netwatch) that continuously monitor user activity</li>\n<li>Integrate and tune security scanners (SAST, SCA) in CI pipelines with tight performance SLAs</li>\n<li>Track abuse trends, measure detection effectiveness, and adapt defenses as attack patterns evolve</li>\n</ul>\n<p><strong>Requirements</strong></p>\n<ul>\n<li>8+ years of experience in security engineering, anti-abuse, trust &amp; safety, or fraud detection</li>\n<li>Strong programming skills in Python and/or TypeScript for building detection systems and automation</li>\n<li>Experience with SQL and data analysis at scale (BigQuery, Snowflake, or similar)</li>\n<li>Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection</li>\n<li>Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors</li>\n<li>Ability to investigate complex abuse patterns and translate findings into automated defenses</li>\n<li>Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse</li>\n<li>Clear communication skills for working across Security, Support, Legal, and Engineering teams</li>\n</ul>\n<p><strong>Nice to Have</strong></p>\n<ul>\n<li>Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, cloud platforms, developer tools)</li>\n<li>Background in fraud detection, payment abuse, or financial crime</li>\n<li>Familiarity with device fingerprinting, IP reputation, and email validation services</li>\n<li>Experience with CI/CD security tooling (SAST, SCA, Dependabot, Snyk)</li>\n<li>Knowledge of container security, Linux internals, or cloud infrastructure (GCP preferred)</li>\n<li>Prior work with abuse reporting pipelines, trust &amp; safety tooling, or content moderation systems</li>\n</ul>\n<p><strong>Tools + Tech Stack for this role</strong></p>\n<ul>\n<li>Languages: Python, TypeScript, Go, SQL</li>\n<li>Data: BigQuery, Hex</li>\n<li>Detection tools: Slurper, Netwatch, Stytch (device fingerprint); ClearOut (email reputation)</li>\n<li>CI/CD Security: Dependabot, Snyk, SAST/SCA scanners</li>\n<li>Infrastructure: GCP, Kubernetes</li>\n<li>Collaboration: Linear, Slack, Zendesk (for abuse reports)</li>\n</ul>\n<p style=\"margin-top:24px;font-size:13px;color:#666;\">XML job scraping automation by <a href=\"https://yubhub.co\">YubHub</a></p>","url":"https://yubhub.co/jobs/job_f2e93c37-5a0","directApply":true,"hiringOrganization":{"@type":"Organization","name":"Replit","sameAs":"https://replit.com/","logo":"https://logos.yubhub.co/replit.com.png"},"x-apply-url":"https://jobs.ashbyhq.com/replit/75e69146-a092-43a1-b1d6-023d433d3ae7","x-work-arrangement":"hybrid","x-experience-level":"staff","x-job-type":"Full time","x-salary-range":"$190K - $240K","x-skills-required":["security engineering","anti-abuse","trust & safety","fraud detection","Python","TypeScript","SQL","BigQuery","Hex","ML/LLM-based classifiers","prompt injection","jailbreaking","common attack patterns","phishing infrastructure","account takeover","credential stuffing","resource abuse"],"x-skills-preferred":["payment abuse","financial crime","device fingerprinting","IP reputation","email validation services","CI/CD security tooling","container security","Linux internals","cloud infrastructure","abuse reporting pipelines","trust & safety tooling","content moderation systems"],"datePosted":"2026-04-24T13:13:20.268Z","jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Foster City, CA (Hybrid) In office M,W,F"}},"employmentType":"FULL_TIME","occupationalCategory":"Engineering","industry":"Technology","skills":"security engineering, anti-abuse, trust & safety, fraud detection, Python, TypeScript, SQL, BigQuery, Hex, ML/LLM-based classifiers, prompt injection, jailbreaking, common attack patterns, phishing infrastructure, account takeover, credential stuffing, resource abuse, payment abuse, financial crime, device fingerprinting, IP reputation, email validation services, CI/CD security tooling, container security, Linux internals, cloud infrastructure, abuse reporting pipelines, trust & safety tooling, content moderation systems","baseSalary":{"@type":"MonetaryAmount","currency":"USD","value":{"@type":"QuantitativeValue","minValue":190000,"maxValue":240000,"unitText":"YEAR"}}}]}