Your responsibilities will include:

• Prototyping apt technical solutions and finding best fits for the context engine.
• Architecting and building the core Context Platform.
• Designing schemas and primitives for Decision Memory and enterprise context.
• Owning context storage systems (graph, vector, event/time-based).
• Building read/write/query APIs used by agents, products, and external apps.
• Designing permission-aware, auditable context access.

You will be working closely with agentic systems engineers and product leadership to ensure the context engine is interoperable, portable, and zero-lock-in by design.

In this role, you will own:

• Context schemas and schema evolution strategies.
• Storage and data modeling choices.
• Platform APIs and interfaces.
• Security, identity propagation, and audit foundations.
• Long-term scalability and correctness of context data.

You will not own:

• Agent behavior or orchestration logic.
• Business rules or governance policy decisions.
• Product UI or workflow automation.

The ideal candidate will have significant experience building distributed systems, data platforms, or infrastructure, and will be comfortable operating in ambiguous, greenfield problem spaces. They will also have deep expertise in data modeling and schema design, experience designing shared platforms used by many teams, and strong instincts around APIs, contracts, and backward compatibility.

Nice to have experience with knowledge graphs, metadata systems, or search/retrieval systems, experience building systems with governance, auditability, or compliance requirements, and familiarity with dbt or modern analytics stacks or developer tooling.

XML job scraping automation by YubHub

Your work will directly shape what users experience: designing APIs that serve rich content, building services that handle real-time interactions, implementing content-matching systems for rights and safety, and ensuring our platform performs under load. You'll architect systems that are fast, correct, and maintainable.

You'll collaborate closely with Product, ML Research, and Mobile/Web teams to ship features that matter. We use Python, Go, BigQuery, Pub/Sub, and a microservices architecture,but we care more about good judgment than specific tool experience.

Responsibilities

• Design and maintain application-level data models that organize rich content into canonical structures optimized for product features, search, and retrieval.
• Build high-reliability ETLs and streaming pipelines to process usage events, analytics data, behavioral signals, and application logs.
• Develop data services that expose unified content to the application, such as metadata access APIs, indexing workflows, and retrieval-ready representations.
• Implement and refine fingerprinting pipelines used for deduplication, rights attribution, safety checks, and provenance validation.
• Own data consistency between ingestion systems, application surfaces, metadata storage, and downstream reporting environments.
• Define and track key operational metrics, including latency, completeness, accuracy, and event health.
• Collaborate with Product teams to ensure content structures and APIs support evolving features and high-quality user experiences.
• Partner with Analytics and Research teams to deliver clean usage datasets for experimentation, model evaluation, reporting, and internal insights.
• Operate large analytical workloads in BigQuery and build reusable Dataflow/Beam components for structured processing.
• Improve reliability and scale by designing robust schema evolution strategies, idempotent pipelines, and well-instrumented operational flows.

Requirements

• Experience building production backend services and APIs at scale
• Experience building ETL/ELT pipelines, event processing systems, and structured data models for applications or analytics
• Strong background in data modeling, metadata systems, indexing, or building canonical representations for heterogeneous content
• Proficiency in Python, Go, SQL, and scalable data-processing frameworks (Dataflow/Beam, Spark, or similar)
• Familiarity with BigQuery or other analytical data warehouses and strong comfort optimizing large queries and schemas
• Experience with event-driven architectures, Pub/Sub, or Kafka-like systems
• Strong understanding of data quality, schema evolution, lineage, and operational reliability
• Ability to design pipelines that balance cost, latency, correctness, and scale
• Clear communication skills and an ability to collaborate closely with Product, Research, and Analytics stakeholders

Nice to Have

• Experience building application-facing APIs or microservices that expose structured content
• Background in information retrieval, indexing systems, or search infrastructure
• Experience with fingerprinting, perceptual hashing, audio similarity metrics, or content-matching algorithms
• Familiarity with ML workflows and how downstream analytics and usage data feed back into research pipelines
• Understanding of batch + streaming architectures and how to blend them effectively
• Experience with Go, Next.js, or React Native for occasional full-stack contributions

Why Join Us

You will design the core data services and pipelines that power our product experience, analytics, and business operations. You’ll work on high-impact data challenges involving real-time signals, large-scale metadata systems, and cross-platform consistency. You’ll join a small, fast-moving team where you’ll shape the structure, reliability, and intelligence of our downstream data ecosystem.

Benefits

• Highly competitive salary and equity
• Quarterly productivity budget
• Flexible time off
• Fantastic office location in Manhattan
• Productivity package, including ChatGPT Plus, Claude Code, and Copilot
• Top-notch private health, dental, and vision insurance for you and your dependents
• 401(k) plan options with employer matching
• Concierge medical/primary care through One Medical and Rightway
• Mental health support from Spring Health
• Personalized life insurance, travel assistance, and many other perks

XML job scraping automation by YubHub