This role requires deep fluency in how large language models work, hands-on experience with AI agent architectures, and the ability to partner closely with Engineering on production systems.

As an AI Agent Architect, you will:

Own Agent retrieval accuracy and relevance. Architect the knowledge systems that enable AI agents to surface the right answer on the first try. Measure and improve retrieval precision, contextual relevance, and hallucination rates.

Drive automated resolution rates. Build the decision frameworks that allow agents to take confident actions. What APIs do agents need to access? When can they make account modifications? You're accountable for encoding business logic into auditable, predictable systems that resolve issues without human intervention.

Manage AI safety and trust. Establish the guardrails that keep resolution rates high while failure rates stay low. You're responsible for what the agent doesn't do wrong: edge cases caught, prompt injection blocked, unintended behaviours prevented.

Own the feedback loop. Monitor the observability layer that turns agent behaviour into actionable insights. Instrument retrieval accuracy, action success rates, and failure patterns. Use this data to drive measurable week-over-week improvements in agent performance.

Continuously improve agent quality. Develop and maintain the prompt architecture that governs how agents reason and respond. Build systematic approaches to versioning, A/B testing, and performance evaluation, measuring consistency, accuracy, and adaptability across scenarios.

Drive integration strategy. Architect how agents connect to external systems,billing platforms, CRMs, internal tools, Airtable APIs. Define authentication patterns, error handling, and data transformation. Uptime, error rates, and data accuracy are your metrics.

You are:

Familiar with concepts like RAG architectures, prompt engineering patterns, chain-of-thought reasoning, and agent frameworks. You've built or significantly contributed to AI-powered systems in production.

Able to think in terms of data flows, state management, error handling, and edge cases. You can design complex systems that are both powerful and reliable. You've likely worked in roles like solutions architecture, platform engineering, or technical program management.

Able to write scripts, work with APIs, query databases, and prototype solutions. You're not a full-time software engineer, but you're dangerous enough to build, test, and validate technical approaches independently.

Able to instrument systems, analyse logs, and use data to diagnose issues and validate improvements. You build dashboards, define metrics, and can tie technical changes to business outcomes like resolution rates and customer satisfaction.

Able to explain complex AI system behaviour to non-technical stakeholders, write clear technical documentation, and translate business requirements into system specifications. You're effective working across engineering, product, and operations teams.

XML job scraping automation by YubHub

This is a hands-on technical role that combines deep engineering expertise with customer-facing problem solving. You'll work directly with customer engineering teams to integrate AI into their critical workflows.

Key Responsibilities

Customer Integration & Deployment

• Partner directly with enterprise customers to understand their technical infrastructure, data pipelines, and business requirements
• Design and implement custom integrations between Scale AI's platform and customer data environments (cloud platforms, data warehouses, internal APIs)
• Build robust data connectors and ETL pipelines to ingest, process, and prepare customer data for AI workflows
• Deploy and configure AI models and agents within customer security and compliance boundaries

AI Agent Development

• Develop production-grade AI agents tailored to customer use cases across domains like customer support, data analysis, content generation, and workflow automation
• Architect multi-agent systems that orchestrate between different models, tools, and data sources
• Implement evaluation frameworks to measure agent performance and iterate toward business objectives
• Design human-in-the-loop workflows and feedback mechanisms for continuous agent improvement

Prompt Engineering & Optimization

• Create sophisticated prompt engineering strategies optimized for customer-specific domains and data
• Build and maintain prompt libraries, templates, and best practices for customer use cases
• Conduct systematic prompt experimentation and A/B testing to improve model outputs
• Implement RAG (Retrieval Augmented Generation) systems and fine-tuning pipelines where appropriate

Technical Leadership & Collaboration

• Serve as the primary technical point of contact for strategic enterprise accounts
• Collaborate with customer data scientists, ML engineers, and software developers to ensure smooth integration
• Provide technical training and knowledge transfer to customer teams
• Work closely with Scale's product and engineering teams to translate customer needs into product improvements
• Document technical architectures, integration patterns, and best practices

Problem Solving & Innovation

• Debug complex technical issues across the entire stack, from data pipelines to model outputs
• Rapidly prototype solutions to unblock customers and prove out new use cases
• Stay current on the latest AI/ML research and tools, bringing innovative approaches to customer problems
• Identify opportunities for productization based on common customer patterns

Required Qualifications

• 4+ years of software engineering experience with strong fundamentals in data structures, algorithms, and system design
• Production Python expertise with experience in modern ML/AI frameworks (e.g., LangChain, LlamaIndex, HuggingFace, OpenAI API)
• Experience with cloud platforms (AWS, GCP, or Azure) and modern data infrastructure
• Strong problem-solving skills with the ability to navigate ambiguous requirements and rapidly iterate toward solutions
• Excellent communication skills with the ability to explain complex technical concepts to both technical and non-technical audiences

Preferred Qualifications

• Agent Development Wiz
• Deep understanding of LLMs including prompting techniques, embeddings, and RAG architectures
• Experience building and deploying AI agents or autonomous systems in production
• Knowledge of vector databases and semantic search systems
• Contributions to open-source AI/ML projects

• Infrastructure Guru
• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines
• Experience using Terraform, Bicep, or other Infrastructure as Code (IaC) tools
• Previous work in a devops, platform, or infra role

• Customer Product Whisperer
• Proven ability to work with customers in a technical consulting, solutions engineering, or product engineering role
• Domain expertise in verticals like finance, healthcare, government, or manufacturing
• Experience with technical enablement or teaching programs

Sample Projects

The following are some examples of the types of projects we’ve worked on with customers. All of these projects leverage customer data, integrate directly into customers’ existing systems, and are deployed on their infrastructure.

• Deep Research for Due Diligence
• Churn Prediction
• Data Extraction Voice Agent

Compensation

Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position, determined by work location and additional factors, including job-related skills, experience, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be eligible for equity grant. You’ll also receive benefits including, but not limited to: Comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. Additionally, this role may be eligible for additional benefits such as a commuter stipend.

Pay Transparency

For pay transparency purposes, the base salary range for this full-time position in the locations of San Francisco, New York, Seattle is: $216,000-$270,000 USD

XML job scraping automation by YubHub

You'll work with enterprise clients to understand their unique challenges, lead a team that architects specific AI solutions, and ensure successful deployment and adoption of AI systems in production environments.

This is a Management role that combines deep engineering and AI expertise, leading a team, and working on customer-facing problems. You'll work directly with customer engineering teams to integrate AI into their critical workflows.

Customer Integration & Deployment

Partner directly with enterprise customers to understand their technical infrastructure, data pipelines, and business requirements.

Design and implement custom integrations between Scale AI's platform and customer data environments (cloud platforms, data warehouses, internal APIs).

Build robust data connectors and ETL pipelines to ingest, process, and prepare customer data for AI workflows.

Deploy and configure AI models and agents within customer security and compliance boundaries.

AI Agent Development

Develop production-grade AI agents tailored to customer use cases across domains like customer support, data analysis, content generation, and workflow automation.

Architect multi-agent systems that orchestrate between different models, tools, and data sources.

Implement evaluation frameworks to measure agent performance and iterate toward business objectives.

Design human-in-the-loop workflows and feedback mechanisms for continuous agent improvement.

Prompt Engineering & Optimization

Create sophisticated prompt engineering strategies optimized for customer-specific domains and data.

Build and maintain prompt libraries, templates, and best practices for customer use cases.

Conduct systematic prompt experimentation and A/B testing to improve model outputs.

Implement RAG (Retrieval Augmented Generation) systems and fine-tuning pipelines where appropriate.

Leadership & Collaboration

Serve as the Engineering Manager and technical point of contact for strategic enterprise accounts.

Lead a team that is collaborating with customer data scientists, ML engineers, and software developers to ensure smooth integration.

Work closely with Scale's product and engineering teams to translate customer needs into product improvements.

Document technical architectures, integration patterns, and best practices.

Problem Solving & Innovation

Debug complex technical issues across the entire stack, from data pipelines to model outputs.

Rapidly prototype solutions to unblock customers and prove out new use cases.

Stay current on the latest AI/ML research and tools, bringing innovative approaches to customer problems.

Identify opportunities for productization based on common customer patterns.

XML job scraping automation by YubHub

LLM & AI Pipeline Engineering - Design, build, and maintain production-grade LLM integration pipelines , including retrieval-augmented generation (RAG), prompt engineering, output parsing, and chain orchestration.

Develop and operate AI features within Jeeves's core financial products: spend categorization, document extraction, anomaly detection, financial Q&A, and automated reconciliation.

Implement structured output validation, fallback handling, and confidence scoring to ensure AI decisions meet reliability standards for financial use cases.

Evaluate and integrate AI frameworks and tools (LangChain, LlamaIndex, OpenAI API, Anthropic API, HuggingFace, vector databases) and advocate for the right tool for the job.

Establish prompt versioning and evaluation practices to ensure AI outputs remain accurate and consistent as models and data evolve.

Retrieval & Vector Search - Design and maintain vector search pipelines using databases such as Pinecone, Weaviate, or pgvector to power semantic search and RAG-based features.

Build document ingestion and chunking pipelines for Jeeves's financial data , processing invoices, receipts, policy documents, and transaction records.

Optimize retrieval quality through embedding model selection, chunk strategy, metadata filtering, and re-ranking techniques.

ML Model Serving & Operations - Collaborate with data scientists to take trained ML models from experimental notebooks to production serving infrastructure.

Build and maintain model serving endpoints with appropriate latency SLOs, input validation, and output monitoring.

Implement model performance monitoring and data drift detection to ensure production models remain accurate over time.

Support model retraining workflows by designing clean data pipelines and feature engineering that can be continuously updated.

Backend Integration & Reliability - Integrate AI services cleanly with Jeeves's backend microservices , designing clear API contracts, circuit breakers, and graceful degradation patterns.

Write high-quality, testable backend code in Python or Go/Node.js to power AI-integrated features.

Instrument AI components with structured logging, distributed tracing, latency dashboards, and alerting to ensure operational visibility.

Build human-in-the-loop review workflows for AI decisions that require oversight , particularly for high-value financial actions.

Collaboration & Growth - Partner with Product, Backend Engineering, and Data Science to define the AI roadmap and translate requirements into reliable systems.

Contribute to a culture of quality by writing design docs, reviewing peers' AI system designs, and sharing learnings openly.

Help grow the AI engineering practice at Jeeves by establishing patterns, tooling, and best practices that the broader team can build on.

XML job scraping automation by YubHub

You bring a strong foundation in AI/ML, hands-on experience with prompt engineering, and a proven track record of integrating advanced tools across diverse platforms.

You thrive in collaborative environments, adept at translating business needs into scalable, secure, and reliable technical solutions. Compliance and vulnerability management are second nature to you, and you proactively embed best practices into your development workflow.

You are excited by the prospect of architecting end-to-end AI automation within the Fluids Business Unit (FBU), enhancing global user experiences and streamlining workflows for maximum impact.

Your background includes MCP architectures, enterprise automation, and secure DevOps practices. You excel at integrating AI-powered tools into large-scale platforms, such as Copilot, and are driven by a relentless desire to innovate.

As a leader and collaborator, you inspire those around you to strive for continuous improvement and technical excellence. You are committed to pushing the boundaries of what’s possible, ensuring the solutions you build are not only powerful but also future-ready.

Designing and developing MCP-based AI tools for seamless cross-platform usage within the Fluids Business Unit. Implementing intelligent code generation, setup validation, automated report creation, and user-defined function (UDF) generation features. Enabling support for multiple LLM providers and models, ensuring portability and extensibility of AI solutions. Integrating FBU’s MCP server with the organization’s centralized MCP server for unified workflow automation. Collaborating with the framework team to embed AI tools into the Copilot platform, enhancing enterprise productivity. Ensuring all tools comply with enterprise security standards, certifications, and vulnerability management practices. Conducting rigorous security assessments and proactively mitigating vulnerabilities in AI pipelines and integrations. Architecting scalable AI automation solutions for end-to-end workflows within Fluids One, optimizing for performance and reliability. Optimizing AI workflows for maintainability, scalability, and seamless user experience.

Drive the AI transformation initiative within the Fluids Business Unit, advancing automation and intelligent system integration. Empower end-to-end automation in Fluids One, reducing manual intervention and accelerating productivity. Enhance the organization’s Copilot platform with advanced AI-powered tools that simplify complex workflows. Improve enterprise security posture by embedding robust compliance and vulnerability management into AI solutions. Foster innovation in AI toolchain development, supporting multi-provider and multi-model orchestration for maximum flexibility. Contribute to Synopsys’ leadership in AI-driven automation, setting industry standards for scalable and secure enterprise solutions.

Bachelor’s or Master’s degree in Computer Science, AI, Software Engineering, or a related field. Strong experience in AI/ML system design and integration, particularly in enterprise environments. Hands-on expertise with RAG architectures, large language models (LLMs), and prompt engineering. Proficiency in multi-model/multi-provider AI integration, developing cross-platform enterprise tools and APIs. Deep understanding of enterprise security practices, vulnerability scanning, and compliance requirements. Experience integrating AI solutions into enterprise platforms (e.g., Copilot-style systems). Strong programming skills in Python and GO Lang. Experience with microservices architecture and server integration.

Innovative and solution-oriented, always seeking to push the boundaries of what AI can achieve. Collaborative, with excellent communication skills and the ability to work across diverse teams. Detail-oriented and meticulous, ensuring high-quality and secure deliverables. Resilient and adaptable, capable of thriving in fast-paced, evolving environments. Proactive in identifying risks and implementing mitigation strategies. Committed to continuous learning and staying updated with industry advancements.

Engineering

XML job scraping automation by YubHub

Responsibilities:

• Design, develop, and deploy end-to-end GenAI solutions (RAG, AI agents, agentic workflows, prompt engineering).
• Integrate AI solutions into existing systems, applications, and business processes.
• Implement LLMOps practices, including Docker containerization, CI/CD pipelines, and versioning strategies.
• Ensure monitoring, observability, cost optimization, and rollback mechanisms for production AI services.
• Define and execute evaluation frameworks (hallucination metrics, A/B testing, offline/online validation).
• Apply security, compliance, and governance guidelines for GenAI implementations.
• Collaborate with stakeholders and contribute to AI delivery standards and onboarding practices.

Requirements:

• Master’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
• Very strong expertise in Python and software engineering (APIs, testing, code reviews).
• Practical experience with RAG architectures, vector databases, and agentic AI workflows.
• Hands-on experience deploying production-grade AI services.
• Solid knowledge of Docker and CI/CD pipelines.
• Understanding of ML fundamentals, evaluation concepts, and LLM behavior.
• Familiarity with cloud environments (preferably Azure) and distributed systems.
• Strong analytical and problem-solving skills.
• Very good level of English.
• Autonomous, reliable, and team-oriented mindset.

What you will get:

• A role with true technical ownership: architecture, scaling, and governance decisions that directly impact production AI solutions.
• Complex projects that go beyond “just pipelines” – covering big data processing and large-scale ML/DL deployment.
• Opportunities to deepen your expertise in Databricks, cloud-native ML, and MLOps.
• A team where your input and technical decisions truly matter.
• A competitive package and benefits.

XML job scraping automation by YubHub

What you'll do

• Design and implement the MCP integration layer, including MCP registry services and MCP endpoints, enabling AI systems and agents to securely discover and interact with enterprise infrastructure tools and platforms.
• Build scalable backend services leveraging expertise in Python that power automation systems, enterprise infrastructure integrations, and AI-driven operational workflows.
• Develop and maintain production-grade DevOps pipelines, including CI/CD workflows, containerized deployments, monitoring, and reliability automation.

What you need

• 10+ years of experience in software engineering, backend development, infrastructure engineering, or platform systems.
• Strong expertise in Python and building scalable backend services, APIs, or platform integration layers.
• Experience designing MCP registry services, MCP endpoints, or similar service discovery and integration layers connecting enterprise systems.
• Strong DevOps experience, including CI/CD pipelines, containerization (Docker/Kubernetes), infrastructure automation, monitoring, and reliability practices.
• Hands-on familiarity with modern AI systems including LLM-based services, RAG architectures, or agent frameworks.

XML job scraping automation by YubHub