AI Agent 开发 × AI 评测
JD 精选 27 篇（中文翻译 + 分析）

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY- Assurance – Senior – Digital Role : GenAI / Agentic AI Evaluation Engineer (Quality, Safety & Reliability) Position Details As part of EY GDS Assurance Digital, you will help design, build, and scale a standardized evaluation capability, focused on evaluating GenAI, RAG-based, and Agentic AI solutions before deployment. This role sits at the intersection of AI evaluation engineering, Responsible AI, and GenAI security/red teaming. The primary objective is to ensure GenAI/agentic systems are safe, reliable, robust, and fit-for-purpose, by designing evaluation strategies, building repeatable test harnesses, and generating auditable evidence that supports go/no-go decisions. You will work with global stakeholders (product teams, solution architects, risk & compliance, and assurance leadership) to define evaluation requirements, request test datasets from product teams, execute rigorous evaluations (functional + non-functional), and recommend mitigations and controls to reduce risk. This is a core full-time role that requires a hands-on AI Development mindset, strong evaluation mindset, and the ability to translate risk concerns into practical testing strategies and measurable acceptance criteria. Responsibilities Define and operationalize evaluation strategies for GenAI systems across use cases like Q&A assistants, summarization, extraction, drafting, agentic systems, and multi-step workflows. Translate business use-cases into a structured evaluation plan: scope, assumptions, success criteria, datasets, metrics, red-team scenarios, thresholds, and reporting requirements. Drive standardization: reusable evaluation templates, test case libraries, scoring rubrics, and reporting formats across product teams. Design structured dataset requirements for product teams and ensure coverage across: Core user journeys and primary business intents Edge cases (rare prompts, ambiguous queries, incomplete context) Adversarial cases (malicious prompts, jailbreak attempts, prompt injections) Bias & fairness cases (sensitive demographic proxies, protected attributes, stereotyping patterns) Define guidance for dataset sufficiency and statistical coverage (e.g., minimum samples, distribution balance, scenario matrices, stratification by intent/risk). Build reusable evaluation pipelines for: Answer quality (correctness, relevance, completeness, clarity) Grounding & faithfulness (RAG-specific: faithfulness, context precision/recall, hallucination rate, citation quality) Agentic behavior (tool-call accuracy, tool misuse, goal completion, step correctness, unnecessary actions, loop detection, safety of tool outputs) Operational quality (latency, cost/token budget, throughput, stability, retries, failure recovery) Combine LLM-as-judge and human evaluation in a calibrated way (rubric design, sampling plans, agreement checks). Implement automated evaluation harnesses in Python (preferred), enabling: batch runs on scenario suites configurable metric definitions reproducible runs with run IDs and artifacts storage of traces and outputs for auditability Execute structured red teaming aligned to OWASP Top 10 for LLM Applications, covering (examples): Prompt injection (direct + indirect) and tool hijacking Sensitive data disclosure / PII leakage Insecure output handling (downstream injection) Training data leakage / memorization probes Model denial-of-service / denial-of-wallet patterns Integrate evals into development lifecycle: pre-release regression gates, CI checks, benchmark comparisons across model versions/prompts/tools/retrievers. Perform adversarial testing for agentic workflows: tool misuse / over-permissioned tool access unauthorized action execution exfiltration via tools/connectors prompt injection via retrieved documents (RAG poisoning) Recommend mitigations: input validation, retrieval filtering, tool sandboxing, least-privilege permissions, guardrails, policy prompting, refusal logic, output encoding, monitoring alerts. Produce high-quality evaluation reports that are auditable and decision-ready, including: methodology, datasets, metrics, thresholds quantitative results qualitative results risk assessment summary and recommended control actions Present findings to stakeholders in a crisp, risk-informed manner; clearly explain residual risk, limitations, and rationale for go/no-go. Key Requirements/Skills & Qualification: Excellent academic background, including at a minimum a bachelor’s or a master’s degree in data science, Statistics, Engineering, Operational Research, or other related field with strong focus on modern data architectures, processes, and environments. 4–7+ years of relevant experience in one or more areas: ML/AI/GenAI/Agentic engineering (NLP/LLMs), evaluation engineering, applied research security testing / red teaming building and designing evaluation harness that ensures safety, reliability and robustness. Strong hands-on Python for building evaluation harnesses (data processing, metric computation, orchestration, reporting pipelines). Practical understanding of GenAI system architectures: RAG, embeddings/vector search, prompt orchestration, tool calling, multi-agent systems, memory, routing. Experience designing metrics and evaluation methods (rubrics, automated scoring, sampling strategy, regression design). Familiarity with LLM risks and mitigations, especially for enterprise contexts (data leakage, hallucinations, prompt injection, unsafe content, bias). Security / Red Teaming Skills (Strong Preference) Understanding of OWASP Top 10 for LLM Applications and how to translate it into test cases and controls. Experience with adversarial testing approaches: jailbreak prompts, injection patterns, tool misuse scenarios, retrieval poisoning patterns. Familiarity with secure-by-design practices for LLM apps: least privilege, safe tool invocation, output encoding/validation, monitoring. Evaluation frameworks and tooling: RAGAS, DeepEval, LangSmith, Phoenix/Arize, custom eval harnesses. Experimentation practices: A/B testing mindset, baseline comparisons, statistical rigor for sample sizes. Observability/tracing: structured logging, OpenTelemetry, Langfuse-style traces, dashboards. Basic DevOps practices: Git, CI/CD, containerization (Docker), reproducible environments. Strong written communication to produce clear evaluation plans and reports for technical + non-technical stakeholders. Ability to challenge assumptions constructively (“effective challenge”) and influence engineering teams toward remediation. Comfort operating in ambiguity with fast-evolving GenAI tooling and risk landscape. Preferred / Nice-to-Have Experience in Assurance/Finance/Regulatory environments (model validation, risk acceptance workflows, audit evidence mindset). Familiarity with responsible AI frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act concepts). Experience evaluating multilingual systems or domain-heavy enterprise assistants. Hands-on with Azure ecosystem (Azure OpenAI, AI Search, Function Apps, App Insights, Key Vault). Additional skills requirements: Excellent written, oral, presentation and facilitation skills Ability to coordinate multiple projects and initiatives simultaneously through effective prioritization, organization, flexibility, and self-discipline. Must have demonstrated project management experience. Knowledge of firm’s reporting tools and processes. Proactive, organized, and self-sufficient with ability to priorities and multitask. Analyses complex or unusual problems and can deliver insightful and pragmatic solutions. Ability to quickly and easily create/ gather/ analyze data from a variety of sources. A robust and resilient disposition able to encourage discipline in team behaviors What We Look For A Team of people with commercial acumen, technical experience, and enthusiasm to learn new things in this fast-moving environment An opportunity to be a part of market-leading, multi-disciplinary team of 7200 + professionals, in the only integrated global assurance business worldwide. Opportunities to work with EY GDS Assurance practices globally with leading businesses across a range of industries What Working At EY Offers At EY, we’re dedicated to helping our clients, from startups to Fortune 500 companies — and the work we do with them is as varied as they are. You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees, and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover, you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus, we offer: Support, coaching and feedback from some of the most engaging colleagues around Opportunities to develop new skills and progress your career The freedom and flexibility to handle your role in a way that’s right for you EY | Building a better working world EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets. Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate. Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.

Senior LLM Evaluation Engineer (AI Quality Engineer) About the Role We are seeking an experienced Senior LLM Evaluation Engineer to ensure the quality, accuracy, and reliability of Generative AI solutions used across our digital platforms. In this role, you will define and execute evaluation strategies for Large Language Models (LLMs), ensuring AI-generated content is factually correct, consistent, compliant, and production-ready. As the quality owner for AI-powered applications, you will design evaluation frameworks, create benchmark datasets, assess model performance, identify hallucinations and reasoning failures, and provide actionable feedback to improve prompts, retrieval pipelines, and model behavior. You will collaborate closely with AI Engineers, Product Managers, Medical, Legal & Regulatory (MLR), and Business stakeholders to ensure AI systems meet the highest quality standards before release. Key Responsibilities Design and execute comprehensive evaluation strategies for LLM-powered applications. Develop benchmark datasets, golden datasets, and evaluation scenarios based on business requirements and approved reference data. Evaluate AI-generated outputs for: Factual accuracy Consistency Completeness Relevance Groundedness Hallucinations Unsupported claims Toxicity and safety concerns Perform manual evaluations and human-in-the-loop assessments of AI responses. Design acceptance criteria and quality metrics for Generative AI applications. Execute regression evaluations following model, prompt, RAG, or configuration changes. Identify and document AI failures, edge cases, reasoning errors, prompt failures, and retrieval issues. Produce detailed evaluation reports, quality scorecards, and release recommendations. Collaborate with AI Engineers to improve prompts, RAG pipelines, embeddings, and model configurations based on evaluation findings. Validate AI responses against trusted knowledge sources and business rules. Support continuous improvement by building reusable evaluation datasets and test assets. Define AI quality standards, governance practices, and evaluation methodologies across AI initiatives. Required Qualifications 8+ years of experience in Software Quality, AI Quality Engineering, Machine Learning, Data Science, or related fields. Hands-on experience evaluating Large Language Models (LLMs) or Generative AI applications. Strong understanding of LLM behavior, prompt engineering, and Retrieval-Augmented Generation (RAG). Experience identifying hallucinations, reasoning failures, factual inaccuracies, and inconsistent AI outputs. Experience designing evaluation datasets, benchmark scenarios, and acceptance criteria. Strong analytical and critical thinking skills with exceptional attention to detail. Experience documenting evaluation results, quality metrics, and production readiness assessments. Excellent communication skills with the ability to collaborate across technical and business teams. Preferred Qualifications Experience with AI evaluation frameworks such as DeepEval, Ragas, LangSmith Evaluations, Promptfoo, OpenAI Evals, or similar tools . Experience evaluating RAG systems and retrieval quality. Understanding of LLM-as-a-Judge methodologies and human evaluation workflows. Experience testing AI applications in regulated industries such as healthcare, pharmaceutical, legal, or finance. Familiarity with Azure AI Foundry, Azure OpenAI, OpenAI APIs, Anthropic Claude, Gemini, or similar platforms. Experience with Playwright or API testing tools is a plus. Technical Skills Large Language Models (LLMs) Generative AI LLM Evaluation AI Quality Assurance Prompt Engineering Prompt Evaluation Retrieval-Augmented Generation (RAG) Hallucination Detection Groundedness Evaluation Benchmark Dataset Creation Human-in-the-Loop Evaluation Regression Evaluation AI Safety & Compliance API Testing Python (preferred) Azure AI Foundry (preferred) LangSmith, Ragas, DeepEval, Promptfoo, or equivalent evaluation frameworks Nice to Have Experience with healthcare or regulated AI solutions. Knowledge of AI governance, responsible AI, and model validation practices. Experience working with vector databases and knowledge retrieval systems. Familiarity with automated evaluation pipelines and AI observability platforms. Experience collaborating with cross-functional AI engineering and product teams. What You'll Bring You are passionate about ensuring AI systems are trustworthy, accurate, and safe. You understand that building an LLM is only part of the solution—evaluating, validating, and continuously improving its outputs is equally important. You enjoy combining analytical thinking with human judgment to ensure AI applications deliver reliable, production-ready experiences.

About Dialpad Dialpad is the AI platform for customer experience, built to resolve customer problems in real time across voice and digital. Our AI agents learn from your best human agents and improve with every interaction, helping organizations understand their customers, deliver better experiences, increase operational efficiencies, and build a lasting competitive advantage. Unlike legacy systems built to route and answer, or standalone agentic bot vendors built to deflect, Dialpad was built to resolve. Our AI agents and human agents operate on a single platform with shared context, allowing Agentic AI to resolve issues, advance deals, and eliminate busywork through automation while seamlessly handing conversations to humans when needed, with full context preserved. Market-leading brands, including Randstad, Motorola Solutions, Netflix, the San Diego Padres, the Colorado Rockies Baseball Club, and Cal Athletics, trust Dialpad. Dialpad is backed by Andreessen Horowitz, GV, ICONIQ Capital, and T-Mobile. Being a Dialer At Dialpad, AI isn’t just a feature; it’s how our teams do their best work every day. We put powerful AI tools in every employee’s hands so they can move faster, think bigger, and achieve more. We believe every conversation matters. And we’ve built the platform that turns those conversations into insight and action, for our customers and ourselves. We look for people who are intensely curious and hold themselves to a high bar. Our ambition is significant, and achieving it requires a team that operates at the highest level. We seek individuals who embody our core traits: Scrappy, Curious, Optimistic, Persistent, and Empathetic . Your role As an AI Evaluation Engineer, you'll be an integral part of our AI Evaluation team, owning evaluation coverage for Dialpad's Agentic AI systems alongside our existing evaluation lead. A key focus will be co-owning LLM-judge metric development and calibration, scenario and benchmark dataset curation, and structured error analysis to support release-readiness decisions for our agentic voice and chat solutions. This position reports to the manager of the AI Evaluation team and has the opportunity to be based in our Vancouver office. What You’ll Do You will design and execute validation strategies for agentic, NLP, and speech workflows across staging, beta, and release candidates. You will build, run, and improve regression evaluations, A/B comparisons, and red teaming analyses to determine whether product and model changes are ready to move forward. You will co-own LLM-judge metric development, calibration, and prompt refinement across evaluation dimensions. You will create, configure, and monitor data annotation jobs to keep evaluation and calibration datasets fed on schedule. You will develop and maintain QA tooling, notebooks, and pipeline components that make recurring evaluations scalable and reusable across teams. You will investigate bugs, triage issues, and decide whether problems should become engineering escalations, test set additions, or follow-up analysis. You will collaborate with cross-functional teams, including applied science, engineering, and Product QA. Skills You’ll Bring Bachelor's or Master's degree in Computer Science, Software Engineering, Computational Linguistics, or a related field. 3+ years of experience in QA, test engineering, model evaluation, or applied ML quality for AI-driven products. Experience designing structured test strategies across manual and automated workflows. Comfort working with complex AI systems such as speech, NLP, LLM, or agentic products. Experience working with evaluation datasets, gold sets, adversarial test sets, or benchmark creation for AI systems. Strong analytical skills for investigating failures, comparing outputs, and identifying actionable quality patterns. Experience collaborating with cross-functional technical teams and communicating clearly through documentation and reporting. For exceptional talent based in Ontario, Canada the target base salary range for this position is posted below. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the target range for new hire salaries for the position. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process. Please note that the compensation details listed in Ontario role postings reflect the base salary only, and do not include bonus, equity, or benefits. Ontario Salary Range $96,000—$116,250 CAD Why Join Dialpad Work at the center of the AI transformation in business communications Build and ship agentic AI products that are redefining how companies operate Join a team where AI amplifies every employee’s impact Competitive salary, comprehensive benefits, and real opportunities for growth We believe in investing in our people. Dialpad offers competitive benefits and perks, cutting-edge AI tools, and a robust training program that help you reach your full potential. We have designed our offices to be inclusive, offering a vibrant environment to cultivate collaboration and connection. Our exceptional culture, repeatedly recognized as a Great Place to Work, ensures that every employee feels valued and empowered to contribute to our collective success. Don’t meet every single requirement? If you’re excited about this role and possess the fundamental traits, drive, and strong ambition we seek, but your experience doesn’t meet every qualification, we encourage you to apply. Dialpad is an equal-opportunity employer. We are dedicated to creating a community of inclusion and an environment free from discrimination or harassment.

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why This Role? North is Cohere’s AI workspace platform for enterprises: a secure, customizable environment where companies can use AI across their real workflows while maintaining control over sensitive data. North connects AI agents with workplace tools, applications, and business context, helping users delegate complex work, build automations, inspect outputs, and collaborate with AI in production environments. As North becomes more capable, one of the most important questions is also one of the hardest: how do we know whether the model is actually getting better for the workflows customers care about? This role is about being the voice of North inside modelling. You will build the evaluation systems, feedback loops, and applied modelling workflows that make sure model progress translates into better product outcomes for North users. You will work closely with North product teams, customer-facing teams, and modelling teams to define what “good” means across the product surface, turn real usage and product direction into high-quality evals, and use those evals to guide model selection, patches, and regular model updates. This is neither a pure research role nor a conventional product engineering role. It is a rigorous applied MLE role for someone who cares deeply about measurement, model behavior, and real-world product quality. You should be excited by the craft of building careful evals: evals that capture messy agentic workflows, reflect actual customer needs, resist superficial benchmark hacking, and provide useful signal for where the product and models need to go next. As a Member of Technical Staff, North Modelling (Evals), You Will: Own the eval strategy for North: define what we need to measure across agent workflows, tool use, enterprise knowledge work such as deep research, document creation or editing, and other human-AI interactions. Build high-quality evals from the realities of the product: user feedback, production failures, privacy-preserving usage logs, internal dogfooding, customer needs, and forward-looking product goals. Create systems that continuously turn what North is learning from users, customers, and feature teams into evals, so measurement keeps pace with the product rather than becoming a static benchmark. Be the voice of North inside modelling: extract clear insight from eval results, customer-derived data and product context, then turn model failures, capability gaps, and North-specific needs into actionable recommendations for the central modelling teams. You May Be a Good Fit If You have improved LLM-powered, agent-powered, or AI-product systems through evals, feedback loops, data curation, prompting, model adaptation, or model selection. You care deeply about evaluation as a craft: representative tasks, precise rubrics, clean data, failure analysis, regression tracking, and knowing when a metric is giving false confidence. You have strong applied MLE judgment and can reason clearly about model behavior, eval validity, product outcomes, and production tradeoffs. You are comfortable working close to users and product teams, while translating messy qualitative signals into measurement that other modelling teams can act on. You are self-directed, practical, and motivated by open-ended problems where the right answer requires both technical depth and product understanding. You care about making AI systems genuinely useful in real enterprise settings, not just better on abstract benchmarks. If some of the above does not line up perfectly with your experience, we still encourage you to apply. Location This team works closely across Europe and East Coast North America time zones. We are open to candidates who can collaborate effectively within those hours. Full-Time Employees At Cohere Enjoy These Perks A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch. Full health and dental benefits, including a separate budget for mental health. RRSP matching, 401K, Pension Scheme. 100% Parental Leave top-up for up to 6 months, for either parent. Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit. Education & learning stipend for conferences, courses, and coaching. 6 weeks of paid vacation (30 working days!) Budget for traveling to other offices if you are remote, plus an annual company offsite. How And Where We Work Cohere is remote-friendly, but we also have offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul with more opening soon. For those in the office: a daily lunch program, plenty of snacks, and regular community and social events. For those not near an office: a co-working benefit so you can work alongside others in your city. Everyone receives a $500 home office stipend to set up your workspace properly. If any of the above doesn’t line up exactly with your experience, we still encourage you to apply. We strive to create an inclusive work environment for all; we welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs. We may use AI-enabled tools to screen and assess applicants against the criteria for this position. This helps our recruiters identify potentially qualified candidates, but it doesn't limit the applications our recruiters may review or consider. Beware of Scams: Cohere will never ask for payment or third-party services (e.g., CV writing) as part of our hiring process. All legitimate roles are listed on the Cohere careers page and LinkedIn only, with all communications from Cohere employees coming from an @cohere.com or @cw.cohere email alias. If jobs are viewed on other sites then please verify these through our official careers page.

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, You Will Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish. Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations. Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency. Build scalable and reusable tools for digging into model performance. You May Be a Good Fit If You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities. You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality. You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about. You have strong software engineering skills. Full-Time Employees At Cohere Enjoy These Perks A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch. Full health and dental benefits, including a separate budget for mental health. RRSP matching, 401K, Pension Scheme. 100% Parental Leave top-up for up to 6 months, for either parent. Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit. Education & learning stipend for conferences, courses, and coaching. 6 weeks of paid vacation (30 working days!) Budget for traveling to other offices if you are remote, plus an annual company offsite. How And Where We Work Cohere is remote-friendly, but we also have offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul with more opening soon. For those in the office: a daily lunch program, plenty of snacks, and regular community and social events. For those not near an office: a co-working benefit so you can work alongside others in your city. Everyone receives a $500 home office stipend to set up your workspace properly. If any of the above doesn’t line up exactly with your experience, we still encourage you to apply. We strive to create an inclusive work environment for all; we welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs. We may use AI-enabled tools to screen and assess applicants against the criteria for this position. This helps our recruiters identify potentially qualified candidates, but it doesn't limit the applications our recruiters may review or consider. Beware of Scams: Cohere will never ask for payment or third-party services (e.g., CV writing) as part of our hiring process. All legitimate roles are listed on the Cohere careers page and LinkedIn only, with all communications from Cohere employees coming from an @cohere.com or @cw.cohere email alias. If jobs are viewed on other sites then please verify these through our official careers page.

About The Role
An agent that demos well is not the same as an agent that works in production, so every agent we ship is backed by evals. We're hiring an AI Quality Engineer to own that system. You'll run and extend the evaluation pipelines that prove our agents work, both in client engagements and across our own internal agent fleet, and keep them honest as the agents, models, and data underneath them change.
What you'll do
Own the evaluation pipelines that gate every change we ship, keeping them automated and running in CI/CD rather than leaning on manual spot-checks.
Keep agents scored at the step level, not just the final answer, across tool selection, planning and reasoning chains, and retrieval quality, so subtle regressions surface before clients do.
Extend the eval toolkit as new failure modes appear: LLM-as-a-judge, RAG faithfulness and relevance, hallucination detection, prompt and agent regression suites, and red-teaming for safety.
Maintain the golden datasets that mirror real production inputs and the tracing and observability that watch our agents once they are live.
Own the quality bar for what ships and raise it over time with the rest of the engineering team.
What we're looking for
Strong software engineering with Python, and a feel for testing non-deterministic systems, where classic assertion-based tests are not enough.
Experience evaluating LLM and agent systems: agent graphs (e.g. LangGraph), eval frameworks (DeepEval, RAGAS, G-Eval, LangSmith, or similar), and LLM-as-a-judge methods.
Fluency with RAG and retrieval quality, plus observability and tracing for agents in production.
A bias for measurement: you define what "good" means, then build the harness that proves it.
Bonus: data-pipeline testing experience, or exposure to private markets, financial services, or enterprise data.
Why Dealstitch
We're a senior team with 40+ years of combined private-market technology experience. You'll work on problems that matter, with direct access to decision-makers, and see the impact of your work in weeks, not quarters.

The Opportunity AI is rapidly transforming the world. Whether it’s developing the next generation of human-level intelligence, enhancing voice assistants, or enabling researchers to analyze genetic markers at scale, AI is increasingly integrated into various aspects of our daily lives. Arize AI is the leading AI observability and evaluation platform, empowering AI engineers to build and deploy high-performing, reliable models. As the AI landscape shifts from traditional ML to generative AI and agentic systems, Arize ensures teams have the tools to monitor, troubleshoot, and improve AI in production. We’re looking for an Open Source AI Engineer to join our growing OSS team to drive the development of new frameworks, metrics, and tooling that help people build, test, and improve LLM tasks. You’ll play a lead role in shaping how developers measure and understand performance in advanced AI systems, all in the open. What You’ll Work On Build LLM Observability Frameworks: Design, architect, and open-source new libraries, pipelines, and APIs that make it simpler to monitor, evaluate, and improve LLM output. Collaborate with the Community: Partner closely with the broader AI open source ecosystem, gather feedback, review pull requests, and steer the direction of the project to address real developer needs. Prototype and Iterate Rapidly: Experiment with state-of-the-art LLM techniques, turning research into practical developer tooling. Improve Observability and Debugging: Integrate with our existing platform to surface deeper insights on LLM behavior—help teams quickly diagnose and fix issues such as hallucinations or bias. Educate and Evangelize: Write blog posts, white papers, tutorials, and documentation to help developers succeed with our open source tools and grow the LLM eval community. What We’re Looking For We’re looking for an engineer who’s deeply passionate about AI, loves working in the open, and thrives in a fast-paced environment where “everyone wears multiple hats”. You likely share our core values: Open Source Champion: You believe collaboration and community-driven development unlocks the best innovations. Creative Problem Solver: You enjoy tackling ambiguous challenges and finding elegant technical solutions. Data & Metrics Driven: You value empirical results, enjoy creating or refining evaluation metrics, and iterate based on real-world feedback. Technically Curious: You’re always learning—exploring new LLM architectures, prompt engineering strategies, or emerging library standards. Builder Mindset: You relish the process of taking ideas from initial prototypes to production-ready solutions that delight users. Desired Skills & Experience Hands-on LLM Experience: Familiarity with popular LLM frameworks, prompt engineering techniques, and agent harnesses. Strong TypeScript Proficiency: You understand TypeScript and can write isomorphic code that executes seamlessly across clients, servers, and diverse edge runtimes. Open Source Track Record: Contributions to open source projects, personal GitHub repos with interesting AI demos, or a history of active engagement in developer communities. ML Observability & Tools: Familiarity with debugging AI applications, exploring embeddings, or building data-heavy dashboards is a plus. Why Work With Us Shape the Future of AI Evaluation: Be at the forefront of designing new ways to measure and improve next-generation LLMs. High Impact, Real Ownership: Join a team that values autonomy and speed. You’ll drive major initiatives from day one and see your work used by developers worldwide. Fully Remote, Flexible Environment: We are a fully remote company with offices in the Bay Area and NYC for those who prefer in-person collaboration. Cutting-Edge Challenges: Our platform already helps analyze millions of AI predictions daily, giving you the chance to refine your evaluation tooling on real, large-scale production workloads. Work With a Talented, Passionate Team: Collaborate closely with top engineers who are dedicated to making AI more transparent, reliable, and impactful. The estimated annual salary and variable compensation for this role is between $185,000 - $200,000, plus a competitive equity package. Actual compensation is determined based upon a variety of job related factors that may include: transferable work experience, skill sets, and qualifications. Total compensation also includes a comprehensive benefit package, including: medical, dental, vision, 401(k) plan, unlimited paid time off, generous parental leave plan, and others for mental and wellness support. While we are a remote-first company, we have opened offices in New York City and the San Francisco Bay Area, as an option for those in those cities who wish to work in-person. For all other employees, there is a WFH monthly stipend to pay for co-working spaces.

The Role
As a Research Engineer, Coding Evaluation & Training Data at Surge, you'll sit at the intersection of software engineering and product to build and run the systems that teach frontier models how to code.
You’ll own end-to-end coding data projects for top AI labs, designing tasks, RL environments, and evaluation schemes that reflect real-world software engineering. This is an ideal role for a software engineer who wants to keep using their engineering skill set every day, but is most excited by training data quality, agentic evaluation, and system design (humans + models + tools) rather than traditional product feature work.
What You'll Do
Own end-to-end coding data projects, from initial scoping and pilot design through execution, iteration, and scale-up.
Design agentic training workflows and task structures that mirror real-world SWE work (e.g., refactoring, debugging, code review, large-repo navigation, tool use).
Define and iterate on rubrics, golden sets, and reward signals that capture true engineering value, not just “does it compile.”
Evaluate data and worker output with strong SWE taste, and make calls about what meets the bar for frontier training.
Design and run qualification processes for coding workers, including hands-on assessments of their coding ability.
Set up or partner on complex technical environments (e.g., containers, repos, test harnesses, sandboxes, code execution infrastructure).
Partner with technical staff at our clients to translate high-level training goals into concrete projects and technical environments.
Collaborate closely with Surge engineering, product, and operations to improve our coding data products, internal tools, and execution processes.
What We're Looking For
3–6+ years of professional software engineering experience building and maintaining real systems.
Strong coding ability in at least one mainstream language and comfort working in production codebases.
High “taste” for good engineering: you care about correctness, code quality, and how real engineering teams actually work.
Ability to reason about and debug technical environments, including containers, dependencies, and automated test setups.
Interest in owning projects end-to-end, including scoping, workflow design, execution, and continuous improvement.
Excellent written and verbal communication skills, with the ability to speak credibly with senior client engineers and translate fuzzy goals into actionable plans.
Excitement about AI/ML systems and the role of data, evaluation, and reward design in improving agentic coding capabilities.
How to Apply
We created a short  exercise
 to help us get to know you better. Once you’ve completed it, please email your submission and resume to careers@surgehq.ai and include the role you’re applying for in the subject line.

What We’re Looking for
This is a senior, hands-on engineering role for builders who want full ownership.
You love building robust, scalable backend systems.
 You're experienced in designing and implementing efficient data processing pipelines, APIs, and database architectures that can handle large volumes of data.
You're passionate about performance optimization.
 You enjoy tackling complex backend challenges and optimizing systems for speed and efficiency.
You care about the problems we’re trying to solve
. You're interested in helping make the next generation of AI models safer and more useful.
You’re excited about joining a small, growing team.
 We value autonomy and empower people to do the best work of their career.
Some Examples of the Problems We’re Solving
What’s the best way to architect large-scale simulated environments that can host hundreds of concurrent agents—each living in its own containerized world—without sacrificing stability or speed?
How can we build infrastructure that supports high-throughput model training loops and scales to thousands of environments, while keeping every agent’s world consistent, performant, and observable in real time?
How to Apply
To apply, please email  careers@surgehq.ai
 with a resume and 2-3 sentences describing your interest in Surge. We love personal projects and writings too!
‍

The Role
As an RL Environments Architect, you’ll design, instrument, and govern the simulated worlds where agents learn — from compact task microcosms to multi-agent, tool-using ecosystems. You’ll define the primitives, reward structures, interfaces, and telemetry that let us stress-test emerging capabilities while keeping training signals faithful, stable, and scalable.
Not only will you build environments, you’ll craft standards for data quality and reproducibility across large-scale agent gyms. This is a role for someone who sweats the details of simulation fidelity, thinks in terms of coverage and failure surfaces, and loves turning messy real-world phenomena into learnable curricula. Your work will form the backbone for safe, rapid progress in agentic systems.
What You'll Do
Architect a modular environment framework with clear APIs, curriculum scaffolds, and configurable reward/termination schemas
Establish quality bars: coverage metrics, invariance checks, and trace audits for environment outputs and agent experience buffers
Instrument rich telemetry for episode rollouts; mitigating reward hacking, mode collapse, and exploitable loopholes
Partner with researchers to translate real-world tasks into robust simulations, including synthetic data generators and evaluation suites
What We’re Looking for
Simulation & Systems Depth
 – Experience building RL environments or simulators (e.g., custom physics, multi-agent, tool APIs) with an eye for determinism, performance, and observability
Data Quality Leadership
 – Strong instincts for designing reward functions, scenario taxonomies, and QA pipelines that keep signals aligned and drift-free
Builder’s Mindset
 – Comfort collaborating across research and engineering to ship pragmatic, testable environments that evolve with model capabilities
How to Apply
To apply, please email  talent@surgehq.ai
 with a resume and 2-3 sentences describing your interest in Surge. We love personal projects and writings too!

The Role
As a Fullstack Engineer, you’ll design and build the core systems that power Surge’s platform — the tools and interfaces that let humans shape frontier AI models. You’ll work across the stack to make data collection, labeling, and evaluation seamless and scalable, owning everything from product design decisions to backend architecture.You’ll partner closely with product, research, and operations to create fast, reliable, and delightful tools that help the world’s top AI labs train safer, smarter, and more capable models. This role is ideal for someone who loves moving quickly, cares deeply about product quality, and thrives in high-ownership environments.
What We're Looking for
Experience building and maintaining fullstack applications using React/TypeScript and Python or Ruby on Rails.
You care about the intersection of performance, usability, and maintainability, and you design systems that are simple, scalable, and resilient.
You translate technical concepts clearly, enjoy working cross-functionally, and have a bias for shipping.
You take ownership from idea to production, iterate fast, and write code that stands the test of time.
How to Apply
To apply, please email  careers@surgehq.ai
 with your resume and 2–3 sentences describing your interest in Surge. We love personal projects, code samples, or writing too
‍

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About The Teams Our Reinforcement Learning teams lead Anthropic's reinforcement learning research and development, playing a critical role in advancing our AI systems. We've contributed to all Claude models, with significant impacts on the autonomy and coding capabilities of Claude Sonnet 4.5 and Opus 4.5. Our work spans several key areas: Developing systems that enable models to use computers effectively Advancing code generation through reinforcement learning Pioneering fundamental RL research for large language models Building scalable RL infrastructure and training methodologies Enhancing model reasoning capabilities We collaborate closely with Anthropic's alignment and frontier red teams to ensure our systems are both capable and safe. We partner with the applied production training team to bring research innovations into deployed models, and are dedicated to implement our research at scale. Our Reinforcement Learning teams sit at the intersection of cutting-edge research and engineering excellence, with a deep commitment to building high-quality, scalable systems that push the boundaries of what AI can accomplish. About The Role As a Research Engineer within Reinforcement Learning, you will collaborate with a diverse group of researchers and engineers to advance the capabilities and safety of large language models. This role blends research and engineering responsibilities, requiring you to both implement novel approaches and contribute to the research direction. You'll work on fundamental research in reinforcement learning, creating 'agentic' models via tool use for open-ended tasks such as computer use and autonomous software generation, improving reasoning abilities in areas such as mathematics, and developing prototypes for internal use, productivity, and evaluation. Representative Projects Architect and optimize core reinforcement learning infrastructure, from clean training abstractions to distributed experiment management across GPU clusters. Help scale our systems to handle increasingly complex research workflows. Design, implement, and test novel training environments, evaluations, and methodologies for reinforcement learning agents which push the state of the art for the next generation of models. Drive performance improvements across our stack through profiling, optimization, and benchmarking. Implement efficient caching solutions and debug distributed systems to accelerate both training and evaluation workflows. Collaborate across research and engineering teams to develop automated testing frameworks, design clean APIs, and build scalable infrastructure that accelerates AI research. You May Be a Good Fit If You Are proficient in Python and async/concurrent programming with frameworks like Trio Have experience with machine learning frameworks (PyTorch, TensorFlow, JAX) Have industry experience in machine learning research Can balance research exploration with engineering implementation Enjoy pair programming (we love to pair!) Care about code quality, testing, and performance Have strong systems design and communication skills Are passionate about the potential impact of AI and are committed to developing safe and beneficial systems Strong Candidates May Have Familiarity with LLM architectures and training methodologies Experience with reinforcement learning techniques and environments Experience with virtualization and sandboxed code execution environments Experience with Kubernetes Experience with distributed systems or high-performance computing Experience with Rust and/or C++ Strong Candidates Need Not Have Formal certifications or education credentials Academic research experience or publication history Deadline to apply: None. Applications will be reviewed on a rolling basis. The annual compensation range for this role is listed below. For sales roles, the range provided is the role’s On Target Earnings ("OTE") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role. Annual Salary £260,000—£630,000 GBP Logistics Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience Required field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices. Visa sponsorship: We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this. We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed. Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself prematurely and to submit an application if you're interested in this work. We think AI systems like the ones we're building have enormous social and ethical implications. We think this makes representation even more important, and we strive to include a range of diverse perspectives on our team. Your safety matters to us. To protect yourself from potential scams, remember that Anthropic recruiters only contact you from @anthropic.com email addresses. In some cases, we may partner with vetted recruiting agencies who will identify themselves as working on behalf of Anthropic. Be cautious of emails from other domains. Legitimate Anthropic recruiters will never ask for money, fees, or banking information before your first day. If you're ever unsure about a communication, don't click any links—visit anthropic.com/careers directly for confirmed position openings. How We're Different We believe that the highest-impact AI research will be big science. At Anthropic we work as a single cohesive team on just a few large-scale research efforts. And we value impact — advancing our long-term goals of steerable, trustworthy AI — rather than work on smaller and more specific puzzles. We view AI research as an empirical science, which has as much in common with physics and biology as with traditional efforts in computer science. We're an extremely collaborative group, and we host frequent research discussions to ensure that we are pursuing the highest-impact work at any given time. As such, we greatly value communication skills. The easiest way to understand our research directions is to read our recent research. This research continues many of the directions our team worked on prior to Anthropic, including: GPT-3, Circuit-Based Interpretability, Multimodal Neurons, Scaling Laws, AI & Compute, Concrete Problems in AI Safety, and Learning from Human Preferences. Come work with us! Anthropic is a public benefit corporation headquartered in San Francisco. We offer competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and a lovely office space in which to collaborate with colleagues. Guidance on Candidates' AI Usage: Learn about our policy for using AI in our application process.

At U.S. Bank, we’re on a journey to do our best. Helping the customers and businesses we serve to make better and smarter financial decisions and enabling the communities we support to grow and succeed. We believe it takes all of us to bring our shared ambition to life, and each person is unique in their potential. A career with U.S. Bank gives you a wide, ever-growing range of opportunities to discover what makes you thrive at every stage of your career. Try new things, learn new skills and discover what you excel at—all from Day One. Job Description The AI Red Team Lead Engineer leads the execution and evolution of offensive security activities focused on AI/ML systems, platforms, and integrations, in addition to traditional enterprise attack surfaces. Owns the design, execution, and reporting of AI-focused red team operations, adversarial testing, and threat emulation exercises targeting models, data pipelines, AI-enabled applications, and supporting infrastructure. Acts as a senior technical authority and program lead for AI red teaming, partnering with teams to identify, validate, and communicate AI-related risks. Drives maturity through repeatable testing methodologies, automation, custom tooling, and clear articulation of business impact. Key Responsibilities: Lead AI Red Team operations, including adversarial testing of: Foundation and custom models (LLMs, vision, speech, decision systems) Model deployment environments (APIs, plugins, agents, RAG pipelines) Training, evaluation, and inference pipelines Data ingestion, labeling, and governance controls Design and execute AI-specific threat emulation aligned to real-world adversaries, misuse scenarios, and emerging attack techniques (e.g., prompt injection, data poisoning, model inversion, jailbreaks, supply chain risks). Develop and maintain custom AI red team tooling, frameworks, and automation to scale testing and improve repeatability. Perform security research into emerging AI attack techniques, model vulnerabilities, and defensive gaps. Partner with detection, engineering, and governance teams to support purple-team and control validation activities. Contribute to AI security standards, testing guidance, and program strategy. Mentor and provide technical leadership to red team engineers. Required Skills/Experience: Bachelor's degree, or equivalent work experience Eight plus years of relevant experience Thorough understanding of the applicable information security systems, policies, and procedures Effective communication, presentation skills, leadership, problem-solving and analytical skills Proven collaboration and influencing skills Preferred Skills/Experience: Hands-on experience testing or securing AI/ML systems, including LLMs or other model classes Knowledge of AI threat models and attack techniques including, but not limited to, prompt injection, model extraction, training data poisoning, inference abuse, hallucination exploitation Familiarity with AI platforms and tooling (e.g., model APIs, orchestration frameworks, evaluation pipelines) Significant red team experience, including adversary emulation and multi-stage attack chains Proven skill developing proof-of-concept exploits and custom offensive tooling Strong understanding of red team and offensive AI techniques and tooling Expertise defeating or bypassing endpoint and AI-adjacent security controls (EDR/XDR, API protections, guardrails) Experience with cloud, containerized, and AI-hosting environments Proficiency in one or more languages (e.g., Python, PowerShell, Go, C/C++, Shell) Ability to translate research into operational tooling Exceptional written and verbal communication skills This role requires working from a U.S. Bank location three (3) or more days per week. If there’s anything we can do to accommodate a disability during any portion of the application or hiring process, please refer to our disability accommodations for applicants. Benefits: Our approach to benefits and total rewards considers our team members’ whole selves and what may be needed to thrive in and outside work. That's why our benefits are designed to help you and your family boost your health, protect your financial security and give you peace of mind. Our benefits include the following: Healthcare (medical, dental, vision) Basic term and optional term life insurance Short-term and long-term disability Pregnancy disability and parental leave 401(k) and employer-funded retirement plan Paid vacation (from two to five weeks depending on salary grade and tenure) Up to 11 paid holiday opportunities Adoption assistance Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year unless otherwise provided by law Review our full benefits available by employment status here. U.S. Bank is an equal opportunity employer. We consider all qualified applicants without regard to race, religion, color, sex, national origin, age, sexual orientation, gender identity, disability or veteran status, and other factors protected under applicable law. E-Verify U.S. Bank participates in the U.S. Department of Homeland Security E-Verify program in all facilities located in the United States and certain U.S. territories. The E-Verify program is an Internet-based employment eligibility verification system operated by the U.S. Citizenship and Immigration Services. Learn more about the E-Verify program. The salary range reflects figures based on the primary location, which is listed first. The actual range for the role may differ based on the location of the role. In addition to salary, U.S. Bank offers a comprehensive benefits package, including incentive and recognition programs, equity stock purchase 401(k) contribution and pension (all benefits are subject to eligibility requirements). Pay Range: $126,820.00 - $149,200.00 U.S. Bank will consider qualified applicants with arrest or conviction records for employment. U.S. Bank conducts background checks consistent with applicable local laws, including the Los Angeles County Fair Chance Ordinance and the California Fair Chance Act as well as the San Francisco Fair Chance Ordinance. U.S. Bank is subject to, and conducts background checks consistent with the requirements of Section 19 of the Federal Deposit Insurance Act (FDIA). In addition, certain positions may also be subject to the requirements of FINRA, NMLS registration, Reg Z, Reg G, OFAC, the NFA, the FCPA, the Bank Secrecy Act, the SAFE Act, and/or federal guidelines applicable to an agreement, such as those related to ethics, safety, or operational procedures. Applicants must be able to comply with U.S. Bank policies and procedures including the Code of Ethics and Business Conduct and related workplace conduct and safety policies. Posting may be closed earlier due to high volume of applicants.

Scale’s mission is to develop reliable AI systems for the world's most important decisions. Our core work consists of: Creating custom AI applications that will impact millions of citizens Generating high-quality training data for national LLMs Upskilling and advisory services to spread the impact of AI Scale is hiring ML Research Engineers to bridge the gap between emerging AI capabilities and mission-critical, real-world impact. In our Global Public Sector (GPS) division, we don’t just implement tools; we conduct applied research to solve the unique challenges of sovereign AI. Your role is to move beyond off-the-shelf implementations. You will lead the research into Agent Design, Reliability, and AI Safety, developing novel system architectures that power high-stakes government applications. You will be the bridge between a research paper and a production-ready system that functions at scale. The Mission Applied Agent Research: Leading the design of reliable, multi-step agentic systems and long-horizon reasoning frameworks that can solve complex problems for national security and public policy. Systemic Evaluation & Red-Teaming: Developing rigorous benchmarks and evaluation protocols to ensure AI systems are safe, unbiased, and performant in high-stakes, non-commercial environments. Model Optimisation & Selection: Conducting deep-dive research into model performance (both open-weight and closed) to identify the best tools for niche domains, optimising them through context engineering, RAG, and other inference-time techniques. What You Will Do Architect Agentic Systems: Design and build agent architectures, the harnesses, tool-use protocols, and logic flows that allow LLMs to function as reliable, autonomous agents in complex workflows. Drive Reliability & Safety: Research and implement robust evaluation frameworks. This includes red-teaming for sovereign AI requirements and developing strategies to mitigate hallucinations in regulated data environments. Synthesise Deep Research: Build agents capable of autonomous information synthesis and long-horizon reasoning, enabling users to analyse massive datasets and extract actionable insights. Optimize for Niche Domains: Evaluate and adapt models for specialised use cases, such as LLM reasoning for low-resource languages, complex OCR tasks, or working in GPU-constrained environments Build Evaluation Frontiers: Create new, automated benchmarks that define what success looks like for AI in the public sector, ensuring our systems meet the highest standards of accuracy and sovereignty. Consult as a Technical Authority: Act as a subject matter expert for public sector leaders, advising on the practical limits, safety requirements, and performance trade-offs of emerging AI technologies. Ideally, You Have Engineering Rigour: Exceptional proficiency in Python and experience building agentic harnesses or AI infrastructure. You write production-ready code that is modular, scalable, and reliable. Applied Research Mindset: A track record of taking theoretical AI concepts and turning them into functional prototypes or products. You know how to read a paper and determine if its methods are actually viable for a production system. Evaluation Expertise: Experience in LLM benchmarking, red-teaming, or building evaluations that go beyond standard academic datasets. Advanced Degree: A Master’s or PhD in Computer Science, Mathematics, or a related field (with a focus on ML) is preferred, but we value demonstrated impact and engineering excellence. Nice to Haves Agentic Systems Expert: Deep experience in building multi-agent systems, including chain-of-thought optimisation and tool-calling reliability. Sovereign AI Experience: Experience working with highly regulated data environments, on-premise deployments, or sensitive government use cases. Inference Optimisation: Knowledge of how to optimise model performance for environments with limited GPU capacity or specific latency requirements. Zero-to-One Mindset: You are comfortable navigating ambiguity and enjoy defining research directions from scratch to solve a specific product or mission need. For those applying based in Qatar: Residency and employment in Qatar requires certain permissions (visa and permits) issued by the Qatari authorities. As part of the application process, candidates will be asked to provide personal information, including residency status and nationality, which is required for visa processing. This information is collected solely for immigration compliance purposes and will be not used as a criterion in any hiring decision unless such use is lawfully permitted. *Visa issuance is at the discretion of the Qatari authorities. If you are successful in your application, you will be required to provide the documentation requested by Scale and the authorities to obtain the necessary permissions for you to live and work in Qatar, and Scale will work with successful candidates to support the visa application process.*