Forward deployed engineers ship into client environments. You own what they ship with: orchestration, state and memory, tool interfaces, cost controls, and the evaluation and tracing that turn our reliability claims into evidence.
The hardest question in this work is not whether a system works on the day it ships. It is whether it still works a month later, when the model provider has changed something, the client's data has drifted and nobody has read a trace in weeks. This role owns that question, and the architecture that makes it answerable.
It is a deep engineering seat with a short feedback loop. What you design runs in a client's production within weeks, and you will hear directly from the engineer who had to debug it at midnight.
What you will do
- Set the architecture of our agentic layer: orchestration and delegation between specialist agents, clean failure semantics, and state that survives a restart
- Own tool and integration interfaces, including MCP servers for the systems our clients run
- Build the evaluation harnesses and golden datasets each client system is released against, including LLM judges calibrated against human labels
- Own tracing from prompt to output, and catch regressions across model upgrades, prompt changes and data drift before a client does
- Design cost guardrails, rate limits and the approval checkpoints for decisions too consequential to automate outright
- Turn patterns that worked once into components and standards the whole delivery team works to
What we look for
- 8+ years of software engineering, including several years designing APIs and abstractions other engineers depend on
- Excellent Python, and deep hands-on work with LLM orchestration, on a framework or hand-rolled
- Distributed systems instincts: idempotency, retries, backpressure and partial failure
- Experience evaluating LLM systems in production, including where offline metrics stop predicting real behaviour
- Statistical literacy sufficient to know when a difference between two runs means nothing
- Firm views on reliability that change with evidence, and the willingness to say a system is not ready
Nice to have
- Open source contributions to agent, evaluation or LLM tooling
- You have built and operated an MCP server against a real production system
- Experience in regulated or safety-critical environments
Your first six months
- Month 1: instrument a live engagement from prompt to output and find something nobody knew was broken
- Month 3: own the orchestration and evaluation layers, with a suite running against every active engagement
- Month 6: regressions are caught by our tooling rather than a client email, and the architecture reflects decisions you can defend