← Insights
Product
Vela: an AI operator that builds, tests and improves Emily
Harry Yu
Vela is an AI operator from Level3AI that builds, tests and maintains Emily, Level3AI’s customer service AI agent, so that one AgentOps specialist can run Emily for a brand. Every change Vela makes is tested against evals before your team sees it, and nothing goes live without human approval.
Emily started as an agent that answered customer questions. Today she works across chat, email and voice, talks with several stakeholders in a single case, looks up information, and takes actions in the systems behind a brand’s support operation.
Every new capability makes Emily more useful. It also makes her harder to keep flexible and accurate at the same time. Keeping that balance is no small feat, and today we’re introducing Vela, the Operator for Emily, to carry it.
“Before Vela, a playbook change took one to two weeks and usually meant pulling in other teams. Now our ops team takes an issue from root cause to live in under an hour.Vela reads the actual API responses behind a ticket and looks across our past cases, so we can see how a change is likely to play out before it ever reaches our members. We’re deciding on evidence rather than instinct.”
Vela at a glance
What it is: an AI operator that works in the Emily Portal, Level3AI’s console for configuring, testing and monitoring Emily.
Who it’s for: CX and AgentOps teams that run Emily across chat, email and voice.
What it does: builds playbooks, variables and workflows, tests them with eval simulations, and finds what is slipping in production and brings back tested fixes.
What stays with people: publishing to production and managing access.
Models: the most advanced frontier models available.
Raaqib Ruslan
AgentOps Specialist, GetGo
Our vision has always been simple: one AgentOps specialist should be able to operate Emily for a brand. AgentOps is the operational function responsible for managing an AI agent throughout its lifecycle. That vision started slipping away as the Emily Portal, where teams configure, test and monitor Emily, grew.
Over the past two years we added dev, UAT and production environments. We added different types of playbooks (numbered, natural-language instructions with explicit conditions that Emily follows) for different use cases, collaborative editing and versioning, and parsing tools for different kinds of documents. Then came QA, evals, gap analysis and MCP.
Each of these made Emily better. Each also asked the AgentOps specialist to learn one more tool.
We knew this in 2024
When Level3AI first launched the Emily system in 2024, we already knew we would need an AI operator to do the heavy lifting. Without one, the Portal would only work well for someone who is a product manager, an engineer and a data scientist at once.
The models of 2024 weren’t ready. They couldn’t master that many tools, and they couldn’t reliably propose good solutions in such a large search space. So we built the tools for people first, and waited.
The tools were always meant for Vela
This year the models caught up, and the waiting is over. The cost is affordable enough for everyday operations.
We also learned something we didn’t expect. The tools we built for AgentOps are far more powerful in Vela’s hands. A person uses them one at a time. Vela chains them together and keeps going.
Vela only hands off changes that pass every eval. We call this the eval-gated loop.
For every change, Vela works in a loop:
Build the change, whether a playbook, a variable or a workflow
Run eval simulations against it
QA the results
Update the change and repeat until every eval passes
Only then does Vela send the change to AgentOps for approval. Your team reviews finished, tested work instead of first drafts.
How Vela is different from an AI copilot
A copilot drafts a playbook or a setting and leaves the testing to your team. Vela drafts the change, tests it, fixes what fails and retests until every eval passes, then sends it to AgentOps with the results attached. Without Vela, the usual alternative is a product manager, an engineer and a data scientist working on every change together.
Give Vela your API documents or an MCP server, and it reads the raw APIs and designs the business variables (the facts Emily uses to make a decision, from API values, the conversation and system metadata) and workflows your playbooks need. Work that used to require an engineer to design APIs now starts with a conversation.
Vela also understands the settings across the Portal. It adapts them to how your business actually runs, so your team doesn’t have to learn each one.
You don’t need to be a data scientist: Vela checks false positives and false negatives
Vela makes sure every analysis covers both sides of the coin. It checks false positives and false negatives, such as escalations that weren’t needed and escalations that were missed, so fixing one failure doesn’t quietly create another.
Vela runs A/B tests to compare options and regression tests to confirm existing conversations still work. When something goes wrong, it finds the root cause and sends the evidence to the team best placed to fix it.
You don’t need to be a product manager: Vela proposes the design options
Vela proposes design options and lets you choose. That covers UI components, Journey Issue design (how Emily recognises the different ways customers ask for the same goal, so the right playbook runs), escalation handoff workflows and proactive nudge messages.
You pick the direction. Vela builds it, tests it and brings it back for your approval.
Vela can do almost everything a person can do in the Portal. Two things stay with people: publishing to production and managing access. Every change Vela proposes arrives with its eval results, and nothing goes live without a human approving it.
This brings us back to our original vision. One AgentOps specialist operates Emily, and Vela does the heavy lifting.
Vela lowers the effort of running CX AI at both ends of the journey: getting started, and getting better over time.
Faster, easier onboarding.
Vela reads your API documents, knowledge and existing processes, then builds and tests the first playbooks, variables and workflows. Going live no longer depends on a long implementation project.
Our own team used Vela to rebuild an Emily project from scratch in 7 days. Building it the first time, two years ago, took us more than 12 weeks.
More effective maintenance and continuous improvement.
Products, policies and customers change every week. Vela watches how Emily performs, finds what is slipping and brings back tested fixes, so Emily keeps improving instead of drifting out of date.
On their first day with Vela, Carousell’s team updated and built a playbook in 15 minutes. The same work used to take them two days.
This matters most for brands that run Emily in several markets and languages, as many of our customers across Asia-Pacific do, because one policy change has to be right in every market.
Together, this turns CX AI transformation and continuous improvement into a much lower lift.
It also changes the role of the people on your team. They spend less time operating tools and more time as the strategic mind, deciding what a better customer experience should look like for their customers. Your team sets the direction. Vela and Emily handle the rest.
Try Vela
Vela is now available. It runs on the most advanced frontier models. If you run a CX team and want to see Vela build and test an Emily project, get in touch.
About the author
Harry Yu is co-founder of Level3AI.
Frequently asked questions
What is Vela?
Vela is an AI operator from Level3AI. It builds, tests and maintains Emily, Level3AI's customer service AI agent, in the Emily Portal, so one AgentOps specialist can run Emily for a brand.
Does Vela publish changes to production on its own?
No. Vela tests every change with eval simulations and sends it to your AgentOps team with the results. Publishing to production and managing access stay with people.
Which AI models does Vela use?
Vela became practical with the latest generation of AI models. It runs on the most advanced frontier models available.
Do I need engineers or data scientists to use Vela?
No. Vela reads your API documents or MCP server to design variables and workflows, and it runs the analysis, A/B tests and regression tests itself. Your team sets the direction and approves each change.
How is Vela different from an AI copilot?
A copilot drafts a change and leaves the testing to your team. Vela builds the change, tests it, fixes what fails and retests until every eval passes, then sends it to your team for approval.
Written by
YOUR WORKFLOW / YOUR RULES
Try Emily
