Helping non-technical teams trust, verify, and control an AI agent
TL;DR
Latch is a seed-stage startup building AI automation for companies without an in-house technical team. For my master's capstone, I led the design of the voice-led workflow capture experience: how the agent listens as an employee talks through their work, when it speaks up to ask for clarification, and how they review and correct what it captured. Latch used the concept to run its first pilots. Designed for people with no technical background, so they could trust and control an AI working on their behalf.
Organization
Latch
Role
Product Designer
Duration
6 months
Team
PM, UX Researcher, 2 Product Designers
Tools Used
Impact
Designed the experience an $8M seed-stage startup took into its first pilots.
Latch had funding, users, and no product. The capture experience, the agent's behavior, and the design system we handed them became the foundation of what they piloted and what is live today.
// WHAT I SHIPPED
Interactive prototype
End-to-end clickable prototype of the product, based on the research and testing findings.
Built in Cursor
Agent behavior
Rules for how the agent listens, when it speaks, when it waits for a natural pause, and when it yields to the user, with the settings that let users tune them.
Specifications
Design system
Machine-readable design system in markdown, covering tokens, states, accessibility, and usage rules.
MD file
The User
The people who most want automation are the least equipped to build it.
Operators in manufacturing, real estate, distribution, construction, financial services. Take an office manager at a 200-person real estate firm: 90 minutes every Monday moving vendor invoices from email into folders, a spreadsheet, and a payment run. She knows the workflow better than anyone, and there is nobody to turn it into software.
Has
All the context
+
Missing
A way to hand it over
+
Missing
Technical skill
=
To get
An automation
The Landscape
Finding where repetitive work lives
I analyzed Bureau of Labor Statistics time-use data on 252,808 workers (2003 to 2024): 44.9% of the workday goes to manual, repetitive tasks, over $20,000 of salary per worker per year. That told us which industries to recruit from.
Of an average workday
Priced at each worker's wage, that time costs $13,600–$26,100 per personper year.
Where repetitive work concentrates
Hours per week on manual, repetitive tasks.
Source: American Time Use Survey, U.S. Bureau of Labor Statistics, 2003 to 2024. Cost annualized over 52 weeks.
Competitive Analysis
Where the friction is: Every failure we heard about happened at the same moment: explaining the workflow to the tool. We scoped the project to that exchange.
DIRECT
PROCESS AUTOMATION
MAJOR PLATFORM
ENTERPRISE
User Problem
The user doesn't know what the agent needs to know.
We carefully picked six participants from different backgrounds, all at companies without an in-house technical team. Two things were consistent: they didn't trust automation they couldn't see, and they didn't feel they could control something they didn't understand.
Setup takes time and knowledge they don't have: Learning a tool on top of a full workload is where people stop.
They can't trust what they can't see: Invisible observation was the biggest adoption blocker.
They can't control what they don't understand: With no mental model of what the automation would do, people went back to doing it by hand.
They're the ones who own it if something goes wrong: An automation runs in their name. That fear alone kept people doing it by hand.
How might we make workflow documentation feel approachable to non-technical users?
Automations fail because the context behind the work never reaches the system. The person doing the job has it. Handing it over is the hard part.
THE TENSION
The more the agent sees and asks, the better the automation. The more it sees and asks, the less comfortable the person.
We had to balance the two: enough that's new to make the agent feel capable, enough that's familiar to make it feel controllable.
Why I thought of meeting users where they are
These users aren't reaching for AI tools daily. What they do every day is jump on a Teams call, share their screen, answer an email. So my bet was to borrow patterns from the apps already in their day, and make showing an agent your work feel like nothing out of the ordinary. I assumed it would be easier for someone to share a screen and talk to an agent about their work than to start a recording and narrate into the air.
Exploration
Prototyping two extremes of agent presence to test where control lives.
We expected the answer in the middle; building both ends was how we'd find it. Both built in Cursor, so testing ran on real interactions.
A more modern approach
Patterns from modern AI products. Latch stays out of sight while you work, then hands back a full transcript and recap to verify and correct.
Meeting users where they already are
Cues from the apps they use every day. Latch joins like a participant in a call: a recording indicator, screen share, a question asked in the moment.
What testing both ends told us
Nobody landed clearly on either side. Users wanted different things at different points in the session. They liked the comfort of familiar patterns, and they wanted less noise than those patterns usually bring. They wanted to stay in control without having to look at as much as they are used to seeing. That meant balancing both, so the session split into states, each carrying only the controls that moment needs.
// FINAL SOLUTION
Designing the session.
A voice-led capture session built around permission, clarification, and review. Three states, each asking something different of the user: permission before anything is captured, context while they work, and verification before anything is built.
STEP 01
Pre-call briefing
What Latch watches, listens to, won't do
STEP 02
Workflow capture
Show Latch how you work
STEP 03
Follow-up
One anchored question at a time
STEP 04
Revisit
Review, correct, re-record
STEP 05
Build
Nothing runs until confirmed
BEFORE · Privacy
The goal: The user should feel in control of what's captured.
Users are wary of AI observation
A system asking for accessibility or microphone access makes these users nervous about how their data will be used. So Latch explains each one in plain language before macOS asks, informing without alarming.
Users are in control
Knowing they could stop the recording is what made people comfortable being recorded at all. So pause and mute sit in the main control bar, borrowed from the recording controls they already know.
DURING · Capture
The goal: The user should not feel uncomfortable when Latch asks for more.
Latch stays peripheral
Users had to know they were being recorded without carrying that weight while working. So Latch sits in a corner overlay: always visible, never in the way, with the controls right there.
In-session clarification
Some users wanted to answer a follow-up right away, others wanted to finish first. So Latch keeps every question in a running list: answer it in the moment, or come back to it later.
AFTER · Review
The goal: The user should never feel wrong for changing something.
Latch shows its work
Seeing the steps written back to them is what made users trust that Latch had understood. So the recording plays beside the step-by-step workflow it extracted.
Users correct without starting over
The user types a correction, re-records a step, or adds context. Prior context is preserved, so fixing a mistake never costs what was already captured.
Latch hands back what it shouldn't own
Every workflow has decisions a person needs to oversee. So those steps are flagged as manual, proof that the automation won't go rogue and run the whole thing without them.
Designing the Agent
Designing an agent users were comfortable being watched by.
Users were uneasy about an agent listening to them work, and that discomfort showed up in how much context they gave. So the behaviors got specified as carefully as any screen, and anything affecting comfort became a setting they control rather than a default we picked.
Designing responsible AI
Agents that observe people at work are new enough that the norms aren't set, and the defaults chosen now become what users expect from every tool after. An agent like this can slide into surveillance, and one person's way of working can quietly become everyone's. So consent is explicit, nothing is kept unless the user saves it, and irreversible actions stay with the human.
Latch listens
The agent is designed to follow along attentively, tracking what's been covered, anchoring new questions in what came before, and confirming by reflecting things back to the user.
Latch watches
The agent is designed to ground the conversation in what's visible on screen. It notices when the user switches views or takes action, referencing this context in its questions.
Latch waits
The agent is designed never to interrupt mid-thought, and never to sit on silence. It listens for the natural beats in the user's speech and uses those moments to confirm, recap, or ask one anchored question.
Latch earns every question
The agent is designed to ask only what the workflow needs. It probes for branches and exceptions instead of taxonomy or trivia.
// Handoff
Handing off to a startup that had no designer.
The design system
I owned the design system and wrote it as latch-design-system.md: tokens, states, contrast ratios, and a do and don't table. Anything built next inherits it, whether a person or a coding agent is building.
The working repo
Commented for intent, with flows and edge cases as markdown beside the code and open issues carrying full reasoning. The team continues from code instead of redrawing mockups.
// Reflection
What I learned
Trust and control for a voice-led agent had no patterns yet.
Agentic capture had no established interaction patterns to lean on. The answer was borrowing trusted vocabulary from elsewhere and letting testing decide where it belonged.
Scoping is the job at this stage.
The pull at a seed-stage startup is to keep building. The harder call was converging on one direction late in the project and making it real enough to hand over.
