Ph.D. Thesis
University of Colorado Boulder
publications
CAIRO Lab produces the foundational research to create robots that behave safely, reliably, and capably. Our robots and agents explain themselves, learn from people, assist people with tasks, and imagine the parts of the world they haven't seen yet. Click on a conference or journal paper for a guided, scrollable walkthrough. Filter by research theme or venue to explore.
Gives assistive devices a real-time sense of where they are inside cluttered, constantly-restocked spaces like grocery stores — using inexpensive cameras and inertial sensors instead of costly infrastructure.
Speeds up robot preference learning by reasoning counterfactually and redesigning the environment itself, so each question a robot asks a human is maximally informative.
IEEE Conference on Neuro-Inspired Computational Elements (NICE 2026) · Atlanta, USA
ICRA 2026 Workshop on Semantics for Reliable Robot Autonomy: From Environment Understanding and Reasoning to Safe Interaction
ICRA 2026 Workshop on Bridging the Gap between Robot Learning and Human-Robot Interaction
Lets exploring robots generatively "fill in" the unseen parts of a building's occupancy map, so they can plan intelligently beyond the reach of their sensors.
A robot coach that explains its policy in a single, semantically meaningful tip — improving human task performance without overwhelming people with detail.
Iteratively folds latent human knowledge into trajectory optimization specifications, improving both what the robot learns and how the human-robot team performs.
Runs a diffusion model at the frontier of exploration to predict the 3D structure ahead of the robot, probabilistically reconciling predictions with the map as it grows.
Shows that a handful of human demonstrations can dramatically cut the computational cost of solving sequential constrained motion-planning problems.
Sustainability Recognition (for contributions to increased computational efficiency of motion planning)
IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2025) · Eindhoven, Netherlands
ACM International Conference on Neuromorphic Systems (ICONS 2025) · Seattle, USA
Workshop on Human-in-the-Loop Robot Learning- Teaching, Correcting, and Adapting at RSS 2025 · Los Angeles, USA
A comprehensive survey mapping how augmented reality is used to communicate, teach, and coordinate in human-robot collaboration.
Generates pattern-based conventions — predictable habits — that make robot teammates far easier for humans to anticipate during collaboration.
SceneSense: a diffusion model that synthesizes complete 3D occupancy from partial observations, letting robots reason about space they haven't seen yet.
Reveals how the timing of a robot's failures skews human perception of its competence and trustworthiness — recency bias in human-robot teams.
Optimizes the shared workspace itself so that human motion becomes easier to predict during close-proximity collaboration.
Nominated for Best Technical Paper
Gives each of a robot's software controllers its own social identity, so people calibrate trust to the program actually running — not the body it runs on.
Automatically assesses quadrotor-piloting skill and coaches learners with adaptive multimodal feedback — yielding more, and safer, landings.
Virtual, Augmented, and Mixed Reality for Human-Robot Interaction Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO, USA
Proceedings of the 23st International Conference on Autonomous Agents and Multi-agent Systems (AAMAS 2024) · Auckland, New Zealand
Causal Learning for Human-Robot Interaction Workshop at ACM/IEEE International Conference on Human-Robot Interaction 2024 (HRI 2024) · Boulder, CO, USA
Virtual, Augmented, and Mixed Reality for Human-Robot Interaction Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO, USA
Robo-Identity Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO, USA
Human-LLM Interaction Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO, USA
Human-LLM Interaction Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO, USA
Proceedings of the 38th AAAI Conference in Artificial Intelligence · Vancouver, Canada
HRI Pioneers Workshop at the ACM/IEEE International Conference on Human-Robot Interaction · Boulder, CO
Robots that justify their recommendations: autonomously generated, AR-delivered justifications improve decision support in human-robot teams.
Studies when and why humans ignore a robot's claims of spatial ownership delivered through augmented reality — with direct implications for teaming safety.
ShelfHelp: a socially assistive robotic cane that guides people with vision impairment through grocery shopping, from locating products to grasping them.
AAAI Fall Symposium Series · Arlington, VA
Workshop on Life-Long Learning with Human Help (L3H2 2023) · London, UK
Workshop on Life-Long Learning with Human Help (L3H2 2023) · London, UK
Workshop on on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction at the ACM/IEEE International Conference on Human-Robot Interaction · Stockholm, Sweden
Human-Robot Interaction Late-Breaking Reports (HRI 2023) · Stockholm, Sweden
PokeRRT treats poking as a first-class manipulation skill — and as a recovery tactic for when grasping fails.
Blends descriptive and prescriptive AR visual guidance to keep human-robot teams sharing the same situational picture.
Best Student Paper Runner-up
Plans robot navigation through pedestrian crowds by folding human intent into an extended-space POMDP.
A perceptive robotic cane with haptic guidance that helps its user choose and reach a seat — restoring the quiet social dynamics of picking where to sit.
IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2022) · Naples, Italy
Social and Cognitive Interactions for Assistive Robotics (SCIAR) workshop, IROS 2022 · Kyoto, Japan
RSS Pioneers Workshop at Robotics: Science and Systems (RSS 2022) · New York City, New York
Workshop on on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction at the ACM/IEEE International Conference on Human-Robot Interaction
A robot that asks the right questions: dialog-driven semantic constraint specification for learning new skills and repairing broken ones.
ARC-LfD lets people inspect and update a robot's learned skills through augmented reality, without re-demonstrating everything from scratch.
Proceedings of the "Accessibility of Robot Programming and Work of the Future" Workshop at RSS 2021 · Virtual
Proceedings of the SpaceCHI Workshop at CHI 2021 · Virtual
Robots for Learning – Learner-Centered Design Workshop at the International Conference on Human-Robot Interaction · Virtual
A structured survey of mental model theory in human-robot teaming — how formalizing what teammates believe about each other enables fluent, trustworthy coordination.
Proceedings of Workshop on Assessing, Explaining, and Conveying Robot Proficiency for Human-Robot Teaming at HRI 2020. · Cambridge, UK
Proceedings of HRI Pioneers Workshop at HRI 2020. · Cambridge, UK
Proceedings of the AIAA Scitech 2020 Forum · Orlando, FL
Proceedings of the 34th AAAI Conference in Artificial Intelligence · New York City, NY
An online algorithm that recognizes and segments human activities from partial motion trajectories — early enough for a robot teammate to plan around them.
RARE: a robot that notices when its human teammate misunderstands the task, and repairs their mental model with a well-timed explanation.
Best Technical Paper Runner-up
HRI Pioneers Workshop at the 2019 ACM/IEEE International Conference on Human Robot Interaction (HRI 2019) · Daegu, South Korea
Second International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction at the ACM/IEEE International Conference on Human-Robot Interaction · Daegu, South Korea
Puts robotic decision support on a hospital labor-and-delivery floor and measures when nurses and doctors trust it — and when they trust it too much.
CC-LfD weaves conceptual constraints into learning from demonstration, so robots learn robust skills — and repair broken ones — despite imperfect human examples.
RAPTOR delivers fast, state-of-the-art activity recognition that can also explain its reasoning — interpretable enough for non-experts to fix.
Robots that describe their own control policies in plain language, answering questions like "what do you do when the part is damaged?"
No publications match that combination of filters — try broadening one of them.
IEEE Robotics and Automation Letters (RA-L) · 2026
Grocery stores and warehouses are "quasi-static" — the shelves stay put, but what's on them changes constantly. The result is repetitive geometry, dynamic clutter, and perceptual noise: every aisle looks like every other aisle, and standard vision-based localization simply gets lost.
Instead of memorizing exact landmarks that will have moved by tomorrow, ShelfAware treats scene semantics as statistical evidence over object categories. Knowing you are "probably near the cereal" is far more durable than remembering exactly which boxes were where.
ShelfAware fuses a depth likelihood with category-centric semantic similarity inside Monte Carlo Localization, and uses a precomputed bank of semantic viewpoints to make "inverse proposals" — fast, targeted guesses about where the camera must be to see what it's seeing. It all runs on low-cost, vision-only hardware.
In a rigorously controlled mock retail environment, ShelfAware achieved a 97% global localization success rate and the highest tracking success across cart, wearable, and dynamic-occlusion conditions. In a real 3,500 sq. ft. operational grocery store, it significantly outperformed both geometric and fixed-landmark semantic baselines.
This is an infrastructure-free localization building block for mobile and assistive robots in messy, real-world spaces — the same technology line that lets our ShelfHelp assistive shopping cane know exactly where its user is standing.
IEEE International Conference on Robotics and Automation (ICRA 2026) · Vienna, Austria · 2026
As robots take on more complex environments, hand-coding the "right" behavior becomes impossible — they need to learn preferences by asking people to compare example behaviors. But existing active preference learning samples queries from fixed trajectory sets, so many of the questions it asks barely distinguish one candidate preference from another.
CRED lets the robot imagine new scenarios rather than reusing old ones. By sampling possible rewards from its current belief and asking "what if this were the true preference?", it generates trajectory pairs that expose exactly where competing reward functions disagree.
CRED jointly optimizes environment design and trajectory selection: an outer optimization designs informative environments, while an inner counterfactual reasoning step selects the trajectory pair whose ranking will teach the robot the most about what the human wants.
Across comprehensive experiments and a user study, CRED significantly outperformed state-of-the-art methods in reward accuracy and sample efficiency — and users rated its questions higher, too.
Preference learning only scales if robots ask few, good questions. A robot that designs its own quiz learns what you want without pestering you.
Autonomous Robots (AURO) · 2025
Exploring robots traditionally plan only over geometry they have directly measured. At every corner and doorway the map runs out, and the robot slows down or stalls while it waits for its sensors to catch up.
People don't stop at doorways — we infer what rooms probably look like. SceneSense gives robots that same commonsense: a diffusion model trained to predict 3D occupancy from partial observations, filling in the unseen parts of the world with plausible structure.
Predictions are fused probabilistically into the robot's running occupancy map in real time, on board a quadruped. Observed space is never overwritten — the generative guesses only ever fill the gaps that sensors haven't reached.
SceneSense-enhanced maps matched ground truth far better than sensor-only maps (a 24% improvement around the robot, 76% at range), and dropping them into an off-the-shelf exploration stack improved robustness and traversal time across two very different environments.
Exploration is the bottleneck for robots in disaster response and inspection. A robot that can imagine what it hasn't seen yet explores faster and more consistently — without new sensors.
Springer Nature Neural Computing and Applications: Special Issue on Explainable Sequential Decision-Making · 2025
A robot that understands a task better than its human teammate faces a delicate problem: how do you fix someone's strategy without lecturing them? Effective coaching means finding the one piece of information that will actually change behavior.
SPEAR models the human as a reinforcement learning agent acting on a slightly-wrong reward function, then computes the single semantic explanation — like "there are bad rewards when cereals are placed in the trash can" — that best reconciles the difference between the robot's understanding and theirs.
SPEAR is a sequential optimization algorithm that searches combinations of planning predicates to build its one-shot advice, augmenting the human's reward function so their own planning produces better behavior. It runs dramatically faster and on much larger problems than prior policy-elicitation approaches.
Across human-subjects studies, SPEAR's single-shot coaching improved people's policies and reduced cognitive load, while boosting interpretability, task awareness, and active thinking.
From disaster evacuation to everyday collaboration, the bottleneck is often human understanding. A teammate that can say the right sentence at the right moment makes the whole team perform better.
IEEE International Conference on Robotics and Automation (ICRA 2025) · Atlanta, USA · 2025
Generative models can infer unseen geometry from partial observations — but early versions were too slow for real-time use and could only predict the area immediately around the robot, right where sensors were about to look anyway.
By removing attention-based visual conditioning and feature extraction from the diffusion model, this work cut runtime by 73% with minimal accuracy loss — fast enough to predict occupancy anywhere on the map, including the frontiers where exploration decisions actually get made.
A new probabilistic update method merges predicted occupancy into the running map, improving frontier occupancy prediction by 71% over previous methods. The code ships as a ROS node that runs on-robot.
Frontiers are where exploration lives or dies. Predicting what's beyond them — online, in real time — turns "explore cautiously" into "explore with foresight."
ACM/IEEE International Conference on Human-Robot Interaction (HRI 2025) · Melbourne, Australia · 2025
Sustainability Recognition (for contributions to increased computational efficiency of motion planning)
Real tasks chain constraints: keep the mug upright, then tilt to pour, then return. Each constraint set defines a manifold in planning space, and a solution must pass through the intersections between them. Pick a bad intersection point and the planner may search forever without finding a way forward.
People already solve these problems fluidly. Human demonstrations don't just show what to do — they implicitly define the sequential manifold planning problem and reveal which transition regions actually work.
The method extracts distributions from Learning-from-Demonstration models near constraint transitions, and uses them as candidate intersection points in an optimization that relaxes the usual (and often false) assumption that any intersection point will do.
Demonstration-informed intersection points let planners solve sequential manifold problems dramatically more efficiently. The work received HRI 2025's Sustainability Recognition for its contribution to computational efficiency.
Constrained planning is a workhorse of practical manipulation. Making it cheaper — using data people already provide — makes capable robot behavior more accessible and less energy-hungry.
IEEE Robotics and Automation Letters (RA-L) · 2025
Trajectory optimization is only as good as its specification — and the people who understand a task best often hold that understanding tacitly, as intuition that never makes it into the robot's objectives and constraints.
Rather than demanding a perfect specification up front, this work elicits human knowledge iteratively: each round of feedback surfaces a little more of what the expert knows, folding it into the optimization specification as updated constraints and objectives.
Iteratively adding latent human knowledge improved both the learned behavior and downstream task outcomes — evidence that the path to better robot behavior often runs through better listening, not just better solvers.
Researchers have been combining augmented reality with human-robot collaboration for nearly three decades, but the last comprehensive review of the area was published in 2008 — before modern headsets existed. This survey maps everything prominent since.
The review covers the forms of AR technology used for collaboration, the application areas where they've been deployed, the main technological contributions of the literature, and — critically for newcomers — commonly used evaluation methods with suggestions for implementation.
The paper closes with future research directions for the field, serving as a primer and reference for anyone whose work combines augmented reality with any kind of human-robot collaboration — including much of CAIRO's own research program.
IEEE International Conference on Intelligent Robots and Systems (IROS 2024) · Abu Dhabi, UAE · 2024
When robots turn corners in hallways or enter new rooms, they typically pause to re-plan over newly visible space. Planning exclusively over directly measured geometry makes exploration slow and hesitant.
SceneSense is a real-time 3D diffusion model that synthesizes occupancy from partial observations, using just a running occupancy map and a single RGB-D camera. Crucially, it never overwrites observed free or occupied space — generative predictions only fill in the unknown.
The model predicts plausible geometry through occlusions and out of view, and works "out of the box" with any system that produces a running occupancy map — no special conditioning required, with a swappable perception backbone for maximum performance.
This is the foundation of a research line (continued at ICRA 2025 and in Autonomous Robots) that gives exploring robots something like spatial common sense.
ACM Transactions on Human-Robot Interaction (THRI) · 2024
Robots plan by optimization; people predict by pattern recognition. That mismatch makes even competent robot teammates hard to anticipate — a human can't run a solver in their head to guess what the robot will do next.
Human teams coordinate through conventions — shared, predictable ways of acting. This work gives robots pattern-based conventions built on features people can actually observe, tapping into pattern-processing abilities so innate that even preschoolers can extend and abstract them.
A filtering algorithm composes rules over task features into candidate patterns, selecting conventions that are simultaneously useful for the task and easy for a human collaborator to spot and follow.
Across evaluation domains — from map-scale response scenarios to tabletop collaboration with a manipulator — pattern-based conventions made the robot's next move something a teammate could see coming.
Predictability is a form of communication. A robot whose habits make sense doesn't have to explain itself nearly as often.
International Conference on Robotics and Automation (ICRA 2024) · Yokohama, Japan · 2024
Human trust in a robot should track its true competence. But human memory is biased: information encountered last looms largest (recency), followed by what came first (primacy) — the serial-position effect. If a robot's most recent act was a failure, that failure gets outsized weight.
Fifty-three participants watched videos of a robot performing the same tasks at the same overall competence — only the ordering differed: rising competence, falling competence, or failures clustered at the midpoint. They rated the robot between every video and again at the end.
Moment-to-moment ratings stayed stable across conditions — but the recalled, post-experiment ratings skewed with ordering. Same performance, different memories: last impressions disproportionately shaped how competent and trustworthy the robot seemed.
If perception of a robot depends on when it fails, not just how often, robots can plan risk-aware task orderings that cultivate appropriately calibrated trust — and designers can stop being blindsided by it.
ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024) · Boulder, CO · 2024
Nominated for Best Technical Paper
Safe close-proximity collaboration hinges on predicting human intent. But human motion data is inherently noisy and high-variance, which limits even the best learned goal-prediction models exactly where coordination matters most.
Don't just model the mess — reduce it. A robot teammate can deliberately configure the shared workspace before interaction begins, arranging the scene so that human motions toward different goals become naturally more distinguishable. The prediction gains are classifier-agnostic.
An algorithmic approach arranges physical objects and projects "virtual obstacles" through augmented reality, optimizing the workspace for human legibility over a given set of tasks.
In two human-subjects studies — 2D virtual navigation and live tabletop manipulation with a robot arm — workspace optimization with virtual obstacles yielded higher prediction accuracy with less training data. The paper was nominated for HRI 2024's Best Technical Paper.
Environment design is an underused channel of coordination: sometimes the best way to read someone's mind is to set the table so their intentions are easy to see.
33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2024) · Pasadena, CA · 2024
People anchor their expectations of a robot on its body: if it looks like the same robot, we assume it's the same agent. That prior is harmful — visibly identical robots can be running completely different software with completely different capability, safety, and behavior.
Rather than trying to recalibrate trust in "the robot," this work links each of the robot's controllers to its own social identity — a name and a cue the robot performs — so coworkers associate capability with the software currently in charge, not the hardware.
Thirty participants supervised a robot through an order-fulfillment task driven by two controllers of very different reliability — with or without the social identity signal distinguishing them.
Participants who experienced the social identity signal understood that the robot's two controllers were different, and calibrated trust for each one separately — the unreliable controller no longer dragged down perceptions of the reliable one.
Robots increasingly swap software the way people swap hats. Identity performance gives bystanders and teammates an honest handle on what is actually running — preventing both over-trust and unearned distrust.
12th International Conference on Human-Agent Interaction (HAI 2024) · Swansea, UK · 2024
The workforce must continually upskill to work with robotic and autonomous systems, but human training doesn't scale: instructors are scarce, and one-size-fits-all curricula ignore the skills a learner already has. Quadrotor teleoperation makes an ideal testbed — a genuinely hard psychomotor skill with high stakes.
The system assesses each flight automatically against temporal-logic task specifications, then turns that assessment into multimodal formative feedback — grounded in the education literature's principles of effective feedback — that tells the learner what to change on the next attempt.
Control participants received what most training systems provide today — summary statistics of their flight. Study participants received text or multimodal formative feedback generated from the same underlying assessment.
Learners perceived the formative feedback as significantly more actionable than summary statistics — and those in the multimodal condition achieved more safe landings, improving faster across the experiment.
Automated, adaptive coaching makes psychomotor training scalable. The same assess-then-explain loop that improves human-robot teams can teach people to operate the robots in the first place.
Robotics Science and Systems (RSS 2023) · Daegu, Korea · 2023
When an autonomous teammate's recommendation contradicts your expectations — especially when it suddenly changes as new information arrives — you're likely to ignore it. Without insight into the robot's reasoning, even good advice goes unused.
Justification explains why a decision is right, using a snapshot of the agent's internal reasoning. The work characterizes four types of counterfactual justification drawn from explainable AI literature, and uses value-of-information theory to time them strategically — during periods of misaligned expectations, when they matter most.
In a collaborative, partially observable search task, a drone shares a probability map with its human teammate and issues recommendations. The justification framework decides both when to justify and which style of justification to deploy.
A human-subjects study comparing the justification types against each other yielded concrete takeaways on which justifications work when — informing how decision-support agents should explain themselves in the field.
Trust follows understanding. Robots that can defend their important decisions at the right moment get their advice taken — and teams perform accordingly.
International Conference on Robotics and Automation (ICRA 2023) · London, UK · 2023
A person and an autonomous quadcopter share a noisy work floor. Through an AR headset, the floor is painted with a color-keyed grid showing who owns which region; the human can request more space with a pointing gesture, and the robot re-allocates ownership to keep paths safe.
Twenty participants performed collection and assembly tasks in one of two conditions: sharing the floor concurrently with the drone (where the grid's rationale was obvious), or taking turns on the grid (where it wasn't).
The study found consistent evidence of potentially dangerous over-trust: 25% of participants intentionally walked across forbidden floor regions. Participants offered a variety of rationales for their safety violations.
AR safety systems can't assume compliance. The paper distills design considerations that production systems will need — explicitly countering the human tendency to trust the hologram and cut the corner.
International Conference on Autonomous Agents and Multi-agent Systems (AAMAS 2023) · London, UK · 2023
Shopping independently is central to quality of life, and it's particularly challenging for people with visual impairments. Stores carry thousands of products, with roughly 30,000 new ones introduced each year in the US alone — a moving target even for modern computer vision.
ShelfHelp extends an instrumented navigation cane into the shopping domain: a visual product locator finds the desired item on the shelf, and a novel planner autonomously issues verbal guidance commands — modeled as a Markov decision process — to steer the user's hand to it.
The perception pipeline proposes product regions, extracts visual features, and matches them against the requested product — locating one box of cereal among hundreds of look-alikes on a crowded shelf.
In a human-subjects study, novice users successfully retrieved products using two autonomous verbal guidance modes that performed comparably to a human assistance baseline — with positive ratings for competence, intelligence, and ease of use.
ShelfHelp is a step toward assistive technology that supports not just getting to the store, but the fine-grained, dignity-preserving work of shopping itself. Its successor, ShelfAware (RA-L 2026), gives it a robust sense of place.
Proceedings of the 21st International Conference on Autonomous Agents and Multi-agent Systems (AAMAS 2022) · Virtual · 2022
Best Student Paper Runner-up
In human-robot teams, poor communication breeds incongruous mental models, mistrust, and failure. Live communication that builds shared situational awareness is the difference between a team and two agents in the same room.
The work characterizes two complementary modalities of AR visual guidance — prescriptive (showing recommended actions) and descriptive (visualizing state information so people can decide for themselves) — each with generative algorithms to produce them.
A min-entropy multi-agent collaborative planning algorithm for uncertain environments decides what guidance to generate, synchronizing human and robot notions of environmental uncertainty through the AR interface.
A human-subjects study in a collaborative, partially observable search task compared the guidance modalities head-to-head, yielding actionable insights on when to prescribe and when to describe. The paper was AAMAS 2022's Best Student Paper runner-up.
Guidance isn't one-size-fits-all: sometimes a teammate needs a recommended move, and sometimes they need to see what you see. Knowing which — and when — is what makes mixed teams fluent.
Proceedings of the 21st International Conference on Autonomous Agents and Multi-agent Systems (AAMAS 2022) · Virtual · 2022
Navigating dense pedestrian crowds means planning under multi-modal uncertainty — every person might continue, stop, or turn. Popular approaches plan a fixed path first, then let a POMDP solver control only speed along it: safe, but rigid.
This work hands the POMDP planner both speed and heading, letting it weave rather than just brake. That vastly expands the state space it must reason over in real time — which makes finding good roll-out policies within the computational budget the crux of the problem.
The key insight: use multi-query motion planning techniques — Probabilistic Roadmaps and the Fast Marching Method — as priors that can rapidly produce an efficient roll-out policy for any state the POMDP tree might reach during its limited-horizon search.
The hybrid planner generated trajectories that were safe and significantly more efficient than path-then-speed baselines, across crowd densities of up to hundreds of pedestrians.
From delivery robots to assistive devices, real deployments happen in crowds. Planners that reason about human intent with full freedom of motion get there safely — and noticeably faster.
IEEE International Conference on Intelligent Robots and Systems (IROS 2022) · Kyoto, Japan · 2022
Choosing where to sit is a quietly social act — we weigh privacy, intimacy, and convenience in seconds. For blind and visually impaired people, a sight-centric society makes participating in these dynamics difficult, eroding independence in public spaces.
A white cane instrumented with a camera and IMU performs SLAM and object localization, scores candidate seats using insights from psychology — optimizing for convenience, privacy, and intimacy — and plans a safe path to the chosen seat.
A novel two-motor haptic feedback system built into the cane's grip conveys navigation guidance through touch, leaving the user's hearing — a critical sense for navigating without vision — free.
In a pilot study, the system successfully classified socially preferred seats and guided users to them safely, increasing participants' confidence in independent navigation in unknown indoor spaces.
Assistive robotics is at its best when it restores not just mobility but agency — including the small, social choices most people never have to think about.
IEEE Robotics and Automation Letters (RA-L) · 2022
Robot manipulation research is dominated by grasping. But grasps fail — the object is too big, too flat, too far away, or the gripper simply misses. Humans respond by poking, pushing, and nudging; most robots just stop.
PokeRRT is a motion planner that treats poking as a skill in its own right: fast, effective at moving objects across the workspace, and able to reach places a gripper can't — expanding the robot's effective reachable workspace.
In simulation and real-world experiments, PokeRRT planned object trajectories through cluttered and uncluttered scenes, and — used synergistically with pick-and-place — recovered from failed grasps, beating pushing and grasping baselines in success rate and task time.
Resilient manipulation isn't about never failing — it's about having another move when you do. Non-prehensile skills like poking are that move.
Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2021) · Virtual · 2021
Collaborative robots learn skills that then meet the real world: new environments, new task variations, and human teammates with their own preferences. The resulting failures and discomforts usually require an engineer to fix — not viable for robots meant to work alongside novices.
PARSEC (Plan Augmentation and Repair through SEmantic Constraints) lets novice users repair and adapt robot behavior by describing what's wrong in natural language, which a semantic hierarchy translates into candidate motion constraints.
When a failure is detected, the robot asks the user targeted questions, processes the feedback against its semantic hierarchy, and ranks candidate constraints — converging quickly on the correction that matches the user's intent.
A case study showed PARSEC efficiently finding corrective constraints matching user intent — giving novices the power of constrained motion planning combined with human-in-the-loop training, no programming required.
The gap between "robot that was trained" and "robot that keeps working" is maintenance. Making repair conversational closes that gap for everyone, not just engineers.
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2021) · Xi'an, China (+Virtual) · 2021
Learning from Demonstration lets novices teach robots new skills, but most LfD methods offer no way to maintain them. When the task requirements or the environment shift, the skill quietly breaks — and the only fix is to re-demonstrate everything.
ARC-LfD renders the robot's learned skill in augmented reality, right in the workspace — keyframes, trajectories, and constraints as holograms you can inspect. What the robot learned stops being a black box and becomes something you can see.
Users edit skills by adding and adjusting constraints — height limits, orientation zones, obstacle regions — directly through the AR interface. The constrained-LfD backend regenerates compliant behavior without requiring a single new demonstration.
Three case studies demonstrated skills being adapted to environmental changes and altered task requirements after initial teaching — maintenance, not re-teaching.
Robot skills, like software, spend most of their life in maintenance. ARC-LfD made that lifecycle visible and editable — a thread that continues through much of CAIRO's later LfD and AR work.
Springer-Nature Current Robotics Reports · 2020
As robots become more prevalent and capable, the roles we hand them — and our expectations of them — grow in kind. Operating safely in human-populated environments means cooperating with human teammates, and mental models are the formal mechanism that makes fluent teamwork possible: awareness between teammates enabling coordinated action.
The paper provides a structured overview of mental model theory and methodology as applied to human-robot teaming, along with evaluation methods and metrics for measuring the different facets of mental modeling during interaction.
Mental model alignment is the connective tissue of CAIRO's research program — the survey frames the questions that RARE, SPEAR, autonomous justification, and the lab's AR guidance work each go on to answer.
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2019) · Montreal, Canada · 2019
A robot working beside a person needs to understand what activity that person is doing — and it can't wait until the motion is finished. Recognizing activities early, from partial trajectories, is what lets a robot plan safe and efficient actions of its own.
The algorithm models transitions through activities as a hidden Markov model, inferred online with an efficient particle filter to estimate the most likely activity sequence, and refines segment boundaries with a search over the task's known partial ordering of activities.
Across three human activity datasets, the approach improved segmentation accuracy over two state-of-the-art baselines by 11 to 65 percent, and could recognize an activity from just the first third of its trajectory, on average.
The algorithm ran on an industrial mobile robot during an automotive assembly task, tracking a worker's progress and delivering the correct materials at the right time — activity recognition in service of genuinely useful assistance.
2019 ACM/IEEE International Conference on Human Robot Interaction (HRI 2019) · Daegu, Korea · 2019
Best Technical Paper Runner-up
Teams run on shared expectations. When a human's understanding of a task diverges from reality — a mislearned rule, a missed consequence — failures can be catastrophic unless someone notices and intervenes.
RARE (Reward Augmentation and Repair through Explanation) lets a robot detect model disparity between itself and a collaborator, infer where the disagreement lives, weigh its consequences — and then offer human-interpretable feedback that patches the human's reward function.
Watching its partner's choices during a collaborative game, the robot estimates their task understanding from behavior alone, and decides between staying quiet, interrupting, or interrupting with a justification for the correction.
In a human-subjects study on a complex collaborative task, explanation-based policy repair worked — and interruptions that came with justifications were rated significantly more helpful and intelligent. The paper was HRI 2019's Best Technical Paper runner-up.
RARE is the seed of CAIRO's explainable-AI research line: its reward-repair idea grows directly into SPEAR's single-shot semantic coaching and the lab's later work on justification.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018) · Madrid, Spain · 2018
Learning from demonstration lets novices teach robots new skills — but novices make mistakes. Sub-optimal demonstrations quietly degrade the learned skill, and finding and removing the bad examples is tedious at best.
Instead of curating demonstrations, annotate them. CC-LfD (Concept Constrained Learning from Demonstration) lets a teacher attach conceptually-grounded constraints — planning predicates like "keep the cup upright" — to live demonstrations, baking the intent into the learning process itself.
Constraints are incorporated directly into the LfD pipeline, so the learned skill satisfies them even when the underlying demonstrations don't. The same mechanism repairs existing skills: a single annotated demonstration can fix a broken behavior.
CC-LfD repaired skills without identifying or removing low-quality demonstrations, and showed promise for transfer learning — using constraints to adapt demonstrations from a related task with few new examples.
This is the origin of a decade-long research thread: CC-LfD's constraints become inspectable holograms in ARC-LfD, conversational repairs in PARSEC, and demonstration-guided manifold planning at HRI 2025.
International Journal of Robotics Research (IJRR) · 2018
On a labor and delivery floor, a resource nurse makes rapid, high-stakes resource decisions all day. Before intelligent hospital robots can take initiative in coordinating patient care, one question has to be answered: will clinicians trust their suggestions appropriately?
The decision support was trained through action-driven learning from expert demonstrations, and delivered to nurses and physicians as recommendations — sometimes from a computer, sometimes from an embodied robot, and at varying levels of quality.
Clinicians accepted high-quality recommendations at a 90% compliance rate, with error rates comparable between robotic and computer-based support. Notably, embodiment helped: when recommendation quality shifted mid-study, people calibrated their reliance on the robot more appropriately.
The work culminated in a pilot demonstration of a robot assisting real resource nurses at a tertiary care center — early evidence that embodied decision support can safely earn a place in high-stakes human teams.
IEEE International Conference on Robotics and Automation (ICRA 2017) · Singapore · 2017
In human-robot collaboration, quickly and reliably reading a teammate's actions is critical to team fluency. But the best-performing recognition models are usually black boxes — when they fail, non-experts have no way to understand or improve them.
RAPTOR (Rapid Activity Prediction Through Object-oriented Regression) is a scalable method for real-time activity recognition and prediction, designed from the ground up to be human-interpretable, with defenses against temporal variation and against learning false correlations between features.
RAPTOR achieved state-of-the-art accuracy on a generic human activity dataset and two collaborative manufacturing datasets — from both full and partial trajectories — while using outlier detection to explain its own reasoning and flag anomalies.
Recognition you can interrogate is recognition you can trust on a factory floor. RAPTOR paired anticipation (for fluent teamwork) with explanation (for accountable failures) years before "explainable AI" became a catchphrase.
ACM/IEEE International Conference on Human-Robot Interaction (HRI 2017) · Vienna, Austria · 2017
Humans and robots don't share a language for intentions, plans, or justifications. Even a co-worker who can read the robot's control code — especially a learned, statistical policy — has little hope of extracting meaningful insight about how it will behave.
This work gives robots the ability to autonomously synthesize descriptions of their own policies, and to answer both general and targeted queries from collaborators — mapping questions onto the policy, finding the relevant states, and composing a concise natural-language answer.
The algorithms operate on the robot's controller regardless of its form — conditional logic, tabular reinforcement learning, or deep reinforcement learning — synthesizing informative descriptions and enabling fault diagnosis by non-experts.
This paper is a foundation of the lab's transparency agenda: if a robot can explain what it does and when, then trust, debugging, and collaboration all get easier. Its questions echo through RARE, justification, and SPEAR.