Research engineer

Project

VR Human-Robot Search Team Testbed

A Unity and Meta XR simulation for studying embodied trust calibration as participants work with robot teammates to search rooms, gather evidence, and identify a target location.

View visuals Back to projects

Overview

A repeatable VR study environment for human-robot teaming.

The testbed places participants in a virtual multi-building search task where they coordinate with robot partners, inspect room evidence, update rationales, and make a final location decision.

The system combines Unity VR interaction, Meta XR deployment, Python WebSocket logging, an AWS bridge, speech interfaces, gaze/eye tracking, decision timing, robot recommendations, and trust survey workflows.

Environment

Virtual search space and robot partners

Exterior view of the VR search environment with two buildings and robot partners
Virtual search environment with two buildings, task entry point, and robot teammates.
Interior VR clue room with table, map, documents, and furniture
Interior clue room where participants inspect evidence during the search task.
Two robot teammate models used in the VR simulation
Robot partner models used in the study environment.
VR map clue showing candidate city locations
Evidence artifact viewed inside the environment.

Workflow

Planning, evidence review, and final decision

Planning interface for selecting rooms to search
Planning phase for assigning rooms and recording search rationale.
Instructions screen for the VR search task
Participant instructions at the start of the task flow.
Solo evidence review interface in the VR simulation
Solo review screen for participant evidence and rationale updates.
Joint meeting interface showing participant clues, robot clues, partner response, and confidence
Joint meeting view where participant evidence, robot clues, partner feedback, and confidence estimates are compared.
Final location choice interface in the VR simulation
Final choice workflow for selecting the target location and recording rationale.

System

What the testbed captures

Stack

Unity, Meta XR, Python, and AWS

Built for internet-connected headset deployment with backend WebSocket logging and cloud-connected study data capture.

Signals

Interaction and trust data

Captures decisions, timing, rationales, recommendation exposure, speech interactions, gaze/eye tracking, and trust survey responses.

Study

Embodied verification

Supports an in-progress manuscript on how participants move from monitoring robot recommendations to relying on them in an embodied task.

Measures

Survey and workload collection

Clue submission and confidence interface
Clue submission interface with confidence and rationale capture.
Trust survey screen in the VR study workflow
Trust survey workflow embedded in the study experience.
NASA TLX-style workload survey screen
Task-load survey for workload measurement.
Qualitative survey prompt in the VR study workflow
Qualitative survey prompts for post-task feedback.

Additional Views

Evidence and navigation details

Document clue artifact inside the VR environment
Document clue viewed during the search workflow.
Participant perspective while navigating a VR room
Participant perspective during room navigation.
Robot partner view in the VR simulation
Robot partner view from inside the virtual environment.
First-person robot view in the VR simulation
First-person robot view used during testing.
Early interface screenshot from the VR search testbed
Early interface iteration used while developing the study flow.

Links

Back to home Resume Email