Human supervisory control

Humans supervising semi-autonomous machines: intervene, take over, and set constraints—teleoperation and shared control included. DeepMind’s Gemini Robotics materials stress steerability and collaboration with people; Gemini Robotics-ER is described as available to trusted testers (named robot companies on the blog) rather than unconstrained public autonomy. This course teaches literacy—not remote-weapon operation.

Most near-term embodied AI is supervised. Interface latency and attention limits are safety-critical. Safety cases must name the human role.

Clear modes (auto/assist/manual) → attention cues → reliable takeover → logs. Avoid mode confusion. Isaac Lab mentions game-pads/keyboards for collecting demonstrations—humans in the data loop as well as the control loop.

A warehouse operator supervising multiple AMRs with a deadman switch and clear “who is driving?” indicators.

Sketch a takeover UI for a warehouse robot with three modes labeled and a maximum allowed takeover latency.

⚠️ Myth: A supervisor can watch 50 robots casually.

✓ Reality: Attention budgets force careful ops design.

⚠️ Myth: Natural-language commands remove the need for supervisors.

✓ Reality: Language makes interaction easier (Gemini Robotics interactivity claims) but does not remove responsibility or low-level safeties.

• Gemini Robotics (interactivity, trusted testers, safety layers): https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/ ↗
• Isaac Lab overview (demo devices): https://docs.isaacsim.omniverse.nvidia.com/latest/isaac_lab_tutorials/index.html ↗
• MIT Manipulation (human-facing kitchen/home task framing): https://manipulation.mit.edu/Fall2025/index.html ↗