Sense, plan, act
The classic robotics loop: **sense** the world, **plan** a next move, **act** with actuators, then sense again—under safety constraints. MIT Manipulation’s course description organizes topics as perception, planning (kinematics, collisio...
What it is
The classic robotics loop: sense the world, plan a next move, act with actuators, then sense again—under safety constraints. MIT Manipulation’s course description organizes topics as perception, planning (kinematics, collision-free motion, task-and-motion planning, uncertainty), and dynamics/control (model-based and learning-based). Modern Robotics separates motion planning (Ch. 10) from robot control (Ch. 11).
Visual Spec & Architecture Diagram
Sense–Plan–Act cycle triangle/loop with icons: cameras/lidar (Sense), planner graph (Plan), motors (Act). Timing note 'classic deliberative loop; modern systems blur layers'.
Why it matters
Embodied AI fails differently than chat: physics, latency, and irreversible actions dominate. Gemini Robotics highlights interactivity—replanning when objects slip or move—as a required quality, not a nice-to-have.
How it works (plain)
Sensors → state estimate → planner/policy → controller → motors. Humans supervise high risk. Simulation helps; reality remains the exam. Software frameworks (ROS ecosystem; Isaac Lab for learning loops) structure how modules talk.
Everyday example
Crossing a street: look, decide, step, look again—not a single frozen plan.
Try it
For a robot vacuum, list sensors, plans, acts, and one required human override.
Myths
- ⚠️ Myth: A smart VLM replaces the control stack.
- ✓ Reality: DeepMind’s Gemini Robotics-ER story still interfaces with low-level safety-critical controllers per embodiment.
- ⚠️ Myth: Slow planning is always safer.
- ✓ Reality: Too-slow loops miss dynamics—design for latency budgets (supervisory-control chapter).
Sources
- MIT Manipulation: https://manipulation.mit.edu/Fall2025/index.html ↗
- Modern Robotics: https://hades.mech.northwestern.edu/index.php/Modern_Robotics ↗
- Gemini Robotics: https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/ ↗
- ROS docs hub: https://docs.ros.org/index.html ↗
