PAC-MAN: Perception-Aware Control for Safe Humanoid Robots in Dodgeball

A dodgeball demo that is really about safety

The eye-catching part of PAC-MAN is easy to describe: a humanoid robot sees an incoming ball and dodges it. The more important part is harder, and it explains why the work has started circulating around IROS 2026 side events and robotics research posts this week. PAC-MAN is not only a robot stunt; it is an attempt to make fast whole-body reactions depend on the same imperfect perception that a deployed robot actually has [1].

At the Robotics Research Night scheduled in Pittsburgh on Monday, September 28, PAC-MAN appeared as a lightning talk by Lizhi “Gary” Yang of Caltech, framed as a project that combines onboard perception, reinforcement learning and control barrier functions to teach humanoids to dodge incoming objects while staying balanced [1]. That framing matters because modern humanoid control often looks strongest in simulation or under privileged sensing. PAC-MAN’s central claim is narrower and more useful: the policy should learn evasive behavior while seeing the threat through a compact, segmentation-masked depth representation from a head-mounted RGB-D camera [1].

The current public discussion around the project also emphasizes the reported real-hardware result: a Unitree G1 dodged 19 of 20 hand throws, or 95%, with zero falls, in a zero-shot deployment from simulation [2]. In other words, the policy was not fine-tuned on the physical robot before the reported hardware run. That is a crucial distinction. If a robot can only dodge after repeated real-world adjustment, the method is a training recipe for one lab setup. If it transfers directly, it is closer to a deployable control pattern.

What PAC-MAN combines

The framework brings together three ingredients that are often studied separately: reinforcement learning, control barrier functions and perception. In PAC-MAN, reinforcement learning supplies the policy that maps proprioception and depth-derived visual observations to evasive whole-body actions. Control barrier functions, or CBFs, supply the safety structure: they encode margins between the incoming ball and the robot’s body links rather than simply rewarding the robot for moving its torso away [1].

This distinction between torso safety and whole-body safety is central. A humanoid can “dodge” with its pelvis while still allowing a hand, elbow, shin or shoulder to be hit. PAC-MAN’s public descriptions stress that the method protects every body link, not just the center of mass or pelvis [2]. That choice makes the task more realistic and more demanding, because the controller must reason about the entire articulated body under time pressure.

The third ingredient is perception. The deployed robot does not receive the exact ball state as a privileged variable. Instead, it relies on depth information from a head-mounted RGB-D camera, after a segmentation model isolates the ball and compresses the observation into a compact ball-only depth image [1]. In one current technical summary, the representation is described as an EfficientTAM-segmented ZED depth stream pooled to a 16-by-9 grid [2]. The point is not that 16 by 9 pixels are visually rich. The point is that a small, task-focused representation may be enough when training aligns perception and control from the beginning.

Why the control-barrier choice matters

CBFs are a standard tool for safety-critical control because they describe a boundary around unsafe states. A common use is a runtime safety filter: the learned or nominal controller proposes an action, and a mathematical supervisor modifies it if the action would violate safety. PAC-MAN uses that idea differently in its deployable form. The public summaries this week describe the main hardware version as using CBF information for reward shaping during training rather than as an online filter during deployment [2].

That shift is subtle but important. A runtime filter can provide stronger formal enforcement, but it needs sufficiently accurate state estimates at the moment of action. In a dodgeball task, the robot’s perception of the ball is partial, delayed and sometimes occluded. If the safety filter depends on ground-truth ball position and velocity, it may work as a simulation ceiling but not as a deployable onboard system. Current discussion around PAC-MAN explicitly identifies perception, not control theory alone, as the binding constraint on provable safety [2].

PAC-MAN therefore distinguishes between a lighter deployable “Link-CBF” strategy and a stronger “Joint-CBF” strategy. The Link-CBF version uses one clearance barrier per body link and puts that information into the training reward, teaching the policy to avoid contacts without requiring online enforcement [2]. The Joint-CBF version can impose a stronger joint-space constraint, but public summaries describe it as depending on privileged ball state and therefore serving mainly as a simulation-only reference or upper bound [2].

The IROS-week development

The fresh development is not a new paper version; it is the subject’s move into live community discussion around IROS 2026. The Pittsburgh Robotics Research Night page lists PAC-MAN as the second lightning talk and describes it as work on fast, whole-body robot reactions where perception and safety must be designed together [1]. The event page also presents the broader motivation: reinforcement-learning policies can perform impressively yet behave unsafely under disturbances or imperfect observations, which is exactly the gap PAC-MAN tries to narrow [1].

A LinkedIn post from Junfan Zhu, one of the event hosts, similarly previewed the September 28 session by listing PAC-MAN as a talk on perception-aware reinforcement learning plus control barrier functions for whole-body humanoid safety [3]. That post repeated the hardware headline: Unitree G1, 19 successful dodges out of 20 and zero falls in real-world experiments [3]. The repetition across the event listing and social preview is useful because it shows what the robotics community is treating as the salient message: not merely that a humanoid dodged a ball, but that it did so with onboard perception and without falls.

A later technical post by Léo Morillon framed the result for a broader robotics audience by unpacking the vocabulary around CBFs, runtime filters, reward shaping, privileged state and zero-shot sim-to-real transfer [2]. That post is especially clear on the trade-off PAC-MAN exposes: formal safety tools become much harder to deploy when their assumptions about state information do not match the robot’s real sensors [2].

Perception-aware safety, not just safer perception

PAC-MAN’s most interesting idea is that safety and perception cannot be bolted together at the end. A robot that trains with perfect ball coordinates and deploys with a noisy camera may learn a policy that depends on information it no longer has. Conversely, a robot trained from the start on segmentation-masked depth can learn what is observable from that representation and what is not.

The event description says PAC-MAN uses a segmentation model to isolate the incoming ball from the depth image, creating a compact perception representation close to the training setup [1]. This design aims to reduce the sim-to-real gap: the controller does not need to interpret a full, messy visual scene if the perception stack can deliver the one object channel relevant to the task. That also explains the “PAC-MAN” label beyond the acronym-like title. The system is less about general vision and more about perception focused on the immediate threat.

This does not make the problem trivial. A ball can approach from a direction that leaves the camera’s field of view. A humanoid’s own motion can change the view. The correct dodge may require ducking, leaning, stepping or jumping without falling. Public descriptions of the work note that the framework uses an adversarial human-motion prior to encourage natural evasive behaviors such as ducking, sidestepping, leaning and jumping [1]. That prior does not replace the safety objective; it regularizes how the robot moves while satisfying the task.

What the 95% result does, and does not, prove

The 19-of-20 figure is impressive, especially because it is reported for real hardware with onboard perception and no policy fine-tuning [2]. Still, it should be read carefully. It does not prove that humanoid robots are now safe around arbitrary fast objects in arbitrary environments. It shows that, in the reported dodgeball setup, the combination of segmentation-masked depth, CBF-shaped reinforcement learning and motion-prior regularization can transfer to a Unitree G1 well enough to avoid most hand-thrown balls without falling [2].

That is a narrower result, but a meaningful one. Robotics progresses when narrow demonstrations reveal reusable structure. Here, the reusable structure is the explicit coupling of what the robot can perceive with what the safety mechanism assumes. PAC-MAN’s comparison between deployable Link-CBF reward shaping and stronger but less deployable Joint-CBF filtering is valuable precisely because it refuses to hide the difference between simulation knowledge and onboard sensing [2].

It also suggests a practical design rule for humanoid learning systems: safety terms should be evaluated not only by how strong they are mathematically, but by whether the information they require is available at deployment. A beautiful safety constraint that needs ground-truth object velocity may be a benchmark ceiling rather than a product feature. A softer training signal that works with actual camera-derived observations may be less formal but more useful.

Why this matters beyond dodgeball

Dodgeball is a laboratory task, but the underlying problem appears across humanoid robotics. Service robots, warehouse robots and home robots will face moving people, tools, doors, pets and unexpected objects. A humanoid that can react with its whole body while maintaining balance is addressing a different safety problem from a wheeled robot that merely stops.

The September 28 event listing situates PAC-MAN among broader Physical AI discussions, including manipulation, simulation-to-real transfer, robot learning and embodied foundation models [1]. That context is important: the field is increasingly asking whether impressive learned behaviors can be trusted when perception is imperfect and the world moves quickly. PAC-MAN’s answer is not to abandon learning, but to give learning a safety-aware structure that matches the robot’s sensor reality.

For now, PAC-MAN should be understood as a focused research result: a perception-aware CBF-RL framework for whole-body humanoid dodgeball, demonstrated on a Unitree G1 and highlighted during IROS-week robotics discussions. Its broader significance is the lesson behind the dodge: safe humanoid control cannot be separated from what the robot can actually see.

Sources from the last 72 hours

1. [1]🍾 IROS 2026 x Saturday Robotics x FAIR Plus — Robotics Research Night | Reading Club 31. Pittsburgh 9/28Sep 28, 2026, 11:30 PM
2. [2]Léo Morillon - NEW RESEARCH: Humanoids are getting really good at dodgeball!Sep 29, 2026, 6:00 PM
3. [3]Junfan Zhu - #iros2026 #robotics #physicalai #robotlearning #worldmodels #humanoidrobotics #vla #embodiedai #robotfoundationmodelsSep 29, 2026, 2:00 AM

AI-generated article based on recent web research, then preserved as a dated editorial snapshot.