Spatial semantics inference: the perception a machine needs to be safe around people
Most physical-AI safety failures begin as perception failures. This work studies a safety perception layer we propose an autonomous machine needs to handle safety-critical edge cases, by completing occluded 3D geometry and inferring physics from partial sensor input.
Nearly every safety failure in a physical-AI system traces back to a perception failure: the machine misreads the scene, and every decision built on that read inherits the error. Today's depth sensors make this worse in ways biological vision does not. They lose transparent and reflective surfaces, drop dark objects, and place the geometry they do capture centimeters off, and centimeters are the difference between a safe motion and a collision with a person.
Human vision does not work this way. The brain completes 3D structure from partial, occluded input using learned priors, then reasons about the physics of what it has recovered. This work studies spatial semantics inference, a perception layer that does the same: it infers complete geometry from partial, noisy sensor input through amodal completion, then attaches physical properties such as material, mass, and support.
The goal is a perception layer trustworthy enough to be a safety foundation: a machine that knows what is in a scene, where it is, and how it will behave, precisely enough to act safely around people. A safety case is only as sound as the perception it rests on.
What you're reading is the abstract.