Visualizing with the 3D scene

The 3D SCENE tab renders your machine in an interactive 3D view: the frames of every component, the geometry attached to them, point clouds from depth cameras, and any custom visuals a module publishes while the machine runs. It turns configuration you would otherwise read as JSON numbers into a picture you can inspect, so you can confirm a gripper sits where its frame configuration places it or watch a motion plan against the obstacles around it. In Build and Move modes, the scene does more than show your machine: you can edit its configuration and command it to move.

This page covers the scene’s modes, where to start for common tasks, and how the scene gets what it draws.

Modes

The scene has four modes. The mode determines what you can do and which tools appear.

In every mode, the World panel lists every entity in the scene, and selecting one shows its details. 3D scene widgets let you drive components and read their values without leaving the scene. For navigation, shortcuts, and settings, see 3D scene controls and settings.

Start from what you want to do

• Set up a machine
  • Add frames, obstacles, and collision geometry without editing JSON: Build mode
  • Enter measured offsets for a camera or other component: Editing frames visually
  • Check that the scene matches your real machine: Measuring between frames
• Watch and operate a machine
  • Find where an end effector is right now: Monitor mode
  • Drive a component or watch its readings without leaving the scene: 3D scene widgets
• Move an arm and debug motion
  • Move an arm to a pose without writing code, and preview the move first: Move mode
  • Replay a saved plan and see the obstacles it was planned against: Replay mode
  • Find out why a plan failed or took a surprising route: Inspect IK
  • Understand why the planner ignores something you can see: Visuals and collisions
• Check cameras and perception
  • Check that a depth camera’s point cloud lines up with the world: Verify point cloud alignment
  • Tune a 3D segmenter against the live view: Vision services in the 3D scene
  • See what a camera or end effector sees from its mount: Frame POV
• Draw your own visuals
  • Draw your module’s visuals in the scene: Publish visuals from a module
  • Preview spatial data from a Go script without deploying: Viam Visualization

Something look wrong? See Troubleshoot the 3D scene.

What the scene simulates, and what it doesn’t

The scene works with kinematics, not physics. It shows where things are and where a plan would take them, and checks for collisions against the geometry you’ve modeled. It doesn’t model dynamics, timing, or anything in the workspace that isn’t modeled.

To try motion without risking hardware:

• Preview a move in Move mode. The preview plays the planned motion before anything moves. Running the plan, with Execute plan or Move, moves the machine; see the caution on the Move mode page.
• Simulated and fake components report poses and accept commands without moving anything physical. With a fake arm, the whole scene works without hardware. See Try it with a fake arm.

What the scene renders

The scene draws four kinds of element, each with its own appearance:

• Component frames render as sets of red, green, and blue coordinate axes (X, Y, and Z), one per component, positioned by each frame’s translation and orientation.
• Geometries render as translucent shapes (a box, sphere, or capsule) at the frame they are attached to.
• Point clouds from depth cameras render as colored point sets.
• Custom visuals render as whatever a module draws them as: markers, arrows, lines, meshes, or extra geometries, published while the machine runs.

Where each element comes from

Every element of your machine’s state in the scene traces back to one source. Select an entity in the World panel to follow it back to the component or module that produced it.

Frames and geometry appear because they are part of the machine’s configuration, which the scene reads whenever you open the tab. Point clouds stream from depth cameras while the machine is online, and custom visuals appear only while a module publishes them.

The scene also draws things that aren’t your machine’s reported state: a Move mode preview, a replayed plan and its obstacles, and Build mode edits you haven’t saved yet.

Built-in content versus custom visuals

The distinction between the sources matters because it tells you where to make a change:

• Built-in content is the frame system and configured geometry. The scene shows it with no code. To change a frame’s position or a geometry’s shape, edit the component’s configuration.
• Custom visuals are everything a module computes or senses at runtime: a detection, a planned path, a sensor’s obstacle readings. The scene shows them only while the module publishes them. To change a custom visual, change the module’s code.

If an element looks wrong, this split tells you where to look: a misplaced frame or geometry is a configuration problem, while a missing or stale custom visual is a module problem.

How the scene stays current

Built-in content reflects your configuration and, when the machine is online, each component’s live pose. Custom visuals stay current a different way: a module publishes them to a world state store service, and the scene subscribes to that service’s stream of transform changes. As the module adds, updates, and removes transforms, the scene applies each change to the one affected visual instead of redrawing everything, so a busy scene keeps up as the underlying data changes.

To publish your own custom visuals this way, see Publish visuals from a module.