Skip to main content
Avala’s multi-sensor viewer supports recordings with any number of camera topics. When you upload an MCAP file containing multiple image streams, the viewer arranges them in a synchronized grid and — when calibration data is available — projects LiDAR point clouds onto each camera view for cross-sensor verification.

​
Uploading Multi-Camera Recordings

1

Upload your MCAP file

Navigate to your dataset in Avala and upload the MCAP file containing your camera topics. The platform parses the recording and extracts all available topics.
2

Select camera topics

After parsing, Avala presents a topic selection screen listing every detected topic. Choose the camera streams you want to visualize. You can also select LiDAR, IMU, and other topics at this stage.
3

Viewer builds the layout

The viewer auto-detects image topics and arranges them in a smart grid. Non-image topics (LiDAR, plots, gauges) are placed in separate panels alongside the camera grid.
You do not need to select every camera topic. If your recording contains 8 surround cameras but you only need front and rear, select just those two. This reduces viewer load and keeps the layout focused.

​
Smart Grid Layout

The viewer arranges camera panels based on how many image topics are active:
Camera CountLayoutDescription
1Full widthSingle camera fills the center area
2Side-by-sideTwo cameras at equal width
32 + 1Two panels on top, one spanning the bottom row
4+Grid2-column grid, rows added as needed
Panels can be resized by dragging the dividers between them. The layout persists while you navigate the timeline.

​
Calibration and Transforms

For LiDAR-to-camera projection to work, the viewer needs two pieces of information: where each camera is in 3D space (extrinsics) and how each camera forms images (intrinsics).

​
Extrinsics from TF Messages

Avala reads coordinate frame transforms from tf2_msgs/TFMessage and foxglove.FrameTransform messages in your recording. These provide the rigid-body transforms (rotation + translation) between sensor frames. The viewer resolves the full transform chain from the LiDAR frame to each camera frame automatically. For example, if your recording contains: 
base_link → lidar_top
base_link → camera_front
base_link → camera_left
Avala computes the LiDAR-to-camera transform by chaining lidar_top -> base_link -> camera_front.
Include both /tf (dynamic transforms) and /tf_static (fixed transforms) topics in your recording. Static transforms are typically published once at the start of the recording and define the fixed mounting positions of sensors on the vehicle.

​
Camera Intrinsics

Avala supports two camera models for projection:
ModelParametersUse Case
Pinholefx, fy, cx, cy + distortion coefficients k1-k4, p1, p2Standard cameras with rectilinear lenses
Double-spherefx, fy, cx, cy, xi, alphaWide-angle and fisheye lenses
Camera intrinsics can be provided through:
  • CameraInfo messages — sensor_msgs/CameraInfo topics published alongside image topics
  • Embedded calibration — Calibration data stored in the MCAP file metadata

​
Projection Behavior

When calibration data is present, the viewer can project LiDAR points onto camera images. Each projected point is colored by its depth, intensity, or label — matching the active visualization mode in the 3D panel. This cross-view projection is useful for:
  • Verifying that 3D cuboid annotations align with objects in camera views
  • Checking sensor calibration accuracy
  • Understanding the spatial relationship between LiDAR returns and visual features
If calibration data is missing or incorrect, LiDAR projection will be misaligned or unavailable. Always verify your calibration before relying on projected overlays for annotation quality checks.

​
Visualization Modes in Camera Panels

When LiDAR projection is active, the projected points inherit the color scheme from the current point cloud visualization mode:
ModeProjected Point Color
NeutralUniform color
IntensityLiDAR return intensity (blue→green→yellow→red gradient)
RainbowCycling hue per frame
LabelSemantic label color from the annotation class
PanopticInstance-level color per annotated object
Image ProjectionCamera pixel color back-projected onto LiDAR (3D panel only)
Switch modes in the 3D panel toolbar and the projected points on camera panels update automatically.

​
Independent Panel Controls

Each camera panel supports independent interaction while maintaining timeline synchronization:
  • Zoom — Scroll to zoom into a specific region of the camera image
  • Pan — Click and drag to pan across zoomed images
  • Reset — Press 0 to reset zoom and pan to the default view
Timeline position is always shared. Zooming into one camera does not affect other panels.

​
Best Practices for Multi-Camera Recordings

Include TF topics

Always record /tf and /tf_static topics. Without transforms, the viewer cannot resolve coordinate frames between sensors.

Publish CameraInfo

Publish sensor_msgs/CameraInfo alongside each image topic. This provides the intrinsics needed for accurate projection.

Use consistent frame IDs

Ensure each sensor topic references the correct frame ID in its message header. Mismatched frame IDs break the transform chain.

Compress images

Use sensor_msgs/CompressedImage with JPEG compression to reduce file sizes. Uncompressed images dramatically increase MCAP file size.

​
Next Steps

Recording Best Practices

Tips for recording data that works well in Avala, including format, compression, and naming conventions.

Rendering Modes

Deep dive into the 6 point cloud visualization modes and when to use each one.

Timeline Navigation

Playback controls, frame stepping, and timestamp seeking across all panels.

MCAP & ROS Overview

Supported formats, message types, and the upload workflow for multi-sensor recordings.