Semantic Segmentation Tool

The Segmentation tool creates pixel-level masks for semantic or instance segmentation. It includes four sub-tools: Brush, Eraser, Fill, and SAM Assist.

​

When to Use

Use segmentation when:

• You need pixel-precise object boundaries
• The task requires semantic segmentation (every pixel labeled) or instance segmentation (separate mask per object)
• Objects have complex, irregular shapes that polygons cannot capture efficiently
• You are training models like DeepLab, U-Net, Mask R-CNN, or similar architectures

Consider a different tool when:

• You only need rough localization — use the Bounding Box tool
• Object boundaries are simple enough for polygon outlines — use the Polygon tool (faster for simple shapes)
• You need 3D annotations — use the 3D Cuboid tool

​

Usage

1. Press M or select the Segmentation tool from the toolbar
2. Choose a label from the label panel
3. Use one of the sub-tools:

​

Brush

• Click and drag to paint a mask region
• Adjust brush size with [ (smaller) and ] (larger)
• Hold Shift to paint in a straight line between two clicks

​

Eraser

• Press X to toggle eraser mode
• Click and drag to erase mask pixels
• Same size controls as the brush

​

Fill

• Click inside an enclosed region to flood-fill it with the current label
• Useful for filling the interior of a polygon outline quickly

​

SAM Assist

• Press S to activate SAM mode within the segmentation tool
• Click on the object to generate a mask automatically
• Refine with positive clicks (include region) and negative clicks (exclude region)
• Press Enter to apply the mask

​

Shortcuts

​

Recommended Workflow

The fastest approach for most objects is:

1. Start with SAM: Click on the object to get an initial mask
2. Refine with positive/negative clicks: Add or remove regions as needed
3. Apply the mask: Press Enter
4. Clean up edges: Switch to the brush or eraser at a small size for precise edge cleanup
5. Zoom in: Use + or scroll to zoom into boundary areas for fine detail work

​

Common Mistakes

• Jagged edges from large brush size: Use a smaller brush near object boundaries for cleaner results
• Missing thin features: Thin parts of objects (legs, antennae, poles) are easy to miss — zoom in to verify
• Overlapping labels: When two labels overlap, the last painted label takes priority — be intentional about overlap order
• Not using SAM: Manually painting an entire mask from scratch is much slower than starting with SAM and refining

​

Advanced Tips

• Start with SAM for the initial mask, then refine edges with the brush and eraser
• Use the eraser at a small size for precise edge cleanup
• Zoom in (+ or scroll) for fine detail work along object boundaries
• Each label gets a distinct color overlay, configurable in the label panel
• Toggle mask visibility with H to see the underlying image and verify boundaries
• For batch labeling of similar objects, SAM assist can dramatically speed up the process

​

Related Tools

• Polygon — For vector-based outlines (faster for simple shapes)
• Bounding Box — For quick rectangular localization
• Classification — For image-level labels without geometry
• Keypoint — For pose estimation skeletons