Skip to main content
Avala can call your ML models to generate pre-annotations, turning a blank canvas into a head start for human annotators. Connect a SageMaker endpoint or any custom HTTP model server, and Avala will send assets to it, receive predictions, and render them as editable annotations in the labeling editor.

​
Supported Providers

ProviderStatusDescription
Amazon SageMakerAvailableManaged inference endpoints with IAM-based authentication. Currently supports SAM (Segment Anything Model) and YOLO.
Custom HTTP EndpointComing SoonAny HTTP server that accepts a POST request and returns predictions in Avala’s format.
Custom HTTP endpoint support is under development. Currently, inference is available through Amazon SageMaker with built-in SAM and YOLO models. The self-service configuration UI described below is planned — model setup is currently handled by the Avala team during onboarding.

​
Amazon SageMaker Setup

​
IAM Role

Create an IAM role that allows Avala to invoke your SageMaker endpoint: 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sagemaker:InvokeEndpoint",
      "Resource": "arn:aws:sagemaker:us-east-1:YOUR_ACCOUNT_ID:endpoint/your-endpoint-name"
    }
  ]
}
Set up a trust relationship so Avala’s AWS account can assume this role. The Avala account ID is provided in Mission Control during configuration.

​
Endpoint Configuration

Your SageMaker endpoint must accept image or point cloud data and return predictions in Avala’s annotation format (see Prediction Response Format below).

​
Connect in Mission Control

  1. Go to Mission Control > Settings > Inference.
  2. Click Add Provider and select Amazon SageMaker.
  3. Enter the Endpoint Name and Region.
  4. Provide the IAM Role ARN that Avala should assume.
  5. Click Test Connection to verify Avala can invoke the endpoint.
  6. Save the configuration.

​
Custom HTTP Endpoint Setup

If you are running your own model server (PyTorch Serve, Triton, BentoML, a plain Flask app, etc.), you can connect it directly.

​
Request Format

Avala sends a POST request to your endpoint with the following JSON body: 
{
  "asset_id": "asset_abc123",
  "asset_url": "https://signed-url-to-the-image-or-pointcloud",
  "asset_type": "image",
  "width": 1920,
  "height": 1080,
  "labels": ["car", "pedestrian", "cyclist", "truck"]
}
FieldTypeDescription
asset_idstringUnique identifier for the asset.
asset_urlstringSigned URL to download the asset. Valid for 1 hour.
asset_typestringOne of image, point_cloud, or video_frame.
widthintegerWidth in pixels (images and video frames only).
heightintegerHeight in pixels (images and video frames only).
labelsstring[]The label set configured for the project. Your model should return predictions using these labels.

​
Prediction Response Format

Your endpoint must return a JSON response with an annotations array: 
{
  "annotations": [
    {
      "type": "bounding_box",
      "label": "car",
      "confidence": 0.94,
      "coordinates": {
        "x": 120,
        "y": 340,
        "width": 200,
        "height": 150
      }
    },
    {
      "type": "polygon",
      "label": "pedestrian",
      "confidence": 0.87,
      "points": [
        [450, 200],
        [470, 200],
        [480, 350],
        [440, 350]
      ]
    }
  ]
}

​
Connect in Mission Control

  1. Go to Mission Control > Settings > Inference.
  2. Click Add Provider and select Custom HTTP Endpoint.
  3. Enter the Endpoint URL (must be HTTPS).
  4. Optionally configure Authentication (Bearer token or custom header).
  5. Set the Timeout (default: 30 seconds).
  6. Click Test Connection to verify Avala can reach the endpoint.
  7. Save the configuration.

​
Supported Prediction Types

TypeKeyDescription
Bounding Boxbounding_boxAxis-aligned rectangle defined by x, y, width, height.
PolygonpolygonClosed polygon defined by an ordered array of [x, y] points.
Segmentation Masksegmentation_maskPixel-level mask as a run-length encoded (RLE) string or a URL to a PNG mask image.
ClassificationclassificationAsset-level or frame-level label with a confidence score.
3D Bounding Boxbounding_box_3dCuboid in 3D space defined by center (x, y, z), dimensions (l, w, h), and rotation (quaternion).
PolylinepolylineOpen polyline defined by an ordered array of [x, y] points. Used for lanes, edges, etc.

​
Auto-Labeling Workflow

Once a provider is connected, you can use it to pre-annotate tasks:
  1. Select a project in Mission Control and open Settings > Auto-Label.
  2. Choose the Inference Provider to use.
  3. Configure the Confidence Threshold. Predictions below this threshold are discarded (default: 0.5).
  4. Click Run Auto-Label to send all unlabeled assets in the project to the model.
  5. Avala displays the predictions as pre-annotations in the labeling editor.
  6. Annotators review each prediction — they can accept it as-is, adjust it, or delete it.
  7. Once reviewed, the task is submitted normally through the project workflow.
You can also trigger auto-labeling via the API: 
curl -X POST https://server.avala.ai/api/v1/projects/{project_id}/auto-label \
  -H "X-Avala-Api-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"provider_id": "inf_abc123", "confidence_threshold": 0.6}'
Model predictions are always treated as suggestions. Every prediction must be reviewed and either accepted or corrected by a human annotator before it becomes a final annotation. This ensures your labeled data meets quality standards even when using AI assistance.