Architecture Overview

The below diagram shows how Smart Capture operates with respect to other system components.

Smart Capture On-Device Trigger Engine

The Smart Capture On-Device Trigger Engine which is delivered as a Python SDK which you implement on your edge devices. The trigger engine is responsible for:

• Loading and deserializing trigger definitions that have been transmitted to the device from the Smart Capture server.
• Evaluating loaded triggers against edge data including the device state (e.g. speed of the vehicle, or driver intervention state), the model inputs (e.g. sensory inputs such as 2D camera and Lidar feeds), the model outputs (e.g. object detection, confidence/uncertainty scores), or more complex definitions such as using auto-tags defined in Nucleus.
• Tracking sample rates so the device is not storing excessive data beyond the user-defined sample rates.
• Recommending if data should be retained based on if the data meets any trigger criteria, and is within the allowable sample rate.

Smart Capture Server

This component is responsible for:

• Maintaining registries containing the triggers definitions, associated devices, and models metadata as well.
• Managing the deployments of triggers to the device fleet.
• Recording the version and deployment history of triggers.
• Serving a API and web interface enabling you to:
  1. define triggers
  2. register devices
  3. initiate deployment of triggers to devices
  4. query for the current state of triggers, devices, and deployments
  5. update the state of the registries

This component is hosted and run on Scale cloud infrastructure.

Device Fleet

The device fleet are your edge devices which are capturing perception data and typically running models as well. These edge devices may be autonomous vehicles, robots, drones, or IoT devices for example.

Update Server

Because most device fleets are not connected to the web for safety and security purposes, we recommend that you create an update server to communicate between the Smart Capture server and your edge devices.

The update server is implemented by you and is responsible for obtaining updated trigger configurations from the Smart Capture Server and transmitting those updates to your device fleet. In turn, this the Update Server is also responsible for informing the Smart Capture Server on the deployment status of triggers on individual devices.

On-Device Update Service

The Update Service is responsible for interacting with the Update Server to periodically check for changes to trigger definitions for the assigned device and then transferring those downloaded payloads to the Smart Capture On-Device Trigger Engine for loading. The Update Service is also responsible for periodically uploading information such as trigger evaluation errors or loading confirmations which are logged by the Smart Capture On-Device Trigger Engine, and then relaying them to the Update Server.

Nucleus

Scale Nucleus is an adjacent products which serves as a dataset management platform that helps ML teams build better datasets. It helps you to:

• Visualize your data
• Curate interesting slices within your dataset
• Review and manage annotations
• Measure and debug your model performance

Access to Nucleus is not required to use Smart Capture, but using the products together unlocks additional capabilities such as automatically uploading captured data into Nucleus for curation, debugging and visualization, as well as using Nucleus-trained Autotags as a Smart Capture triggers.

Data Lake

The data lake is a system component managed by you where you warehouse your captured production data including perception data, sensor readings, device state, and model outputs.