Edge and IoT

Some ML workloads cannot rely on cloud connectivity — remote factories, vehicles, field equipment, or any environment with unreliable internet. AWS provides services to deploy trained models to edge devices for local inference, along with physical devices for data transfer and edge compute.

Overview

Service	What It Does	When to Use
AWS IoT Greengrass	Run ML inference on edge devices locally	Near-real-time inference at the edge with unreliable or no internet connectivity
AWS Snowball Edge Compute Optimized	Physical device for petabyte data transfer with NVIDIA Tesla GPU for edge compute	Petabytes of data to move plus GPU-powered ML inference at the edge
AWS Snowcone	Smallest Snow family device (8 TB, 2 vCPUs)	Small-scale edge data collection. Very limited compute

The Edge ML Deployment Pipeline

The standard workflow for deploying ML models to edge devices:

1. Train the model on SageMaker (in the cloud)
2. Compile with SageMaker Neo (optimizes for target hardware — ARM, Intel, NVIDIA)
3. Deploy via IoT Greengrass to edge devices for local inference

This pipeline lets you train with full cloud resources and deploy lightweight, optimized models to constrained edge hardware.

Device Comparison

	IoT Greengrass	Snowball Edge Compute	Snowcone
Purpose	Edge inference on existing devices	Data transfer + edge compute	Small-scale data collection
GPU	Depends on host device	NVIDIA Tesla GPU (yes)	No GPU
Storage	Host device storage	Petabytes	8 TB
ML capability	Full inference (Neo-compiled models)	GPU-powered training/inference	Very limited — not for ML inference
Key trigger	"Unreliable connectivity" + "local inference"	"Petabytes" + "GPU at edge"	Small data collection at edge

When to Use

Use IoT Greengrass when you need ML inference at the edge with unreliable connectivity — it is the standard solution for edge ML deployment on AWS. Use Snowball Edge Compute Optimized when you also need to move petabytes of data and have GPU compute requirements at the edge. Do not use Snowcone for ML inference — it lacks GPU capability.

Flashcards

1 / 5

Question

What is the standard pipeline for deploying ML models to edge devices on AWS?

Click to reveal

Answer

Train on SageMaker → Compile with SageMaker Neo → Deploy via IoT Greengrass. Neo optimizes the model for the target hardware.

When to Use

The key trigger for IoT Greengrass is "unreliable connectivity" combined with the need for local inference. If connectivity is reliable, it is simpler to call a SageMaker endpoint in the cloud. Greengrass adds complexity that is only justified when you cannot depend on a network connection.