Search by Tags

AI at the Edge, Pasta Detection Demo with AWS

 

Article updated at 03 Dec 2019
Compare with Revision




Subscribe for this article updates

Amazon Web Services

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 165 fully-featured services from data centers globally. Millions of customers - including the fastest-growing startups, largest enterprises, and leading government agencies — trust AWS to power their infrastructure, become more agile, and lower costs.

AI at the Edge Pasta Detection

This is a computer vision demonstration that integrates several technologies to showcase an edge computing scenario that involves the detection of pasta using artificial intelligence technology - deep learning, and the visualization of live and historic data processing both on a cloud dashboard, using cloud services from Amazon Web Services, as well as a local graphical user interface (GUI), using a GUI framework that relies on modern web technologies.


  • pasta demo components

    The AI at the Edge Pasta Detection Demo ( click to enlarge)

Here is a brief technical overview of the technologies used:

  • An industrial MIPI-CSI2 camera to capture a video stream from the conveyor belt.
  • Deep learning inference running on the Apalis iMX8 at the edge, using the DLR inference engine.
    • Trained using Amazon SageMaker and optimized on Amazon SageMaker Neo.
  • A local GUI on the HDMI video output built with the Quasar Framework, displaying live pasta detection.
  • A web dashboard to display historic data, built using several connected AWS services:
    • AWS IoT Greengrass for MQTT data ingestion and relaying to other AWS services.
    • Amazon DynamoDB for data storage.
    • Amazon Cognito for secure user management.
    • Amazon CloudFront for web dashboard hosting.

See below an illustration of how the AI at the Edge Pasta Detection demonstration integrates the AWS services:


  • AWS Pasta Demo Architecture

    AWS Pasta Demo Architecture

AWS IoT Greengrass

AWS IoT Greengrass extends AWS to edge devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. With AWS IoT Greengrass, connected devices can run AWS Lambda functions, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely – even when not connected to the Internet.

In the context of this demo, AWS IoT Greengrass provides a seamless, secure and resilient connection to the cloud. System data such as the deep learning inference results and the overall system status are collected, pre-processed and sent to an MQTT broker. At the same time, commands to control the conveyor belt and the LED brightness are received from the web dashboard, providing two-way communication between device and cloud.

Amazon SageMaker Neo

Amazon SageMaker Neo enables developers to train machine learning models once and run them anywhere in the cloud and at the edge. Amazon SageMaker Neo optimizes models to run up to twice as fast, with less than a tenth of the memory footprint, with no accuracy loss.

Amazon SageMaker is used to train a deep learning inference model from a pasta dataset, focusing on object detection and using the MobileNet SSDv1 algorithm, while Amazon SageMaker Neo then optimizes the trained model for the NXP i.MX 8QuadMax processor, which is the core of Toradex Apalis iMX8.

Supported Modules

The following Computer on Module is supported:

Supported Displays

Prerequisites

Optional Items

The Apalis iMX8 Embedded Vision Kit with Allied Vision does not contain the mechanical parts and accessories to assemble the full demo as showcased by Toradex, NXP, and AWS in tradeshows and events. These items are not mandatory to have the functional demo running. However, you can source or manufacture the components by yourself, and then assemble the full demo:

How to Get Started

This section provides instructions for you to get started with the AI at the Edge Pasta Detection demo.

Assemble the Apalis iMX8 Embedded Vision Kit with Allied Vision

To assemble the kit only, follow the instructions enumerated and illustrated by the gif below:

  1. Unbox all items.
  2. Connect the camera to the adapter.
  3. Connect the adapter to the board.
  4. Connect the Apalis iMX8 to the Ixora Carrier Board.
  5. Make sure that the Apalis iMX8 is locked and well connected.
  6. Mount the Apalis heatsink on top of the module and fasten with screws.
  7. Screw the lens to the camera.
  8. Remove the lens protection.
  9. Plug all cables to the Ixora Carrier Board - HDMI, Ethernet, USB mouse / keyboard, power supply barrel jack.

  • Assemblying the kit

    Assemblying the kit

We do not yet provide instructions for assembling the conveyor belt - which consists of the optional items described in the Prerequisites section.

Install the Local User Interface

Power on the system. Toradex Easy Installer comes pre-installed and will be display on the HDMI interface:

Note: Your module should have come with the Toradex Easy Installer pre-installed. If this isn't the case, you can easily follow the Loading Toradex Easy Installer article before proceeding.


  • Toradex Easy Installer

    Toradex Easy Installer (click to enlarge)

Do the following:

  • Write down the Ethernet IP from Network Information.
  • Write down the unique Serial number from Module Version.
  • Select AWS and NXP AI at the Edge Pasta Detection Demo image from the list and install it.

  • Installing AWS and NXP AI at the Edge Pasta Detection Demo using the Toradex Easy Installer

    Installing AWS and NXP AI at the Edge Pasta Detection Demo using the Toradex Easy Installer (click to enlarge)

After the installation and reboot, you will see at the HDMI display the welcome screen.


  • After the installation, HDMI Display will show the welcome screen

    After the installation, HDMI Display will show the welcome screen

It may take 5 minutes or more for the demo to start. It happens because Docker containers are fetched online after the first boot. After the containers are downloaded and started you can see some inferences in the local user interface.


  • HDMI Display will show the local User-Interface

    HDMI Display will show the local User-Interface

Now you can test the local user interface with some pasta.

Connect the Demo to Cloud

This section provides instructions to setup the connect your board to your AWS account.

Create an AWS Account and User Access Key

Follow AWS page instructions to create a new AWS account.

Once you have access to the account, create a user access key. Follow the instructions on the article Managing Access Keys for Your AWS Account Root User to understand how to create it. You will create the access key ID and secret access key as a set.

Make sure to download your AWS credentials to your local machine, which will be used for future reference, by clicking on the "Download Key File" button.

Attention: if you have just registered and this is a fresh AWS account, some services are not available before AWS validates your account which can take up to 24 hours.


  • Creating AWS Security Credential Keys

    Creating AWS Security Credential Keys

Attention: During access key creation, AWS gives you only one opportunity to view and download the secret access key part of the access key. If you miss or lose it, you will need to create a new access key.

Connect Your Device

When you installed the demo image using the Toradex Easy Installer, you were instructed to write down the serial number and the Ethernet IP. You will select one of them to access a web-based user interface to easily create the cloud infrastructure directly from the Computer on Module.

On a desktop PC connected to the same network as the Computer on Module, open a web-browser (for example Chrome or Firefox) and use either one of the following URLs:

  • http://<Board's Ethernet IP>:8080

    or

  • http://apalis-imx8-<serial number>.local:8080

See the example below for my Apalis iMX8, with Ethernet IP 192.168.10.5 and serial number 0333444555. The zero to the left cannot be disregarded:

  • http://192.168.10.5:8080

    or

  • http://apalis-imx8-0333444555.local:8080


Which method should I use


You will see the following screen:


  • Pasta Demo Credentials Setup

    Pasta Demo Credentials Setup Page

Note: the browser may mark the website as not secure.

Create the Cloud Infrastructure

Fill the access key ID and secret access key created previously and hit the Run CloudFormation button. It will deploy the entire AWS infrastructure required to run the demonstration.

Note: this step can take around 15 minutes to 1 hour, since Amazon CloudFront needs time to register the internet domains. Now is the perfect time to go grab a cup of coffee.


  • Inserting AWS credentials in the demo

    Inserting AWS credentials in the demo

Once the progress bar hits 100%, the full deployment is finished. Within seconds your board will start sending data to your dashboard. Copy the URL for your own dashboard as illustrated in the gif above.

Access the Web Dashboard

Use the URL retrieved above to access, sign-up and sign-in to your web dashboard:


  • Cloud-based Web Interface sign-up and sign-in

    Cloud-based Web Interface sign-up and sign-in

Play With the Pasta Demo

Put some pasta under the camera and see it reflect on the local user-interface and the web dashboard.

Next Steps

The AI at the Edge Pasta Detection demo is open-source. Notice that it is provided as-is.

You can tweak the demo in several ways, from the web UI to the inference model and much more. Here are the public GitHub repositories and additional resources:

GitHub Repository Description Additional Resources
aws-nxp-ai-at-the-edge AWS Lambdas, containers, inference application, etc -
aws-nxp-ai-at-the-edge-gui Local graphical user interface (GUI) that outputs on HDMI -
aws-nxp-ai-at-the-edge-credentials-setup Tool to add AWS IoT Greengrass Core device credentials and set up the cloud infrastructure to a fresh setup hardware -
meta-pasta-demo Yocto layer to re-build and/or customize the Linux image Instructions how to re-build the Linux image on Build the AWS AI at the Edge Demo Image
aws-nxp-ai-at-the-edge-cloud-dashboard Cloud dashboard interface GUI and infrastructure -

Release Notes

1.0.0

  • Initial Release