Search by Tags

TorizonCore Builder Tool - Customization for Production Programming and Torizon OTA

 

  1. Torizon 5.0.0
    1. Introduction
    2. Requirements
    3. Typical Use Cases
      1. Customizing a TorizonCore Image for Production
      2. Deploying TorizonCore OS Images
    4. Installing TorizonCore Builder
      1. OS Compatibily
      2. Note for Users Installing TorizonCore Builder on Windows Machine
      3. Setup Script
    5. Using TorizonCore Builder
      1. Workflow
      2. Creating a configuration file
      3. Customizing the configuration file
        1. Minimal Configuration File
        2. Fetching Remote Images
        3. Fetching remote images: URL autogenerated
        4. Splash Screen
        5. Device-tree
        6. Kernel Modules
        7. Kernel Arguments (Parameters)
        8. Capturing Configuration Changes in the Target Device
        9. Pre-provisioning Docker Container Images
        10. Pre-provisioning Docker Container Images: Pre-built Container Images
      4. Building the image
        1. Different configuration file name
        2. Variable substitution
      5. Deploying the image
        1. Toradex Easy Installer
        2. Before image deployment (SSH and Torizon Platform)
        3. Directly, through SSH
        4. Torizon Platform Remote Update Service
    6. Additional Resources
    7. TorizonCore Builder - Commands Manual
      1. TorizonCore Builder and OSTree
      2. The torizoncore-builder command
        1. Usage
      3. The Build Command
        1. Usage
      4. The Bundle Command
        1. Usage
          1. Example
      5. The Combine Command
        1. Usage
      6. The Deploy Command
        1. Usage
        2. Arguments
          1. Positional Argument
        3. Example
          1. with Docker volume
          2. without Docker volumes
      7. The Dt (Device Tree) Command
        1. dt checkout
          1. Usage
        2. dt apply
          1. Usage
          2. Example
        3. dt status
          1. Example
      8. The Dto (Device Tree Overlay) Command
        1. dto apply
          1. Usage
          2. Example
        2. dto list
          1. Usage
          2. Example
        3. dto status
          1. Usage
          2. Example
        4. dto remove
          1. Usage
          2. Example
        5. dto deploy
          1. Usage
          2. Example
      9. The Images Command
        1. images download
        2. Usage
        3. Arguments
        4. Example
          1. With docker volume
          2. Without docker volume
        5. images unpack
          1. Usage
        6. Example
          1. With docker volume
          2. Without Docker Volume
      10. The Isolate Command
        1. Usage
        2. Example
      11. The Kernel Command
        1. kernel buildmodule
          1. Usage
          2. Arguments
          3. Example
        2. kernel setcustomargs
          1. Usage
          2. Example
        3. kernel getcustomargs
          1. Usage
          2. Example
        4. kernel clearcustomargs
          1. Usage
          2. Example
      12. The OSTree Command
        1. Usage
        2. ostree serve
        3. Usage
      13. The Push Command
        1. Usage
        2. Arguments
        3. Example
      14. The Splash Command
        1. Usage
      15. The Union Command
        1. Usage
      16. Setting attributes to files and directories using the ".tcattr" file
        1. Example of creating a ".tcattr" file for an additional changes directory
      17. TorizonCore Builder and OSTree
      18. The torizoncore-builder command
        1. Usage
      19. The Bundle Command
        1. Usage
          1. Example
      20. The Combine Command
        1. Usage
      21. The Deploy Command
        1. Usage
        2. Arguments
          1. Positional Argument
        3. Example
          1. with Docker volume
          2. without Docker volumes
      22. The Dt (Device Tree) Command
        1. dt checkout
          1. Usage
        2. dt apply
          1. Usage
          2. Example
        3. dt status
          1. Example
      23. The Dto (Device Tree Overlay) Command
        1. dto apply
          1. Usage
          2. Example
        2. dto list
          1. Usage
          2. Example
        3. dto status
          1. Usage
          2. Example
        4. dto remove
          1. Usage
          2. Example
        5. dto deploy
          1. Usage
          2. Example
      24. The Images Command
        1. images download
        2. Usage
        3. Arguments
        4. Example
          1. With Docker volume
          2. Without Docker volume
        5. images unpack
          1. Usage
        6. Example
          1. With Docker volume
          2. Without Docker Volume
      25. The Isolate Command
        1. Usage
        2. Example
      26. The Kernel Command
        1. kernel buildmodule
          1. Usage
          2. Arguments
          3. Example
        2. kernel setcustomargs
          1. Usage
          2. Example
        3. kernel getcustomargs
          1. Usage
          2. Example
        4. kernel clearcustomargs
          1. Usage
          2. Example
      27. The OSTree Command
        1. ostree serve
        2. Usage
      28. The Push Command
        1. Usage
        2. Arguments
        3. Example
      29. The Splash Command
        1. Usage
      30. The Union Command
        1. Usage
      31. Setting attributes to files and directories using the ".tcattr" file
        1. Example of creating a ".tcattr" file for an additional changes directory
    8. Extra Resources for TorizonCore Customization
  2. Torizon 4.0.0
    1. Introduction
    2. Workflow
    3. Commands
    4. Operation
    5. Usage
      1. Basic
        1. unpack
        2. Isolate
        3. Union
        4. Deploy
        5. Bundle
        6. Combine
        7. Example
      2. Advanced
        1. Base Command
        2. Unpack
          1. Arguments
          2. Example
            1. With docker volume
            2. Without Docker Volume
        3. Isolate
          1. Arguments
          2. Example
            1. with docker volume
            2. without docker volume
        4. Union
          1. Arguments
          2. Example
            1. with docker volume
            2. without docker volume
        5. Deploy
          1. Arguments
          2. Example
            1. with docker volume
            2. without docker volumes
        6. Bundle
          1. Arguments
          2. Examples
        7. Combine
          1. Arguments
            1. Example
Article updated at 05 Mar 2021
Compare with Revision



Subscribe for this article updates

Select the version of your OS from the tabs below. If you don't know the version you are using, run the command cat /etc/os-release or cat /etc/issue on the board.



Remember that you can always refer to the Torizon Documentation, there you can find a lot of relevant articles that might help you in the application development.

Torizon 5.0.0

Introduction

TorizonCore Builder Tool enables a developer to easily customize Torizon with device configuration, preinstalled containers, settings, and more. The main purpose of TorizonCore Builder is to enable the preparation of TorizonCore for production programming. In the end, you will have a custom TorizonCore image that you can flash onto the SoM using the Toradex Easy Installer or push to Torizon OTA for deploying it remotely over-the-air.

This article complies to the Typographic Conventions for Torizon Documentation.

Requirements

  • A Toradex SoM with Torizon installed

Typical Use Cases

TorizonCore builder tool provides multiple commands to be used in different situations. We will describe below typical TorizonCore Builder use cases and a high-level description of which TorizonCore Builder command to use:

Customizing a TorizonCore Image for Production

TorizonCore Builder provides standalone commands that allow you to easily customize a TorizonCore image. Such changes could be:

  • Pre-provisioning Docker Container Images: As a method for Deploying Container Images to TorizonCore, TorizonCore Builder can download user Docker containers images and combine them with an off-the-shelf TorizonCore image to generate a new, monolithic, image. This process is especially useful when preparing an image to be flashed on several units in production. Its usage is described in the Pre-provisioning Docker Container Images sections.

  • Setting the System's Splash Screen: You can replace the System's Splash Screen using TorizonCore Builder as described in the Splash Screen section.

  • Adding New Hardware to the System: When setting up new hardware, for example, adding a display to the board, or changing some SoC's pin functionality, sometimes a developer will have to make changes to the Linux Device Tree by applying a Device Tree Overlays. TorizonCore Builder can build a full device tree or apply device tree overlays. See the Device-tree section to learn how to use it.

  • Building External Kernel Modules: It is often necessary to build an external kernel module (a.k.a. out-of-tree module) when this module doesn't exist in the standard Linux tree. Some examples where this is necessary are when adding support to some device/peripheral. Check the Building Kernel Module section to learn how to do it.

  • Capturing Configuration Changes in a Device: If a user has made any configuration changes in TorizonCore ( e.g., the /etc directory) in an SoM, the tool is able to capture all those changes (modifications, addition, and deletion) through an ssh connection between the host PC and the target board and prepare it to deploy into a new custom TorizonCore image.
    Here are some examples of configuration that modify /etc:

    For more details on how to do it, please refer to the Capturing Configuration Changes in the Target Device section.

Deploying TorizonCore OS Images

Users can deploy the image in three different ways:

To learn how to do it, refer to the Deploying the image section of this page.

Installing TorizonCore Builder

OS Compatibily

TorizonCore Builder executes in a Docker Runtime and it was tested on the following Operating Systems:

  • Linux
  • Windows 10 with Windows Subsystem for Linux (WSL2)

Warning: WSL2 requires Windows 10, updated to version 2004 (also known as the Windows 10 May 2020 Update) or higher.

Note for Users Installing TorizonCore Builder on Windows Machine

TorizonCore Builder support on a Windows desktop requires the "Windows Subsystem for Linux 2" (WSL 2). This is due to TorizonCore Builder relying on a Linux-based filesystem.

Before proceeding with any TorizonCore Builder instructions please set up a WSL 2 distro and Docker on your development host as shown on Configure Build Environment for Torizon Containers. As a reminder, any and all TorizonCore Builder commands should be run in your WSL 2 distro.

Finally, as a quick tip, you'll occasionally need to transfer files to and from your WSL 2 distro and your Windows host. You can actually do this by using the built-in Windows file explorer utility. This and other Windows utilities can actually be invoked from within your WSL 2 Distro. For example by doing the following in your WSL 2 Distro:

$ explorer.exe .

The above command will open the Windows file explorer in your current directory. From here you'll be able to copy/paste and move files between the WSL 2 world and the normal Windows world.

Setup Script

Create and enter a working directory where your customization will be stored:

$ mkdir ~/tcbworkdir
$ cd ~/tcbworkdir

To use TorizonCore Builder, you need to install it on your host machine by creating an alias. We provide a setup script named tcb-env-setup.sh on Toradex Github to facilitate the installation and setup TorizonCore Builder.

For basic usage it can be run as follows with no arguments:

Warning: Make sure to source the script. Otherwise, it will not work as intended and most importantly the torizoncore-builder alias will not be created properly.

$ wget https://raw.githubusercontent.com/toradex/tcb-env-setup/master/tcb-env-setup.sh
$ source tcb-env-setup.sh

For more advanced usage of the setup script please run source tcb-env-setup.sh -h, or see the project README.

Example expected output for the first installation:

TorizonCore Builder is not installed. Pulling the latest version from Docker Hub...
Setting up TorizonCore Builder with version 2.

Pulling TorizonCore Builder...
2: Pulling from torizon/torizoncore-builder
7783ecb52c24: Pull complete
d2c061578333: Pull complete
96046ec03d42: Pull complete
feb49f8f9a32: Pull complete
6f5a136384f4: Pull complete
1b448ab72455: Pull complete
9738893ca531: Pull complete
eb0c196b0855: Pull complete
0ef22c696c8d: Pull complete
58da04a4b089: Pull complete
bc364f234f04: Pull complete
3c10f77095c8: Pull complete
2c4d076b4725: Pull complete
65d31292f35e: Pull complete
f553605a42f8: Pull complete
6824da993f58: Pull complete
86f495c51638: Pull complete
Digest: sha256:b5b94cbc962885752ebe2bef498e9a8bcf2bee4fe89c0c162dc857655284ed84
Status: Downloaded newer image for torizon/torizoncore-builder:2
docker.io/torizon/torizoncore-builder:2
Done!

Setup complete! TorizonCore Builder is now ready to use.
For more information, run 'torizoncore-builder -h' or go to https://developer.toradex.com/knowledge-base/torizoncore-builder-tool

As shown the script will guide you to ensure that the latest version of TorizonCore Builder is set up on your machine if no previous versions were found. If there are previous versions you will be asked if you'd like to use these or look for the latest updates online instead.

Tip: When configuring the latest version in your machine, the setup script will also install a bash completion script for TorizonCore Builder, making it possible to autocomplete commands and parameters by just pressing the TAB key.

Using TorizonCore Builder

This section shows how to use TorizonCore Builder in a simple way, yet covering almost all use cases. It is based on the build command, which allows the full image customization in a single step, using a configuration YAML file. The other commands that are necessary in conjunction with the build command in order to cover most use cases are also explained in this section.

If your use case is not covered here, please refer to the TorizonCore Builder - Commands Manual section for a detailed explanation of all TorizonCore Builder commands.

Workflow

  • Create a configuration file with the desired settings (the build command can even help you with that);
  • Customize the configuration file in order to select the desired input image, include the desired customization and get the desired outputs. Also, run the auxiliary commands where needed so all local artifacts possibly referenced by the configuration file are available;
  • Run the build command;
  • Deploy the output to the board or to Torizon OTA server;

Creating a configuration file

The easiest and thus recommended way for creating an initial configuration file is by running:

$ torizoncore-builder build --create-template

The previous command will create a file named tcbuild.yaml containing all possible configuration parameters accepted by the command.

The configuration file is in a standard YAML format which should be recognized by most modern programming editors. Any other name can be chosen for the file, but tcbuild.yaml is the name that the build command will expect by default when running in build mode. Most parameters in the generated file will be commented out which can be seen by the # character at the beginning of each line. Lines having the characters >> are just explaining the contents of the file and should be left as-is or removed if desired.

The contents of the file are organized in a tree structure where the leaves are configurable properties. In order to set a certain property, you need to make sure all its parent properties are also uncommented (by removing the first # character).

The configuration file follows the YAML format and syntax. The tool expects it to be made up of three sections, represented by the top-level properties (keys): input, customization and output as outlined below:

tcbuild.yaml
# Configuration file outline:
input:
  # Input section items (required).
customization:
  # Customization section items.
output:
  # Output section items (required).

Customizing the configuration file

Minimal Configuration File

A minimal configuration file has to specify at least one input and one output. Below we assume that the input is a previously downloaded Toradex Easy Installer image that is stored in a directory called images relative to the working directory (set by the alias of the tool).

When downloading a Toradex Easy Installer image, be sure to download an image without evaluation containers, because the input image to be customized MUST NOT contain bundled container images. To download the image, go to the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.

tcbuild.yaml
# Sample configuration file:
input:
  easy-installer:
    local: images/torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

The directory structure of such a "project" would look like this:

.
├── images
│   └── torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
└── tcbuild.yaml

Fetching Remote Images

The input to the tool can also be specified as a URL to a remote server instead of a directory with a downloaded image, as in the Minimal Configuration section. If the input is passed as a remote image, the image will be downloaded every time the image is built.

In the sample configuration below, we directly download an image from Toradex's artifacts server:

tcbuild.yaml
# Sample configuration input, specifying a remote:
input:
  easy-installer:
    remote: "https://artifacts.toradex.com:443/artifactory/torizoncore-oe-prod-frankfurt/dunfell-5.x.y/release/7/colibri-imx6/torizon-upstream/torizon-core-docker/oedeploy/torizon-core-docker-colibri-imx6-Tezi_5.3.0%2Bbuild.7.tar;sha256sum=49fc6a32ab3fc21a184fb8333c19ad53f22875975219883c1087d929bedf470f"

Notice the presence of the sha256sum parameter which can be used to ensure that the downloaded file has a certain SHA-256 checksum – the tool will stop if the actual file checksum does not match that value. For more information about how a remote location can be specified please refer to the build command Detailed Manual.

Fetching remote images: URL autogenerated

Another way to specify an input is the sample configuration below:

tcbuild.yaml
# Sample configuration input, specifying parameters for URL autogeneration:
input:
  easy-installer:
    toradex-feed:
      version: "5.3.0"
      release: quarterly
      machine: colibri-imx6
      distro: torizon-upstream
      variant: torizon-core-docker
      build-number: "7"
      #build-date: "202107"

The build command then autogenerates the URL shown in the remote parameter of the Fetching Remote Images section. For monthly and nightly releases the parameter build-date is needed.

Splash Screen

To customize the splash screen, insert the configuration below:

tcbuild.yaml
#Sample customization: insert a Splash Screen
customization:
  # >> Splash screen:
  splash-screen: files/splash.png

The file files/splash.png should be available before running the build command. A demo image can be downloaded by clicking on the button below:

Example Splash Screen Image

Check the Splash Screen on TorizonCore article for more details.

Device-tree

This customization can be used both with Toradex's device-trees and overlays as well as any other compatible device-tree and overlay. The files containing the source code of the device-trees and overlays should be available before running the build command.

To get the source code of the Toradex supplied device-tree files (including overlays), one could use the command below, accordingly with the device:

$ git clone -b toradex_5.4.y https://github.com/toradex/device-trees.git
$  git clone -b toradex_5.4-2.3.x-imx https://github.com/toradex/device-trees.git
$ torizoncore-builder dt checkout

To see the available device-trees and select the appropriate one for your device, run the command below, passing the parameter -name accordingly to your device.

$ find device-trees/dts-arm32 -name "*imx6q-apalis*.dts"
device-trees/dts-arm32/imx6q-apalis-ixora-v1.2.dts
device-trees/dts-arm32/imx6q-apalis-ixora.dts
device-trees/dts-arm32/imx6q-apalis-ixora-v1.1.dts
device-trees/dts-arm32/imx6q-apalis-eval.dts
$ find device-trees/dts-arm64 -name "*imx8qp-apalis*.dts"
device-trees/dts-arm64/imx8qp-apalis-v1.1-ixora-v1.1.dts
device-trees/dts-arm64/imx8qp-apalis-v1.1-eval.dts
$ find device-trees/dts-arm64 -name "*imx8qxp-apalis*.dts"
device-trees/dts-arm64/imx8qxp-apalis-v1.1-eval.dts
$ find device-trees/dts-arm32 -name "*imx6dl-colibri*.dts"
device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
$ find device-trees/dts-arm32 -name "*imx7d-colibri*.dts"
device-trees/dts-arm32/imx7d-colibri-aster.dts
device-trees/dts-arm32/imx7d-colibri-emmc-eval-v3.dts
device-trees/dts-arm32/imx7d-colibri-emmc-aster.dts
device-trees/dts-arm32/imx7d-colibri-eval-v3.dts
$ find device-trees/dts-arm64 -name "*imx8qxp-colibri*.dts"
device-trees/dts-arm64/imx8qxp-colibri-aster.dts
device-trees/dts-arm64/imx8qxp-colibri-dsihdmi-eval-v3.dts
device-trees/dts-arm64/imx8qxp-colibri-eval-v3.dt
$ find device-trees/dts-arm64 -name "*imx8mm-verdin*.dts"
device-trees/dts-arm64/imx8mm-verdin-nonwifi-dev.dts
device-trees/dts-arm64/imx8mm-verdin-wifi-dev.dts
device-trees/dts-arm64/imx8mm-verdin-wifi-dahlia.dts
device-trees/dts-arm64/imx8mm-verdin-nonwifi-dahlia.dts
$ find device-trees/dts-arm64 -name "*imx8mp-verdin*.dts"
device-trees/dts-arm64/imx8mp-verdin-wifi-dahlia.dts
device-trees/dts-arm64/imx8mp-verdin-nonwifi-dahlia.dts
device-trees/dts-arm64/imx8mp-verdin-nonwifi-dev.dts
device-trees/dts-arm64/imx8mp-verdin-wifi-dev.dts

Then to check the available overlays names for a particular device-tree (imx6dl-colibri-eval-v3.dtb in this case), run the command below:

$ torizoncore-builder dto list --device-tree ./device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
Overlays compatible with device tree imx6dl-colibri-eval-v3.dts:
- device-trees/overlays/colibri-imx6_atmel-mxt-adapter_overlay.dts
- device-trees/overlays/colibri-imx6_atmel-mxt-connector_overlay.dts
- device-trees/overlays/colibri-imx6_hdmi_overlay.dts
- device-trees/overlays/colibri-imx6_parallel-rgb-lvds_overlay.dts
- device-trees/overlays/colibri-imx6_parallel-rgb_overlay.dts
- device-trees/overlays/colibri-imx6_stmpe-ts_overlay.dts
- device-trees/overlays/display-dpi-lt170410_overlay.dts
- device-trees/overlays/display-edt5.7_overlay.dts
- device-trees/overlays/display-edt7_overlay.dts
- device-trees/overlays/display-fullhd_overlay.dts
- device-trees/overlays/display-lt161010_overlay.dts
- device-trees/overlays/display-vga_overlay.dts

Then to include the device-tree and the overlays in the image, use the customization below:

tcbuild.yaml
#Sample customization: insert the resistive 7" display overlay in the IMX6 device-tree with Aster Carrier Board
customization:
  device-tree:
    include-dirs:
      - device-trees/include/
    custom: device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
    overlays:
      add:
        - device-trees/overlays/display-edt7_overlay.dts

See the following articles for more information:

Also, for more details on how the dt commands work, please check the dt command in the commands manual.

Kernel Modules

The build command allows you to build and install kernel modules into the deployed OS image. This is demonstrated in this configuration:

tcbuild.yaml
# Sample customization: build hello_mod module into the image
customization:
  kernel:
    modules:
      - source-dir: hello_mod/
        autoload: no

The directory hello_mod containing the source code files of the kernel modules should be available before running the command. To get the hello_mod module from our Github repository, run the command below:

$ git clone https://github.com/toradex/hello-mod

For more general information about the subject, please refer to Building External Kernel Modules With Torizon article.

Kernel Arguments (Parameters)

Kernel arguments (parameters) can be included in the OS image or modified, if they already exist, with the build command. This is done in the configuration file below:

tcbuild.yaml
# Sample customization: include or change (if already exists) key1 and key2 kernel arguments
customization:
  kernel:
    arguments:
      - key1=val1
      - key2=val2

To find the current parameters, run the following command on the board:

# dmesg | grep "Kernel command line"

Capturing Configuration Changes in the Target Device

If the user made configuration changes on TorizonCore in the SoM, this command is necessary to capture them. For more details about what it does and for which cases it is applicable, please check the Typical Use Cases section.

To capture the configuration changes made in the target device, run the command below:

$ torizoncore-builder isolate --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --changes-directory changes1

Then to include these changes in the image, use the customization below:

tcbuild.yaml
#Sample customization: insert the configuration changes made in the device
customization:
  filesystem:
     - changes1/

See the Capturing Changes in the Configuration of a Board on TorizonCore article for detailed instructions.

Also, for more details on how the isolate command works, please check the isolate command in the commands manual.

Pre-provisioning Docker Container Images

The build command also simplifies the process of bundling container images into an installer image. A sample configuration leveraging that feature is this:

tcbuild.yaml
#Sample customization: insert pre-provisioned Docker container images
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
  bundle:
      compose-file: custom/docker-compose.yml

With such an input, the tool will fetch the container images referenced by the specified docker-compose.yml file and combine them with the installer image.

A sample Docker Compose YAML file can be downloaded, based on your device, by clicking in one of the buttons below:

Example 32-bit Docker Compose file

Example 64-bit Docker Compose file

Pre-provisioning Docker Container Images: Pre-built Container Images

If you don't want the tool to download the images every time the build command is run, you can pass a directory with the pre-built container images instead of the Docker Compose file as a parameter.

To create a tarball with the container images from a Docker Compose file, in the directory bundle, use the command below:

$ torizoncore-builder bundle docker-compose.yml --bundle-directory bundle --platform linux/arm/v7

Then to include these pre-built container images in the image, through the directory, use the configuration below:

tcbuild.yaml
#Sample customization: insert pre-provisioned Docker container images with pre-built container images
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
  bundle:
    dir: bundle/

Find more details in the Pre-provisioning Docker Containers onto a TorizonCore image article.

For more details on how the bundle command works, please check the bundle command in the commands manual.

Building the image

To run the tool one would simply execute:

$ torizoncore-builder build

Once the build process finishes, the custom Easy Installer image would be present in the directory defined in the local of the output section of the configuration file, as it can be seen in the Minimal Configuration File section.

Different configuration file name

The default configuration file used is the tcbuild.yaml file, created in the Creating a Configuration file section. In case of using a configuration file with a different name, run the command specifying the configuration file name:

$ torizoncore-builder build -f <configuration_file_name>

Variable substitution

A feature that can be especially useful in CI scripts is called variable substitution which allows the replacement of property values (or parts of them) based on parameters passed on the command line of the tool. The following configuration file employs that feature:

tcbuild.yaml
# Sample configuration input with variable substitution
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-with-containers
    name: "${IMAGE_NAME-Default image name}"
    description: "${IMAGE_DESC?Please provide a description for this image}"

The value of the property output/easy-installer/name in the example is set to the value of variable IMAGE_NAME or to "Default image name" when the variable is not set. The property output/easy-installer/description depends on IMAGE_DESC which is required and if not set will cause the tool to exit showing the specified message. Running the build command by itself would fail like this:

$ torizoncore-builder build
Building image as per configuration file 'tcbuild.yaml'...

** Exiting due to user-defined error: Please provide a description for this image

In order to successfully build an image with the previous configuration file (using the default value for IMAGE_NAME), one could run:

$ torizoncore-builder build --set IMAGE_DESC="This is the required description"

For more information on the notation for replacing variables in the configuration file, please refer to the build command Detailed Manual.

Deploying the image

The output image can be deployed in three ways to the board:

  • Using Toradex Easy Installer
  • Directly on the board through SSH
  • Through Torizon OTA server.

Toradex Easy Installer

Toradex Easy Installer Tool allows users to install an image into the internal flash memory of Toradex modules in an extremely simple way.

Copy the output image into a USB stick and follow the steps in the Toradex Easy Installer article linked above. The copy of the output image can be done with the command below:

$ cp -a torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM ~/media/user/myUSBstick

Where media/user/myUSBstick is, in my case, the path to USB stick mount point.

Before image deployment (SSH and Torizon Platform)

Before deployment in the next two cases, the output image needs to be unpacked, using the command below:

$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

For more details on how this command works, please check the images unpack command in the commands manual.

Directly, through SSH

To deploy it directly to the board, through SSH, use the command below:

$ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot

For more details on how this command works, please check the deploy command in the commands manual.

Torizon Platform Remote Update Service

To deploy it to the Torizon Platform Remote Update Service, use the command below:

$ torizoncore-builder push --credentials credentials.zip base

For more details on how this command works, please check the push command in the commands manual.

This way does not support the deployment of the pre-provisioned containers, so their Docker Compose file will need to be deployed separately in the server. To deploy the image from the Torizon Platform Remote Update Service to the board, please follow the steps in the OTA Web Interface article.

To obtain credentials (credentials.zip file) and to obtain more information on how to sign and securely deploy a custom image using Over-the-Air (OTA) updates, check the Signing and pushing the image to Torizon OTA article.

Additional Resources

Proceed to the Documentation tab on this page.

TorizonCore Builder - Commands Manual

TorizonCore Builder and OSTree

TorizonCore Builder interacts with the OSTree library to capture/apply changes made on a filesystem. We develop TorizonCore Builder in a way that OSTree is abstracted as much as possible for our customers.

OSTree is a library, which official name is libostree, which handles updates for the entire filesystem tree (the whole Linux root filesystem). An essential aspect of OSTree is that it uses a “git-like” model for committing and downloading bootable filesystem trees. The library is a key technology used by Torizon OTA.

The torizoncore-builder command

Note: TorizonCore Builder tool can work either by providing the working directory (capable of carrying Linux filesystem metadata) or Docker volumes for intermediate data storage. So all commands are explained with both approaches.

The usage of the torizoncore-builder command (or, more precisely, alias) can be shown by its --help switch:

Usage

$ torizoncore-builder --help
usage: torizoncore-builder [-h] [--verbose] [--log-level LOG_LEVEL] [--log-file LOG_FILE] [--storage-directory STORAGE_DIRECTORY] [-v]
                           {build,bundle,combine,deploy,dt,dto,images,isolate,kernel,ostree,push,splash,union} ...

TorizonCore Builder is an utility that allows to create customized TorizonCore OSTree commits and Toradex Easy Installer images without rebuilding the complete operating system.

optional arguments:
  -h, --help            show this help message and exit
  --verbose             show more output
  --log-level LOG_LEVEL
                        set global log level (debug, info, warning, error, critical)
  --log-file LOG_FILE   write logs to a file instead of console
  --storage-directory STORAGE_DIRECTORY
                        path to internal storage. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr)
  -v, --version         show program's version number and exit

Commands:
  {build,bundle,combine,deploy,dt,dto,images,isolate,kernel,ostree,push,splash,union}
    build               Customize a Toradex Easy Installer image based on settings specified via a configuration file.
    bundle              Create container bundle from a Docker Compose file. Can be used to combine with a TorizonCore base image.
    combine             Combines a container bundle with a specified Toradex Easy Installer image.
    deploy              Deploy the current image as a Toradex Easy Installer image
    dt                  Manage device trees
    dto                 Manage device tree overlays
    images              Manage Toradex Easy Installer Images.
    isolate             capture /etc changes.
    kernel              Manage and modify TorizonCore Linux Kernel.
    ostree              OSTree operational commands
    push                Push branch to OTA server
    splash              change splash screen
    union               Create a commit out of isolated changes for unpacked Toradex Easy Installer Image

Learn more on https://developer.toradex.com/knowledge-base/torizoncore-builder-tool

The various arguments listed are common to all commands. For example, one could enable the debug messages during the dt status command by doing:

$ torizoncore-builder --log-level debug dt status

In the following items, we describe the commands offered by the tool (in alphabetical order).

The Build Command

build provides full customization of an image as a single step based on a configuration file. Please refer to the TorizonCore Builder Tool “build” command article for a few examples and a detailed explanation of the command.

Usage
$ torizoncore-builder build --help
usage: torizoncore-builder build [-h] [--create-template] [--file CONFIG] [--force]
                                 [--set ASSIGNMENT] [--no-subst]

optional arguments:
  -h, --help         show this help message and exit
  --create-template  Request that a template file be generated with the name defined by --file;
                     dump to standard output if file name is set to '-'.
  --file CONFIG      Specify location of the build configuration file (default: tcbuild.yaml).
  --force            Force program output (remove output directory before starting the build
                     process).
  --set ASSIGNMENT   Assign values to variables (e.g. VER="1.2.3"). This can be used multiple
                     times.
  --no-subst         Disable the variable substitution feature.

The Bundle Command

This creates a tar archive of the container images referenced by the provided Docker Compose file and stores that archive in a bundle directory (which, by default, is named "bundle", but this can be changed via the --bundle-directory parameter). The bundle directory can later be provided to the combine command to create a Toradex Easy Installer image containing pre-installed container images. The Toradex Easy Installer tool copies containers data to appropriate places in rootfs while flashing the image on the target board.

Usage

$ torizoncore-builder bundle --help
usage: torizoncore-builder bundle [-h] [--bundle-directory BUNDLE_DIRECTORY] [--force]
                                  [--platform PLATFORM] [--login USERNAME PASSWORD]
                                  [--login-to REGISTRY USERNAME PASSWORD]
                                  [--dind-param DIND_PARAM]
                                  [compose_file]

positional arguments:
  compose_file          Compose file to be processed (REQUIRED); commonly named 'docker-
                        compose.yml'.

optional arguments:
  -h, --help            show this help message and exit
  --bundle-directory BUNDLE_DIRECTORY
                        Container bundle directory
  --force               Force program output (remove output directory before starting the bundle
                        generation process).
  --platform PLATFORM   Default platform for fetching container images when multi-platform images
                        are specified in the compose file (e.g. linux/arm/v7 or linux/arm64).
  --login USERNAME PASSWORD
                        Request that the tool logs in to the default [Docker Hub] registry using
                        specified USERNAME and PASSWORD.
  --login-to REGISTRY USERNAME PASSWORD
                        Request that the tool logs in to registry REGISTRY using specified
                        USERNAME and PASSWORD (can be employed multiple times).
  --dind-param DIND_PARAM
                        Parameter to forward to the Docker-in-Docker container executed by the
                        tool (can be employed multiple times). The parameter will be processed by
                        the Docker daemon (dockerd) running in the container. Please see Docker
                        documentation for more information.

NOTE: following switches have been removed: --docker-username, --docker-password, --registry,
--host-workdir and --file (-f); please review your command line if using any of them.
Example

In its most basic form the command would be run simply as:

$ torizoncore-builder bundle docker-compose.yml

If the Docker Compose file references multi-platform images, then the --platform parameter would be required such as:

$ torizoncore-builder bundle docker-compose.yml --platform linux/arm/v7

The parameter --dind-param allows you to pass parameters to the Docker-in-Docker (DIND) container instance that torizoncore-builder uses to fetch images. One case where this is useful is when your Docker Compose file references insecure registries, in which case the DIND container needs to be informed about the registries to be accessed insecurely. In this example, registry running on machine with IP address 192.168.0.30, port 5001 is to be accessed in this manner:

$ torizoncore-builder bundle docker-compose.yml --platform linux/arm/v7 --dind-param="--insecure-registry=192.168.0.30:5001"

The Combine Command

combine command is used to create Toradex Easy Installer image of Torizon by combining user-provided Toradex Easy Installer image of Torizon (either base Torizon Image OR output (output image) from deploy command) with bundled Docker container(s) from bundle command.

Usage

$ torizoncore-builder combine --help
usage: torizoncore-builder combine [-h] [--bundle-directory BUNDLE_DIRECTORY] [--image-name IMAGE_NAME]
                                   [--image-description IMAGE_DESCRIPTION] [--image-licence LICENCE_FILE] 
                                   [--image-release-notes RELEASE_NOTES_FILE] IMAGE_DIRECTORY OUTPUT_DIRECTORY

positional arguments:
  IMAGE_DIRECTORY       Path to TorizonCore Toradex Easy Installer source image, which needs to be updated with docker bundle.
  OUTPUT_DIRECTORY      Path to combined TorizonCore Toradex Easy Installer image, which needs to be updated with docker bundle.

optional arguments:
  -h, --help            show this help message and exit
  --bundle-directory BUNDLE_DIRECTORY
                        Container bundle directory
  --image-name IMAGE_NAME
                        Image name to be used in Easy Installer image json
  --image-description IMAGE_DESCRIPTION
                        Image description to be used in Easy Installer image json
  --image-licence LICENCE_FILE
                        Licence file which will be shown on image installation
  --image-release-notes RELEASE_NOTES_FILE
                        Release notes file which will be shown on image installation

NOTE: the switches --image-directory and --output_directory have been removed.

The Deploy Command

deploy takes a branch(containing the commit for changes) from the union command, make it default deployment. Deployment can be either set on an unpacked Toradex Easy Installer image of Torizon or directly on the target board after creating an ssh session.

Suppose deployment is to be set on the unpacked Toradex Easy Installer image of Torizon. In that case, it makes a directory (inside the bind-mounted working directory) with a user-provided name and stores the Torizon image in it as output.

If pre-installed container(s) are needed, the combine command is necessary to create a new Toradex Easy Installer image of Torizon with a bundled container(s) from the bundle command.

Usage

Note: Output directory is not needed to be created.

$ torizoncore-builder deploy --output-directory <name of directory to store Torizon image> <name or checksum of branch from Union command>

If the deployment is to be set on the target board, the user needs to provide hostname or IP, username, and password.

$ torizoncore-builder deploy --remote-host <IP or hostname(.local) of target board> --remote-username <username of target board> --remote-password <password of user on target board> <name or checksum of branch from Union command>

Arguments

--output-directory: directory to store Toradex Easy Installer image of Torizon with deployment set to branch from union command. The output directory should not be created by the user.

--remote-host: remote target machine IP or hostname (.local)

--remote-username: username of the remote target machine

--remote-password: user password of the remote target machine

--reboot: reboot remote target machine after deploying default: do not reboot after deployment

--deploy-sysroot-directory (optional): directory to store the intermittent deployment sysroot (used internally by the tool). default: docker volume mounted at /deploy

Note: OSTree needs to be able to write extended attributes in this directory. Better to be used without Docker volume option.

Positional Argument

branch name/checksum to be deployed from union command default: active deployment in OSTree repo

Example

with Docker volume

To create an output image

$ torizoncore-builder deploy --output-directory <directory to store Toradex Easy Installer image of Torizon with deployment set to provided reference> <name or checksum of a branch within unpacked OSTree repo (output from union command or any other branch)>

To set branch for deployment on the remote target machine.

$ torizoncore-builder deploy <name or checksum of branch within unpacked OSTree repo (output from union command or any other branch)> --remote-host <IP or hostname(.local) or remote target machine> --remote-username <user name of remote target machine> --remote-password <user password of remote target machine> --reboot <indicates reboot machine after deploying>
without Docker volumes

To create an output image:

$ torizoncore-builder --storage-directory <Path to store intermediate data. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr)> deploy <name or checksum of the branch within unpacked OSTree repo (output from union command or any other branch)> --output-directory <directory to store Toradex Easy Installer image of Torizon image with deployment set to provided reference> --deploy-sysroot-directory <directory to store the intermittent deployment sysroot>

To set branch for deployment on a remote target machine

$ torizoncore-builder deploy <name or checksum of branch within unpacked OSTree repo (output from union command or any other branch)> --remote-host <IP or hostname(.local) or remote target machine> --remote-username <user name of remote target machine> --remote-password <password of remote target machine> --reboot <indicates reboot machine after deploying>

The Dt (Device Tree) Command

dt has subcommands to build and apply custom device trees to devices. Refer to Device Tree Overlays on Torizon for details on usage.

 $ torizoncore-builder dt --help
usage: torizoncore-builder dt [-h] {status,checkout,apply} ...

Manage application of device trees.

optional arguments:
  -h, --help            show this help message and exit

Commands::
  {status,checkout,apply}
    status              Show the current device tree
    checkout            Checkout the Toradex device tree and overlay repository
    apply               Compile and enable a device tree

dt checkout

The dt checkout is simply a convenience wrapper for retrieving the Toradex device tree and overlays repository at https://github.com/toradex/device-trees. This repository contains sources for common combinations of device trees and overlays compatible with Toradex SOMs and carrier boards. The repository is checked out in subdirectory device-trees in torizoncore-builder's working directory.

Usage
$ torizoncore-builder dt checkout

dt apply

Compiles a device tree source file and, on success, stores the compiled device tree blob to be deployed as the current device tree of the deployed image (see dt deploy).

Usage
$ torizoncore-builder dt apply -h
usage: torizoncore-builder dt apply [-h] [--include-dir DIR] DEVICE_TREE

Compile and enable a device tree

positional arguments:
  DEVICE_TREE        Path to the device tree source file

optional arguments:
  -h, --help         show this help message and exit
  --include-dir DIR  Search directory for include files during device tree compilation.
                     Can be passed multiple times. If absent, defaults to 'device-trees/include'.
Example
$ torizoncore-builder dt apply device-trees/dts-arm64/imx8qxp-colibri-aster.dts
<...>
/tmp/tmpiyorpx7w: Device Tree Blob version 17, size=120412, boot CPU=0, string block size=7360, DT structure block size=112996
Device tree imx8qxp-colibri-aster.dtb successfully applied.

dt status

Shows the current device tree of the image, if any. Note: On some devices, the device tree is selected at boot time according to characteristics of the hardware. In such cases dt status does not show the enabled device tree.

Example
$ torizoncore-builder dt status
Current device tree is: imx8qxp-colibri-aster.dtb

The Dto (Device Tree Overlay) Command

dto has subcommands to build and apply custom overlays to the current device tree. Refer to Device Tree Overlays on Torizon for details on usage.

$ torizoncore-builder dto --help
usage: torizoncore-builder dto [-h] {apply,list,status,remove,deploy} ...

Manage device tree overlays

optional arguments:
  -h, --help            show this help message and exit

Commands:
  {apply,list,status,remove,deploy}
    apply               Apply a device tree overlay
    list                List the device tree overlays compatible with the current device tree
    status              List the applied device tree overlays
    remove              Remove a device tree overlay
    deploy              Deploy a device tree overlay in the device

dto apply

Compiles a device tree overlay source file. On success, test if it's applicable against the current device tree (+ other previously applied overlays). On success, stores the compiled device tree overlay blob to be deployed as part of the set of overlays that are applied to the current device tree of the deployed image during boot.

Usage
$ torizoncore-builder dto apply --help
usage: torizoncore-builder dto apply [-h] [--include-dir DIR] [--device-tree FILE] [--force] OVERLAY

Apply a further overlay to the current device tree.

positional arguments:
  OVERLAY             Path to the overlay source file

optional arguments:
  -h, --help          show this help message and exit
  --include-dir DIR   Search directory for include files during overlay compilation. Can be passed multiple times.
                      If absent, defaults to 'device-trees/include'.
  --device-tree FILE  Test overlay application over this device tree blob instead of the current device tree.
  --force             Apply the overlay even on failure checking it against the current device tree.
Example
$ torizoncore-builder dto apply device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts
/tmp/tmpfmxpe871: Device Tree Blob version 17, size=2323, boot CPU=0, string block size=275, DT structure block size=1992
Overlay 'device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts' compiles successfully.
/tmp/tmpp64__jp9: Device Tree Blob version 17, size=121434, boot CPU=0, string block size=7462, DT structure block size=113916
Overlay colibri-imx8x_dsihdmi_overlay.dtbo successfully applied.

dto list

Scan subdirectory device-trees/overlays of TorizonCore Builder's working directory for overlay source files that are declared to be compatible to the current device tree.

Usage
$ torizoncore-builder dto list --help
usage: torizoncore-builder dto list [-h] [--device-tree FILE]

List overlays compatible to the current device tree.

optional arguments:
  -h, --help          show this help message and exit
  --device-tree FILE  Check for overlay compatibility against this device tree blob instead of the current device tree.
Example
 $ torizoncore-builder dto list
Overlays compatible with device tree imx8qxp-colibri-aster.dtb:
- device-trees/overlays/colibri-imx8x_ad7879_overlay.dts
- device-trees/overlays/colibri-imx8x_atmel-mxt_overlay.dts
- device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts
- device-trees/overlays/colibri-imx8x_parallel-rgb_overlay.dts
- device-trees/overlays/display-dpi-lt170410_overlay.dts

dto status

Shows the applied device tree overlays that will be deployed (see dt deploy).

Usage
 $ torizoncore-builder dto status --help
usage: torizoncore-builder dto status [-h]

List the applied overlays.

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder dto status
Enabled overlays over device tree imx8qxp-colibri-aster.dtb:
- colibri-imx8x_dsihdmi_overlay.dtbo
- colibri-imx8x_atmel-mxt_overlay.dtbo

dto remove

Remove a device tree overlay blob from the set of applied overlays.

Usage
$ torizoncore-builder dto remove --help
usage: torizoncore-builder dto remove [-h] [--all] [OVERLAY]

Remove an overlay that is not yet deployed.

positional arguments:
  OVERLAY     Name of the device tree overlay blob

optional arguments:
  -h, --help  show this help message and exit
  --all       Remove all overlays instead.
Example
$ torizoncore-builder dto remove colibri-imx8x_dsihdmi_overlay.dtbo

dto deploy

Deploy a device tree overlay in the device with a single command. This command will download a TEZI image based on the running device, unpack this image, apply the Device Tree overlay, create a union branch, and deploy the image to the device.

Usage
$ torizoncore-builder dto deploy --help
usage: torizoncore-builder dto deploy [-h] --remote-host REMOTE_HOST --remote-username REMOTE_USERNAME --remote-password REMOTE_PASSWORD [--reboot] [--mdns-source MDNS_SOURCE] [--include-dir DIR]
                                      [--force] [--device-tree FILE] [--clear]
                                      OVERLAY [OVERLAY ...]

Deploy a device tree overlay in the device

positional arguments:
  OVERLAY               Path to the device tree overlay source file(s)

optional arguments:
  -h, --help            show this help message and exit
  --remote-host REMOTE_HOST
                        Name/IP of remote machine
  --remote-username REMOTE_USERNAME
                        User name of remote machine
  --remote-password REMOTE_PASSWORD
                        Password of remote machine
  --reboot              Reboot device after deploying device tree overlay(s)
  --mdns-source MDNS_SOURCE
                        Use the given IP address as mDNS source. This is useful when multiple interfaces are used, and mDNS multicast requests are sent out the wrong network interface.
  --include-dir DIR     Search directory for include files during overlay compilation. Can be passed multiple times. If absent, defaults to 'device-trees/include'
  --force               Apply the overlay even on failure checking it against a device tree.
  --device-tree FILE    Test the overlay against a specific device tree.
  --clear               Remove all currently applied device tree overlays.
Example
$ torizoncore-builder dto deploy --remote-host <Hostname/IP address of target board> --remote-username <User on target board> --remote-password <Password of user on target board> --force --reboot device-trees/overlays/display-lt161010_overlay.dts 
Downloading image from: https://artifacts.toradex.com/artifactory/torizoncore-oe-prod-frankfurt/dunfell-5.x.y/release/1/verdin-imx8mm/torizon/torizon-core-docker/oedeploy/torizon-core-docker-verdin-imx8mm-Tezi_5.1.0+build.1.tar

The download may take some time. Please wait...
Download Complete!

Unpacking Toradex Easy Installer image.
Copying Toradex Easy Installer image.
Unpacking TorizonCore Toradex Easy Installer image.
Importing OSTree revision 8c514a595469283bb69603f2fa32395b5dbbc3d80a8f9d824dc661f1b68c4d82 from local repository...
0 metadata, 0 content objects imported; 0 bytes content written                                                                                                                                            
Unpacked OSTree from Toradex Easy Installer image:
  Commit checksum: 8c514a595469283bb69603f2fa32395b5dbbc3d80a8f9d824dc661f1b68c4d82
  TorizonCore Version: 5.1.0+build.1
'device-trees' directory already exists
'display-lt161010_overlay.dts' compiles successfully.
warning: --force was used, bypassing checking overlays against the device tree.
Overlay display-lt161010_overlay.dtbo successfully applied.
Commit 40aec4200054c9865aa6e0b13acdeb9ed63ac5a91e9276800e4a22850f42d84b has been generated for changes and ready to be deployed.
Pulling OSTree with ref dto_deploy (checksum 40aec4200054c9865aa6e0b13acdeb9ed63ac5a91e9276800e4a22850f42d84b) from local archive repository...
Starting http server to serve OSTree.
Starting OSTree pull on the device...
Deploying new OSTree on the device...
Deploying successfully finished.
Device reboot initiated...

The Images Command

images gets the base Toradex Easy Installer image of Torizon and unpacks the rootfs (OSTree filesystem) to a Docker volume. This command has the following subcommands and options:

$ torizoncore-builder images --help
usage: torizoncore-builder images [-h] [--remove-storage] {download,unpack} ...

optional arguments:
  -h, --help         show this help message and exit
  --remove-storage   Automatically clear storage prior to unpacking a new Easy Installer image.

Commands:
  {download,unpack}
    download         Download image from Toradex Artifactory and unpack it.
    unpack           Unpack a specified Toradex Easy Installer image so it can be modified with
                     union subcommand.

images download

Warning: Pre-production level Easy Installer images (monthly, nightly) are pruned from the Toradex Artifacts server on a regular basis. This may result in download not being able to locate the image.

Connects to a remote Torizon device and via the version information determines the source Easy Installer image. The deduced image is then downloaded from the Toradex Artifacts repository into the user's working directory. Finally, the command will unpack the rootfs (OSTree filesystem) of the downloaded to a Docker volume.

Usage

$ torizoncore-builder images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>

Arguments

--remote-host: IP or hostname of the target board

--remote-username: username of the target board, to be used for creating ssh session.

--remote-password: user password of the target board, to be used for creating ssh session

Example

With docker volume
$ torizoncore-builder images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>
Without docker volume

Additional argument --storage-directory specified with alias torizoncore-builder, sets the path (capable of handling Linux filesystem metadata) to be used to store intermediate data. It will be used to store unpacked OSTree rootfs as a result of this command.

$ torizoncore-builder --storage-directory <path to user storage directory> images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>

images unpack

Takes a user-provided Toradex Easy Installer image and unpacks the rootfs (OSTree filesystem) to a Docker volume. It is not necessary to use this command if you have previously used images download.

Usage
$ torizoncore-builder images unpack <base Toradex Easy Installer image of Torizon>

Example

Note: If there is already an unpacked OSTree filesystem, the user is prompted to delete it because currently, a single OSTree filesystem can be handled. This prompt can be automated via the --remove-storage argument of the images command.

With docker volume
$ torizoncore-builder images unpack <path to base Toradex Easy Installer image of Torizon in the working directory>
Without Docker Volume

Additional argument --storage-directory specified with alias torizoncore-builder, sets the path(capable of handling Linux filesystem metadata) to be used to store intermediate data. It will be used to store unpacked OSTree rootfs as a result of this command.

$ torizoncore-builder --storage-directory <path to user storage directory> images unpack <path to base Toradex Easy Installer image of Torizon in working directory>

The Isolate Command

isolate get configuration changes(modifications/addition/deletion/files and directories permissions and ownership) from the target board. It creates an ssh session with the target board to get changes and store them within Docker volume on the host.

isolate command gets the configuration (/etc) changes (modifications, additions, deletions, files and directories permissions and ownership) from the user target module. It ignores the following files/directories for any change.

'gshadow', 'machine-id', 'group', 'shadow', 'systemd/system/sysinit.target.wants/run-postinsts.service',
'ostree/remotes.d/toradex-nightly.conf', 'docker/key.json', '.updated', '.pwd.lock', 'group-',
'gshadow-', 'hostname', 'ssh/ssh_host_rsa_key', 'ssh/ssh_host_rsa_key.pub', 'ssh/ssh_host_ecdsa_key',
'ssh/ssh_host_ecdsa_key.pub','ssh/ssh_host_ed25519_key','ssh/ssh_host_ed25519_key.pub',
'ipk-postinsts', 'fw_env.conf'

Files and directories permissions and ownership will be saved in the /etc/.tcattr metadata file together with all other isolated files.

Usage

$ torizoncore-builder isolate --help
usage: torizoncore-builder isolate [-h] [--changes-directory CHANGES_DIR] [--force]
                                   --remote-host REMOTE_HOST --remote-username REMOTE_USERNAME 
                                   --remote-password REMOTE_PASSWORD [--mdns-source MDNS_SOURCE]

optional arguments:
  -h, --help            show this help message and exit
  --changes-directory CHANGES_DIR
                        Directory to save the isolated changes from the device. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr). If not passed,
                        defaults to a directory in the storage volume.
  --force               Force removal of storage changes directory
  --remote-host REMOTE_HOST
                        name/IP of remote machine
  --remote-username REMOTE_USERNAME
                        user name of remote machine
  --remote-password REMOTE_PASSWORD
                        password of remote machine
  --mdns-source MDNS_SOURCE
                        Use the given IP address as mDNS source. This is useful when multiple interfaces are used, and mDNS multicast requests are sent out the wrong network interface.

Example

You can find the example and detailed instructions on how to use isolate at the Capturing Changes in the Configuration of a Board on TorizonCore article.

The Kernel Command

kernel is used to make modifications related to the Linux kernel of TorizonCore.

$ torizoncore-builder kernel --help
usage: torizoncore-builder kernel [-h] {build_module,set_custom_args,get_custom_args,clear_custom_args} ...

optional arguments:
  -h, --help            show this help message and exit

Commands:
  {build_module,set_custom_args,get_custom_args,clear_custom_args}
    build_module        Build the kernel module at the provided source directory.
    set_custom_args     Set the TorizonCore kernel arguments.
    get_custom_args     Get the TorizonCore kernel arguments.
    clear_custom_args   Clear the TorizonCore kernel arguments.

kernel buildmodule

Build a kernel module from sources and stores the binary to be deployed to a TorizonCore system. Can also configure built kernel modules to be loaded on system boot.

Usage
$ torizoncore-builder kernel build_module --help
usage: torizoncore-builder kernel build_module [-h] [--autoload] [SRC_DIR]

positional arguments:
  SRC_DIR     Path to directory with kernel module source code.

optional arguments:
  -h, --help  show this help message and exit
  --autoload  Configure kernel module to be loaded on startup.
Arguments

--autoload: Configures all built kernel modules to be loaded on system boot.

Example

You can find an example and detailed instructions on how to use kernel build_module at the Building External Kernel Modules With Torizon article.

kernel setcustomargs

Define custom arguments to be passed to the kernel of the image to be deployed. Later executions of this command will completely override the set of arguments previously defined. At runtime, the custom arguments are appended to the default kernel arguments defined in the image. This command is a wrapper around dto apply and, as such, it applies a device-tree overlay (named custom-kargs_overlay.dtbo) to the image.

Refer to Customizing Kernel Arguments in Torizon for details on usage.

Warning: Since the custom kernel arguments are stored in an overlay, the command dto status will display that overlay as part of its output. Also removing the corresponding overlay with dto remove will clear any custom kernel arguments (though the recommended way of doing that is via kernel clear_custom_args).

Usage
$ torizoncore-builder kernel set_custom_args --help
usage: torizoncore-builder kernel set_custom_args [-h] KERNEL_ARGS [KERNEL_ARGS ...]

positional arguments:
  KERNEL_ARGS  Kernel arguments to be added.

optional arguments:
  -h, --help   show this help message and exit
Example
$ torizoncore-builder kernel set_custom_args "param1=val1" "param2=val2"
'custom-kargs_overlay.dts' compiles successfully.
Overlay custom-kargs_overlay.dtbo successfully applied.
Kernel custom arguments successfully configured with "param1=val1 param2=val2".

kernel getcustomargs

Get the custom arguments to be passed to the TorizonCore kernel of the image to be deployed (as previously set by kernel set_custom_args).

Usage
$ torizoncore-builder kernel get_custom_args --help
usage: torizoncore-builder kernel get_custom_args [-h]

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder kernel get_custom_args
Currently configured custom kernel arguments: "param1=val1 param2=val2".

kernel clearcustomargs

Clear the custom arguments to be passed to the kernel (previously set by kernel set_custom_args). This will basically remove the overlay custom-kargs_overlay.dtbo from the image to be deployed.

Usage
$ torizoncore-builder kernel clear_custom_args --help
usage: torizoncore-builder kernel clear_custom_args [-h]

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder kernel clear_custom_args
Custom kernel arguments successfully cleared.

The OSTree Command

OSTree related operations.

Usage

$ torizoncore-builder ostree --help
usage: torizoncore-builder ostree [-h] {serve} ...

optional arguments:
  -h, --help  show this help message and exit

Commands:
  {serve}
    serve     Serve OSTree on the local network using http

ostree serve

Serve an OSTree repository via HTTP.

Usage

$ torizoncore-builder ostree serve --help
usage: torizoncore-builder ostree serve [-h] [--ostree-repo-directory OSTREE_REPO_DIRECTORY]

optional arguments:
  -h, --help            show this help message and exit
  --ostree-repo-directory OSTREE_REPO_DIRECTORY
                        Path to the OSTree repository to serve (defaults to internal archive repository)

The Push Command

The push command will take an image referenced by an OSTree repo or a docker-compose.yml file, will sign with the certificates from a given Torizon OTA account credentials, and will push it to the same Torizon OTA account for an OTA deployment.

Usage

torizoncore-builder push [-h] --credentials CREDENTIALS [--repo OSTREE] [--hardwareid HARDWAREIDS] [--package-name TARGET] [--package-version VERSION] [--verbose] [REF]

Arguments

--credentials: Relative path to the credentials.zip file obtained by the user Torizon OTA account.

--repo (optional): OSTree repository to push from.

--hardwareid (optional): Hardware IDs to use when creating and signing targets.json.

--package-name (optional): Package name for docker-compose file.

--package-version (optional): Package version for docker-compose file.

--REF: OSTree reference or docker-compose file to push to Torizon OTA.

Example

You can find the example and detailed instructions on how to use push at Signing and pushing the image to Torizon OTA.

The Splash Command

Set a splash screen (image shown during boot) for the image. Refer to Splash Screen on TorizonCore for details.

Usage

$ torizoncore-builder splash --help
usage: torizoncore-builder splash [-h] SPLASH_IMAGE

positional arguments:
  SPLASH_IMAGE  Path and name of splash screen image (REQUIRED).

optional arguments:
  -h, --help    show this help message and exit

NOTE: the switches --image and --work-dir have been removed.

The Union Command

union makes an OSTree branch (containing the commit for changes) for all changes provided by the user to be made in OSTree rootfs of unpacked base Torizon image. It commits changes made by isolate, splash, dt and kernel commands to the OSTree repo i.e., unpacked Toradex Easy Installer image of Torizon.

Note: If the user has used the union command, the deploy command is necessary to deploy the OSTree branch either on the target machine connected via ssh OR onto the OSTree file system (from unpacked Toradex Easy Installer image of Torizon) and to create a new Toradex Easy Installer image of Torizon.

When using the optional --changes-directory parameter, the user has the possibility to set attributes (permissions and ownership) for files and directories that they would like to have set in the OSTree rootfs. That could be made using the .tcattr file.

Usage

usage: torizoncore-builder union [-h] [--changes-directory CHANGES_DIRS]
                                 [--subject SUBJECT] [--body BODY] UNION_BRANCH

positional arguments:
  UNION_BRANCH          Name of branch containing the changes committed to the unpacked repository (REQUIRED).

optional arguments:
  -h, --help            show this help message and exit
  --changes-directory CHANGES_DIRS
                        Path to the directory containing user changes. Can be specified multiple times!
  --subject SUBJECT     OSTree commit subject. Defaults to TorizonCore Builder [timestamp]
  --body BODY           OSTree commit body message

Setting attributes to files and directories using the ".tcattr" file

When using --changes-directory (or --extra-changes-directory in older versions of the tool), you can create a ".tcattr" file which will set the attributes for files and directories in the OSTree filesystem. Basically, you just have to provide the permissions and ownership for files and directories that you want to be set in your final TorizonCore image. The .tcattr file uses the Linux utility getfacl format, so you can learn more about that format by reading the man getfacl.

Example of creating a ".tcattr" file for an additional changes directory

To set attributes for files and directories under /usr/dir1 in your final image or OSTree rootfs of TorizonCore, you should create, in your host machine, the directory ./extra/usr. Inside that directory, you can have whatever files and or directories you like. For each file and directory, you must create entries in the ./extra/.tcattr file with the desired attributes.

For example, imagine that you want the directory usr/dir1 in your ./extra changes directory to have user id 1000, group id 1000 and that only the owner of this directory has execute, read and write permissions. So, you should create an entry in the ./extra/.tcattr file for that directory using the following steps:

$ cd extra/
$ mkdir -p usr/dir1
$ chown 1000.1000 usr/dir1
$ chmod 700 usr/dir1
$ getfacl -n usr/dir1 >> .tcattr

You should end up with a ".tcattr" file with the content:

./extra/.tcattr
# file: usr/dir1
# owner: 1000
# group: 1000
user::rwx
group::---
other::---

If you want to set more attributes for other files, you just need to follow the same logic and add more entries to the .tcattr file. For example, if you want a ./extra/usr/dir1/file1 having user id 1000, group id 2000 and only read and write execution permissions for the owner and the group of this file but no permissions at all for the others, you can do the following:

$ cd extra/
$ touch usr/dir1/file1
$ chown 1000.2000 usr/dir1/file1
$ chmod 660 usr/dir1/file1
$ getfacl -n usr/dir1/file1 >> .tcattr

And after these two changes, you should have an ./extra/.tcattr file, in your host machine, with the following content:

./extra/.tcattr
# file: usr/dir1
# owner: 1000
# group: 1000
user::rwx
group::---
other::---
 
# file: usr/dir1/file1
# owner: 1000
# group: 2000
user::rw-
group::rw-
other::---

Note: Files and directories not explicitly listed in the ".tcattr" file will be set with default attributes: regular files (user id root, group id root, and 660 permissions) - executable files (user id root, group id root, and 0770 permissions) - directories (user id root, group id root, and 755 permissions).

Note: All .tcattr files will not be committed to the OSTree rootfs, so they will not exist in your final TorizonCore image or OSTree filesystem.

TorizonCore Builder and OSTree

TorizonCore Builder interacts with the OSTree library to capture/apply changes made on a filesystem. We develop TorizonCore Builder in a way that OSTree is abstracted as much as possible for our customers.

OSTree is a library, which official name is libostree, which handles updates for the entire filesystem tree (the whole Linux root filesystem). An essential aspect of OSTree is that it uses a “git-like” model for committing and downloading bootable filesystem trees. The library is a key technology used by Torizon OTA.

The torizoncore-builder command

Note: TorizonCore Builder tool can work either by providing the working directory (capable of carrying Linux filesystem metadata) or Docker volumes for intermediate data storage. So all commands are explained with both approaches.

The usage of the torizoncore-builder command (or, more precisely, alias) can be shown by its --help switch:

Usage

$ torizoncore-builder --help
usage: torizoncore-builder [-h] [--verbose] [--log-level LOG_LEVEL] [--log-file LOG_FILE]
                           [--bundle-directory BUNDLE_DIRECTORY] [--storage-directory STORAGE_DIRECTORY] [-v]
                           {batch,build,bundle,combine,deploy,dt,dto,images,isolate,kernel,push,serve,splash,union} ...

TorizonCore Builder is an utility that allows to create customized TorizonCore OSTree commits and Toradex Easy Installer images without rebuilding the complete operating system.

optional arguments:
  -h, --help            show this help message and exit
  --verbose             show more output
  --log-level LOG_LEVEL
                        set global log level (debug, info, warning, error, critical)
  --log-file LOG_FILE   write logs to a file instead of console
  --bundle-directory BUNDLE_DIRECTORY
                        container bundle directory
  --storage-directory STORAGE_DIRECTORY
                        path to internal storage. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr)
  -v, --version         show program's version number and exit

Commands:
  {batch,build,bundle,combine,deploy,dt,dto,images,isolate,kernel,push,serve,splash,union}
    batch               Automatically downloads a set of Toradex Easy Installer images and adds the specified containers to it.
    build               Customize a Toradex Easy Installer image based on settings specified via a configuration file.
    bundle              Create container bundle from a Docker Compose file. Can be used to combine with a TorizonCore base image.
    combine             Combines a container bundle with a specified Toradex Easy Installer image.
    deploy              Deploy the current image as a Toradex Easy Installer image
    dt                  Manage device trees
    dto                 Manage device tree overlays
    images              Manage Toradex Easy Installer Images.
    isolate             capture /etc changes.
    kernel              Manage and modify TorizonCore Linux Kernel.
    push                Push branch to OTA server
    serve               Serve OSTree on the local network using http
    splash              change splash screen
    union               Create a commit out of isolated changes for unpacked Toradex Easy Installer Image

Learn more on https://developer.toradex.com/knowledge-base/torizoncore-builder-tool

The various arguments listed are common to all commands. For example, one could enable the debug messages during the dt status command by doing:

$ torizoncore-builder --log-level debug dt status

In the following items, we describe the commands offered by the tool (in alphabetical order).

The Bundle Command

This creates a tar archive of the container images referenced by the provided docker-compose file and stores that archive in a bundle directory (which, by default, is named "bundle", but this can be changed via the --bundle-directory parameter). The bundle directory can later be provided to the combine command to create a Toradex Easy Installer image containing pre-installed container images. The Toradex Easy Installer tool copies containers data to appropriate places in rootfs while flashing the image on the target board.

Usage

$ torizoncore-builder bundle --help
usage: torizoncore-builder bundle [-h] [-f COMPOSE_FILE] [--platform PLATFORM] [--host-workdir HOST_WORKDIR]
                                  [--docker-username DOCKER_USERNAME] [--docker-password DOCKER_PASSWORD]
                                  [--registry REGISTRY]

optional arguments:
  -h, --help            show this help message and exit
  -f COMPOSE_FILE, --file COMPOSE_FILE
                        Specify an alternate compose file
  --platform PLATFORM   Specify platform to make sure fetching the correct container image when multi-platform container images are specified (e.g. linux/arm/v7 or linux/arm64)
  --host-workdir HOST_WORKDIR
                        Location where Docker needs to bind mount to to share data between this script and the DIND instance.
  --docker-username DOCKER_USERNAME
                        Optional username to be used to access container image.
  --docker-password DOCKER_PASSWORD
                        Password to be used to access container image.
  --registry REGISTRY   Alternative container registry used to access container image.
Example

In its most basic form the command would be run simply as:

$ torizoncore-builder bundle -f docker-compose.yml

If the Docker Compose file references multi-platform images, then the --platform parameter would be required such as:

$ torizoncore-builder bundle -f docker-compose.yml --platform linux/arm/v7

The Combine Command

combine command is used to create Toradex Easy Installer image of Torizon by combining user-provided Toradex Easy Installer image of Torizon (either base Torizon Image OR output (output image) from deploy command) with bundled Docker container(s) from bundle command.

Usage

$ torizoncore-builder combine --help
usage: torizoncore-builder combine [-h] --image-directory IMAGE_DIRECTORY --output-directory OUTPUT_DIRECTORY
                                   [--image-name IMAGE_NAME] [--image-description IMAGE_DESCRIPTION]
                                   [--image-licence LICENCE_FILE] [--image-release-notes RELEASE_NOTES_FILE]

optional arguments:
  -h, --help            show this help message and exit
  --image-directory IMAGE_DIRECTORY
                        Path to TorizonCore Toradex Easy Installer source image, which needs to be updated with docker bundle.
  --output-directory OUTPUT_DIRECTORY
                        Path to combined TorizonCore Toradex Easy Installer image, which needs to be updated with docker bundle.
  --image-name IMAGE_NAME
                        Image name to be used in Easy Installer image json
  --image-description IMAGE_DESCRIPTION
                        Image description to be used in Easy Installer image json
  --image-licence LICENCE_FILE
                        Licence file which will be shown on image installation
  --image-release-notes RELEASE_NOTES_FILE
                        Release notes file which will be shown on image installation

The Deploy Command

deploy takes a branch(containing the commit for changes) from the union command, make it default deployment. Deployment can be either set on an unpacked Toradex Easy Installer image of Torizon or directly on the target board after creating an ssh session.

Suppose deployment is to be set on the unpacked Toradex Easy Installer image of Torizon. In that case, it makes a directory (inside the bind-mounted working directory) with a user-provided name and stores the Torizon image in it as output.

If pre-installed container(s) are needed, the combine command is necessary to create a new Toradex Easy Installer image of Torizon with a bundled container(s) from the bundle command.

Usage

Note: Output directory is not needed to be created.

$ torizoncore-builder deploy --output-directory <name of directory to store Torizon image> <name or checksum of branch from Union command>

If the deployment is to be set on the target board, the user needs to provide hostname or IP, username, and password.

$ torizoncore-builder deploy --remote-host <IP or hostname(.local) of target board> --remote-username <username of target board> --remote-password <password of user on target board> <name or checksum of branch from Union command>

Arguments

--output-directory: directory to store Toradex Easy Installer image of Torizon with deployment set to branch from union command. The output directory should not be created by the user.

--remote-host: remote target machine IP or hostname (.local)

--remote-username: username of the remote target machine

--remote-password: user password of the remote target machine

--reboot: reboot remote target machine after deploying default: do not reboot after deployment

--deploy-sysroot-directory (optional): directory to store the intermittent deployment sysroot (used internally by the tool). default: docker volume mounted at /deploy

Note: OSTree needs to be able to write extended attributes in this directory. Better to be used without Docker volume option.

Positional Argument

branch name/checksum to be deployed from union command default: active deployment in OSTree repo

Example

with Docker volume

To create an output image

$ torizoncore-builder deploy --output-directory <directory to store Toradex Easy Installer image of Torizon with deployment set to provided reference> <name or checksum of a branch within unpacked OSTree repo (output from union command or any other branch)>

To set branch for deployment on the remote target machine.

$ torizoncore-builder deploy <name or checksum of branch within unpacked OSTree repo (output from union command or any other branch)> --remote-host <IP or hostname(.local) or remote target machine> --remote-username <user name of remote target machine> --remote-password <user password of remote target machine> --reboot <indicates reboot machine after deploying>
without Docker volumes

To create an output image:

$ torizoncore-builder --storage-directory <Path to store intermediate data. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr)> deploy <name or checksum of the branch within unpacked OSTree repo (output from union command or any other branch)> --output-directory <directory to store Toradex Easy Installer image of Torizon image with deployment set to provided reference> --deploy-sysroot-directory <directory to store the intermittent deployment sysroot>

To set branch for deployment on a remote target machine

$ torizoncore-builder deploy <name or checksum of branch within unpacked OSTree repo (output from union command or any other branch)> --remote-host <IP or hostname(.local) or remote target machine> --remote-username <user name of remote target machine> --remote-password <password of remote target machine> --reboot <indicates reboot machine after deploying>

The Dt (Device Tree) Command

dt has subcommands to build and apply custom device trees to devices. Refer to Device Tree Overlays on Torizon for details on usage.

 $ torizoncore-builder dt --help
usage: torizoncore-builder dt [-h] {status,checkout,apply} ...

Manage application of device trees.

optional arguments:
  -h, --help            show this help message and exit

Commands::
  {status,checkout,apply}
    status              Show the current device tree
    checkout            Checkout the Toradex device tree and overlay repository
    apply               Compile and enable a device tree

dt checkout

The dt checkout is simply a convenience wrapper for retrieving the Toradex device tree and overlays repository at https://github.com/toradex/device-trees. This repository contains sources for common combinations of device trees and overlays compatible with Toradex SOMs and carrier boards. The repository is checked out in subdirectory device-trees in torizoncore-builder's working directory.

Usage
$ torizoncore-builder dt checkout

dt apply

Compiles a device tree source file and, on success, stores the compiled device tree blob to be deployed as the current device tree of the deployed image (see dt deploy).

Usage
$ torizoncore-builder dt apply -h
usage: torizoncore-builder dt apply [-h] [--include-dir DIR] DEVICE_TREE

Compile and enable a device tree

positional arguments:
  DEVICE_TREE        Path to the device tree source file

optional arguments:
  -h, --help         show this help message and exit
  --include-dir DIR  Search directory for include files during device tree compilation.
                     Can be passed multiple times. If absent, defaults to 'device-trees/include'.
Example
$ torizoncore-builder dt apply device-trees/dts-arm64/imx8qxp-colibri-aster.dts
<...>
/tmp/tmpiyorpx7w: Device Tree Blob version 17, size=120412, boot CPU=0, string block size=7360, DT structure block size=112996
Device tree imx8qxp-colibri-aster.dtb successfully applied.

dt status

Shows the current device tree of the image, if any. Note: On some devices, the device tree is selected at boot time according to characteristics of the hardware. In such cases dt status does not show the enabled device tree.

Example
$ torizoncore-builder dt status
Current device tree is: imx8qxp-colibri-aster.dtb

The Dto (Device Tree Overlay) Command

dto has subcommands to build and apply custom overlays to the current device tree. Refer to Device Tree Overlays on Torizon for details on usage.

$ torizoncore-builder dto --help
usage: torizoncore-builder dto [-h] {apply,list,status,remove,deploy} ...

Manage device tree overlays

optional arguments:
  -h, --help            show this help message and exit

Commands:knowledge-base/toradex-easy-installer-os-and-demo-images#TorizonCore_Embedded_Linux_Images
  {apply,list,status,remove,deploy}
    apply               Apply a device tree overlay
    list                List the device tree overlays compatible with the current device tree
    status              List the applied device tree overlays
    remove              Remove a device tree overlay
    deploy              Deploy a device tree overlay in the device

dto apply

Compiles a device tree overlay source file. On success, test if it's applicable against the current device tree (+ other previously applied overlays). On success, stores the compiled device tree overlay blob to be deployed as part of the set of overlays that are applied to the current device tree of the deployed image during boot.

Usage
$ torizoncore-builder dto apply --help
usage: torizoncore-builder dto apply [-h] [--include-dir DIR] [--device-tree FILE] [--force] OVERLAY

Apply a further overlay to the current device tree.

positional arguments:
  OVERLAY             Path to the overlay source file

optional arguments:
  -h, --help          show this help message and exit
  --include-dir DIR   Search directory for include files during overlay compilation. Can be passed multiple times.
                      If absent, defaults to 'device-trees/include'.
  --device-tree FILE  Test overlay application over this device tree blob instead of the current device tree.
  --force             Apply the overlay even on failure checking it against the current device tree.
Example
$ torizoncore-builder dto apply device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts
/tmp/tmpfmxpe871: Device Tree Blob version 17, size=2323, boot CPU=0, string block size=275, DT structure block size=1992
Overlay 'device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts' compiles successfully.
/tmp/tmpp64__jp9: Device Tree Blob version 17, size=121434, boot CPU=0, string block size=7462, DT structure block size=113916
Overlay colibri-imx8x_dsihdmi_overlay.dtbo successfully applied.

dto list

Scan subdirectory device-trees/overlays of TorizonCore Builder's working directory for overlay source files that are declared to be compatible to the current device tree.

Usage
$ torizoncore-builder dto list --help
usage: torizoncore-builder dto list [-h] [--device-tree FILE]

List overlays compatible to the current device tree.

optional arguments:
  -h, --help          show this help message and exit
  --device-tree FILE  Check for overlay compatibility against this device tree blob instead of the current device tree.
Example
 $ torizoncore-builder dto list
Overlays compatible with device tree imx8qxp-colibri-aster.dtb:
- device-trees/overlays/colibri-imx8x_ad7879_overlay.dts
- device-trees/overlays/colibri-imx8x_atmel-mxt_overlay.dts
- device-trees/overlays/colibri-imx8x_dsihdmi_overlay.dts
- device-trees/overlays/colibri-imx8x_parallel-rgb_overlay.dts
- device-trees/overlays/display-dpi-lt170410_overlay.dts

dto status

Shows the applied device tree overlays that will be deployed (see dt deploy).

Usage
 $ torizoncore-builder dto status --help
usage: torizoncore-builder dto status [-h]

List the applied overlays.

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder dto status
Enabled overlays over device tree imx8qxp-colibri-aster.dtb:
- colibri-imx8x_dsihdmi_overlay.dtbo
- colibri-imx8x_atmel-mxt_overlay.dtbo

dto remove

Remove a device tree overlay blob from the set of applied overlays.

Usage
$ torizoncore-builder dto remove --help
usage: torizoncore-builder dto remove [-h] [--all] [OVERLAY]

Remove an overlay that is not yet deployed.

positional arguments:
  OVERLAY     Name of the device tree overlay blob

optional arguments:
  -h, --help  show this help message and exit
  --all       Remove all overlays instead.
Example
$ torizoncore-builder dto remove colibri-imx8x_dsihdmi_overlay.dtbo

dto deploy

Deploy a device tree overlay in the device with a single command. This command will download a TEZI image based on the running device, unpack this image, apply the Device Tree overlay, create a union branch, and deploy the image to the device.

Usage
$ torizoncore-builder dto deploy --help
usage: torizoncore-builder dto deploy [-h] --remote-host REMOTE_HOST --remote-username REMOTE_USERNAME --remote-password REMOTE_PASSWORD [--reboot] [--mdns-source MDNS_SOURCE] [--include-dir DIR]
                                      [--force] [--device-tree FILE] [--clear]
                                      OVERLAY [OVERLAY ...]

Deploy a device tree overlay in the device

positional arguments:
  OVERLAY               Path to the device tree overlay source file(s)

optional arguments:
  -h, --help            show this help message and exit
  --remote-host REMOTE_HOST
                        Name/IP of remote machine
  --remote-username REMOTE_USERNAME
                        User name of remote machine
  --remote-password REMOTE_PASSWORD
                        Password of remote machine
  --reboot              Reboot device after deploying device tree overlay(s)
  --mdns-source MDNS_SOURCE
                        Use the given IP address as mDNS source. This is useful when multiple interfaces are used, and mDNS multicast requests are sent out the wrong network interface.
  --include-dir DIR     Search directory for include files during overlay compilation. Can be passed multiple times. If absent, defaults to 'device-trees/include'
  --force               Apply the overlay even on failure checking it against a device tree.
  --device-tree FILE    Test the overlay against a specific device tree.
  --clear               Remove all currently applied device tree overlays.
Example
$ torizoncore-builder dto deploy --remote-host <Hostname/IP address of target board> --remote-username <User on target board> --remote-password <Password of user on target board> --force --reboot device-trees/overlays/display-lt161010_overlay.dts 
Downloading image from: https://artifacts.toradex.com/artifactory/torizoncore-oe-prod-frankfurt/dunfell-5.x.y/release/1/verdin-imx8mm/torizon/torizon-core-docker/oedeploy/torizon-core-docker-verdin-imx8mm-Tezi_5.1.0+build.1.tar

The download may take some time. Please wait...
Download Complete!

Unpacking Toradex Easy Installer image.
Copying Toradex Easy Installer image.
Unpacking TorizonCore Toradex Easy Installer image.
Importing OSTree revision 8c514a595469283bb69603f2fa32395b5dbbc3d80a8f9d824dc661f1b68c4d82 from local repository...
0 metadata, 0 content objects imported; 0 bytes content written                                                                                                                                            
Unpacked OSTree from Toradex Easy Installer image:
  Commit checksum: 8c514a595469283bb69603f2fa32395b5dbbc3d80a8f9d824dc661f1b68c4d82
  TorizonCore Version: 5.1.0+build.1
'device-trees' directory already exists
'display-lt161010_overlay.dts' compiles successfully.
warning: --force was used, bypassing checking overlays against the device tree.
Overlay display-lt161010_overlay.dtbo successfully applied.
Commit 40aec4200054c9865aa6e0b13acdeb9ed63ac5a91e9276800e4a22850f42d84b has been generated for changes and ready to be deployed.
Pulling OSTree with ref dto_deploy (checksum 40aec4200054c9865aa6e0b13acdeb9ed63ac5a91e9276800e4a22850f42d84b) from local archive repository...
Starting http server to serve OSTree.
Starting OSTree pull on the device...
Deploying new OSTree on the device...
Deploying successfully finished.
Device reboot initiated...

The Images Command

images gets the base Toradex Easy Installer image of Torizon and unpacks the rootfs (OSTree filesystem) to a Docker volume. This command has the following subcommands and options:

$ torizoncore-builder images --help
usage: torizoncore-builder images [-h] [--remove-storage] {download,unpack} ...

optional arguments:
  -h, --help         show this help message and exit
  --remove-storage   Automatically clear storage prior to unpacking a new Easy Installer image.

Commands:
  {download,unpack}
    download         Download image from Toradex Artifactory and unpack it.
    unpack           Unpack a specified Toradex Easy Installer image so it can be modified with
                     union subcommand.

images download

Warning: Pre-production level Easy Installer images (monthly, nightly) are pruned from the Toradex Artifacts server on a regular basis. This may result in download not being able to locate the image.

Connects to a remote Torizon device and via the version information determines the source Easy Installer image. The deduced image is then downloaded from the Toradex Artifacts repository into the user's working directory. Finally, the command will unpack the rootfs (OSTree filesystem) of the downloaded to a Docker volume.

Usage

$ torizoncore-builder images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>

Arguments

--remote-host: IP or hostname of the target board

--remote-username: username of the target board, to be used for creating ssh session.

--remote-password: user password of the target board, to be used for creating ssh session

Example

With Docker volume
$ torizoncore-builder images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>
Without Docker volume

Additional argument --storage-directory specified with alias torizoncore-builder, sets the path (capable of handling Linux filesystem metadata) to be used to store intermediate data. It will be used to store unpacked OSTree rootfs as a result of this command.

$ torizoncore-builder --storage-directory <path to user storage directory> images download --remote-host <hostname/IP address of target board> --remote-username <username on target board> --remote-password <password of user on target board>

images unpack

Takes a user-provided Toradex Easy Installer image and unpacks the rootfs (OSTree filesystem) to a Docker volume. It is not necessary to use this command if you have previously used images download.

Usage
$ torizoncore-builder images unpack <base Toradex Easy Installer image of Torizon>

Example

Note: If there is already an unpacked OSTree filesystem, the user is prompted to delete it because currently, a single OSTree filesystem can be handled. This prompt can be automated via the --remove-storage argument of the images command.

With Docker volume
$ torizoncore-builder images unpack <path to base Toradex Easy Installer image of Torizon in the working directory>
Without Docker Volume

Additional argument --storage-directory specified with alias torizoncore-builder, sets the path(capable of handling Linux filesystem metadata) to be used to store intermediate data. It will be used to store unpacked OSTree rootfs as a result of this command.

$ torizoncore-builder --storage-directory <path to user storage directory> images unpack <path to base Toradex Easy Installer image of Torizon in working directory>

The Isolate Command

isolate get configuration changes(modifications/addition/deletion/files and directories permissions and ownership) from the target board. It creates an ssh session with the target board to get changes and store them within Docker volume on the host.

isolate command gets the configuration (/etc) changes (modifications, additions, deletions, files and directories permissions and ownership) from the user target module. It ignores the following files/directories for any change.

'gshadow', 'machine-id', 'group', 'shadow', 'systemd/system/sysinit.target.wants/run-postinsts.service',
'ostree/remotes.d/toradex-nightly.conf', 'docker/key.json', '.updated', '.pwd.lock', 'group-',
'gshadow-', 'hostname', 'ssh/ssh_host_rsa_key', 'ssh/ssh_host_rsa_key.pub', 'ssh/ssh_host_ecdsa_key',
'ssh/ssh_host_ecdsa_key.pub','ssh/ssh_host_ed25519_key','ssh/ssh_host_ed25519_key.pub',
'ipk-postinsts', 'fw_env.conf'

Files and directories permissions and ownership will be saved in the /etc/.tcattr metadata file together with all other isolated files.

Usage

$ torizoncore-builder isolate --help
usage: torizoncore-builder isolate [-h] [--changes-directory CHANGES_DIR] [--force]
                                   --remote-host REMOTE_HOST --remote-username REMOTE_USERNAME 
                                   --remote-password REMOTE_PASSWORD [--mdns-source MDNS_SOURCE]

optional arguments:
  -h, --help            show this help message and exit
  --changes-directory CHANGES_DIR
                        Directory to save the isolated changes from the device. Must be a file system capable of carrying Linux file system metadata (Unix file permissions and xattr). If not passed,
                        defaults to a directory in the storage volume.
  --force               Force removal of storage changes directory
  --remote-host REMOTE_HOST
                        name/IP of remote machine
  --remote-username REMOTE_USERNAME
                        user name of remote machine
  --remote-password REMOTE_PASSWORD
                        password of remote machine
  --mdns-source MDNS_SOURCE
                        Use the given IP address as mDNS source. This is useful when multiple interfaces are used, and mDNS multicast requests are sent out the wrong network interface.

Example

You can find the example and detailed instructions on how to use isolate at the Capturing Changes in the Configuration of a Board on TorizonCore article.

The Kernel Command

kernel is used to make modifications related to the Linux kernel of TorizonCore.

$ torizoncore-builder kernel --help
usage: torizoncore-builder kernel [-h] {build_module,set_custom_args,get_custom_args,clear_custom_args} ...

optional arguments:
  -h, --help            show this help message and exit

Commands:
  {build_module,set_custom_args,get_custom_args,clear_custom_args}
    build_module        Build the kernel module at the provided source directory.
    set_custom_args     Set the TorizonCore kernel arguments.
    get_custom_args     Get the TorizonCore kernel arguments.
    clear_custom_args   Clear the TorizonCore kernel arguments.

kernel buildmodule

Build a kernel module from sources and stores the binary to be deployed to a TorizonCore system. Can also configure built kernel modules to be loaded on system boot.

Usage
$ torizoncore-builder kernel build_module --help
usage: torizoncore-builder kernel build_module [-h] [--autoload] [SRC_DIR]

positional arguments:
  SRC_DIR     Path to directory with kernel module source code.

optional arguments:
  -h, --help  show this help message and exit
  --autoload  Configure kernel module to be loaded on startup.
Arguments

--autoload: Configures all built kernel modules to be loaded on system boot.

Example

You can find an example and detailed instructions on how to use kernel build_module at the Building External Kernel Modules With Torizon article.

kernel setcustomargs

Define custom arguments to be passed to the kernel of the image to be deployed. Later executions of this command will completely override the set of arguments previously defined. At runtime, the custom arguments are appended to the default kernel arguments defined in the image. This command is a wrapper around dto apply and, as such, it applies a device-tree overlay (named custom-kargs_overlay.dtbo) to the image.

Refer to Customizing Kernel Arguments in Torizon for details on usage.

Warning: Since the custom kernel arguments are stored in an overlay, the command dto status will display that overlay as part of its output. Also removing the corresponding overlay with dto remove will clear any custom kernel arguments (though the recommended way of doing that is via kernel clear_custom_args).

Usage
$ torizoncore-builder kernel set_custom_args --help
usage: torizoncore-builder kernel set_custom_args [-h] KERNEL_ARGS [KERNEL_ARGS ...]

positional arguments:
  KERNEL_ARGS  Kernel arguments to be added.

optional arguments:
  -h, --help   show this help message and exit
Example
$ torizoncore-builder kernel set_custom_args "param1=val1" "param2=val2"
'custom-kargs_overlay.dts' compiles successfully.
Overlay custom-kargs_overlay.dtbo successfully applied.
Kernel custom arguments successfully configured with "param1=val1 param2=val2".

kernel getcustomargs

Get the custom arguments to be passed to the TorizonCore kernel of the image to be deployed (as previously set by kernel set_custom_args).

Usage
$ torizoncore-builder kernel get_custom_args --help
usage: torizoncore-builder kernel get_custom_args [-h]

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder kernel get_custom_args
Currently configured custom kernel arguments: "param1=val1 param2=val2".

kernel clearcustomargs

Clear the custom arguments to be passed to the kernel (previously set by kernel set_custom_args). This will basically remove the overlay custom-kargs_overlay.dtbo from the image to be deployed.

Usage
$ torizoncore-builder kernel clear_custom_args --help
usage: torizoncore-builder kernel clear_custom_args [-h]

optional arguments:
  -h, --help  show this help message and exit
Example
$ torizoncore-builder kernel clear_custom_args
Custom kernel arguments successfully cleared.

The OSTree Command

OSTree related operations.

ostree serve

Serve an OSTree repository via HTTP.

Usage

$ torizoncore-builder serve --help
usage: torizoncore-builder serve [-h] [--ostree-repo-directory OSTREE_REPO_DIRECTORY]

optional arguments:
  -h, --help            show this help message and exit
  --ostree-repo-directory OSTREE_REPO_DIRECTORY
                        Path to the OSTree repository to serve (defaults to internal archive repository).

The Push Command

The push command will take an image referenced by an OSTree repo or a docker-compose.yml file, will sign with the certificates from a given Torizon OTA account credentials, and will push it to the same Torizon OTA account for an OTA deployment.

Usage

torizoncore-builder push [-h] --credentials CREDENTIALS [--repo OSTREE] [--hardwareid HARDWAREIDS] [--package-name TARGET] [--package-version VERSION] [--verbose] [REF]

Arguments

--credentials: Relative path to the credentials.zip file obtained by the user Torizon OTA account.

--repo (optional): OSTree repository to push from.

--hardwareid (optional): Hardware IDs to use when creating and signing targets.json.

--package-name (optional): Package name for docker-compose file.

--package-version (optional): Package version for docker-compose file.

--REF: OSTree reference or docker-compose file to push to Torizon OTA.

Example

You can find the example and detailed instructions on how to use push at Signing and pushing the image to Torizon OTA.

The Splash Command

Set a splash screen (image shown during boot) for the image. Refer to Splash Screen on TorizonCore for details.

Usage

$ torizoncore-builder splash --help
usage: torizoncore-builder splash [-h] --image IMAGE [--work-dir WORK_DIR]

optional arguments:
  -h, --help           show this help message and exit
  --image IMAGE        Path and name of splash screen image
  --work-dir WORK_DIR  Working directory

The Union Command

union makes an OSTree branch (containing the commit for changes) for all changes provided by the user to be made in OSTree rootfs of unpacked base Torizon image. It commits changes made by isolate, splash, dt and kernel commands to the OSTree repo i.e., unpacked Toradex Easy Installer image of Torizon.

Note: If the user has used the union command, the deploy command is necessary to deploy the OSTree branch either on the target machine connected via ssh OR onto the OSTree file system (from unpacked Toradex Easy Installer image of Torizon) and to create a new Toradex Easy Installer image of Torizon.

When using the optional --changes-directory parameter, the user has the possibility to set attributes (permissions and ownership) for files and directories that they would like to have set in the OSTree rootfs. That could be made using the .tcattr file.

Usage

usage: torizoncore-builder union [-h] [--changes-directory CHANGES_DIRS]
                                 [--extra-changes-directory EXTRA_CHANGES_DIRS]
                                 --union-branch UNION_BRANCH
                                 [--subject SUBJECT] [--body BODY]

optional arguments:
  -h, --help            show this help message and exit
  --changes-directory CHANGES_DIRS
                        Path to the directory containing user changes. Can be specified multiple times!
  --extra-changes-directory EXTRA_CHANGES_DIRS
                        Additional path with user changes to be committed. Can be specified multiple times!
  --union-branch UNION_BRANCH
                        Name of branch containing the changes committed to the unpacked repo.
  --subject SUBJECT     OSTree commit subject. Defaults to "TorizonCore Builder [timestamp]"
  --body BODY           OSTree commit body message

Setting attributes to files and directories using the ".tcattr" file

When using --changes-directory (or --extra-changes-directory in older versions of the tool), you can create a ".tcattr" file which will set the attributes for files and directories in the OSTree filesystem. Basically, you just have to provide the permissions and ownership for files and directories that you want to be set in your final TorizonCore image. The .tcattr file uses the Linux utility getfacl format, so you can learn more about that format by reading the man getfacl.

Example of creating a ".tcattr" file for an additional changes directory

To set attributes for files and directories under /usr/dir1 in your final image or OSTree rootfs of TorizonCore, you should create, in your host machine, the directory ./extra/usr. Inside that directory, you can have whatever files and or directories you like. For each file and directory, you must create entries in the ./extra/.tcattr file with the desired attributes.

For example, imagine that you want the directory usr/dir1 in your ./extra changes directory to have user id 1000, group id 1000 and that only the owner of this directory has execute, read and write permissions. So, you should create an entry in the ./extra/.tcattr file for that directory using the following steps:

$ cd extra/
$ mkdir -p usr/dir1
$ chown 1000.1000 usr/dir1
$ chmod 700 usr/dir1
$ getfacl -n usr/dir1 >> .tcattr

You should end up with a ".tcattr" file with the content:

./extra/.tcattr
# file: usr/dir1
# owner: 1000
# group: 1000
user::rwx
group::---
other::---

If you want to set more attributes for other files, you just need to follow the same logic and add more entries to the .tcattr file. For example, if you want a ./extra/usr/dir1/file1 having user id 1000, group id 2000 and only read and write execution permissions for the owner and the group of this file but no permissions at all for the others, you can do the following:

$ cd extra/
$ touch usr/dir1/file1
$ chown 1000.2000 usr/dir1/file1
$ chmod 660 usr/dir1/file1
$ getfacl -n usr/dir1/file1 >> .tcattr

And after these two changes, you should have an ./extra/.tcattr file, in your host machine, with the following content:

./extra/.tcattr
# file: usr/dir1
# owner: 1000
# group: 1000
user::rwx
group::---
other::---
 
# file: usr/dir1/file1
# owner: 1000
# group: 2000
user::rw-
group::rw-
other::---

Note: Files and directories not explicitly listed in the ".tcattr" file will be set with default attributes: regular files (user id root, group id root, and 660 permissions) - executable files (user id root, group id root, and 0770 permissions) - directories (user id root, group id root, and 755 permissions).

Note: All .tcattr files will not be committed to the OSTree rootfs, so they will not exist in your final TorizonCore image or OSTree filesystem.

Extra Resources for TorizonCore Customization

See below more documents related to TorizonCore Builder and TorizonCore Customization

Article Brief Description
TorizonCore Builder Tool - Customization for Production Programming and Torizon OTA Create a custom TorizonCore image that you can use in production programming with Toradex Easy Installer, or push your changes to Torizon OTA
TorizonCore Builder Tool “build” command Learn how to fully customize an image as a single step using the build command from TorizonCore Builder
Device Tree Overlays on Torizon How to modify the device tree without having to re-compile it
Device Tree Customization Examples Some examples of Device Tree Customization
Pin Multiplexing - Changing Pin Functionalities in the Linux Device Tree How to create a dts file to change the pin muxing configuration
Building External Kernel Modules With Torizon How to deploy an external kernel module using TorizonCore Builder with an example
Customizing Kernel Arguments in Torizon How to use TorizonCore Builder to customize kernel arguments in Torizon
Splash Screen Customization on TorizonCore Learn how to change the splash screen using initramfs
Touch Screen Calibration (Torizon) How to use weston-touch-calibrator to calibrate the touch
Capturing Changes in the Configuration of a Board on TorizonCore Use TorizonCore Builder to save and reproduce board customization
Pre-provisioning Docker Containers onto a TorizonCore image How to preinstall a Docker Container onto a TorizonCore image using your PC
Reliability on Torizon Learn how to improve reliability on TorizonCore
How to Store Docker Data on an External Storage Device (USB/SD Card) How to storage Docker data on an external device like an SD Card or a USB stick
Persistent Journald Logging How switch from in-RAM to persistent Journald logging
Using Private Registries With Torizon OTA How to deploy private registry credentials to your devices

Torizon 4.0.0

Introduction

Toradex provides Easy Installer to flash images on modules. Developer may make changes in configuration (/etc) of the system and/or work with containers for the end user solution. Now how to replicate the same on other modules? Toradex has developed a new tool, TorizonCore Builder, to let the developer pick those modifications/containers and create another image out of it, keeping the original Toradex Easy Installer image of Torizon as base. TorizonCore Builder is containerized and run on the host system.

This article complies to the Typographic Conventions for Torizon Documentation

Workflow


  • TorizonCore Builder Workflow

    TorizonCore Builder Workflow

Commands

TorizonCore Builder is provided in the form of Docker image and has following commands to create Toradex Easy Installer image of Torizon with required changes

  • Unpack

  • Bundle

  • Isolate

  • Union

  • Deploy

  • Combine

Operation

Firstly, user needs a base Toradex Easy Installer image of Torizon which will be used by TorizonCore Builder container to add user changes in it. User has two options, either to perform all tasks in a “working directory”, which must be a file system capable of carrying Linux file system metadata (such as Unix style file system permissions and extended attributes), by setting appropriate arguments OR let the commands use Docker volumes for intermediate storage to perform the operations on image.

  • The very first command to use is unpack, which unpacks the OSTree file system from user provided Toradex Easy Installer image of Torizon

  • If pre-installation of Docker container(s) onto a Torizon image with Docker engine is required, bundle command is to be used by providing a Docker compose file stating all image(s).

  • If user has made any changes in configuration (/etc), isolate command is to be used to get all those changes (modifications, addition and deletion). It requires ssh connection to the target board.

  • If user has used isolate command, union command is to be used to capture all changes in the form of a commit.

  • If user has used union command, deploy command is to be used to deploy the OSTree commit onto the OSTree file system and create a new Toradex Easy Installer image of Torizon.

  • Combine is used to create a Toradex Easy Installer image of Torizon by combining outputs from deploy command and bundle command i.e. combine user configuration changes and pre-provisioned Docker container(s).

Usage

Warning: Make sure to use an empty working directory and navigate to it. This will be bind mounted to the container this tool is running in and will be used to store the final image and may also use to store temporary files.

Note: Toradex Easy Installer image of Torizon can be downloaded from Toradex Easy Installer

Extract the downloaded image tar file

$ mkdir <directory to extract image into>
$ tar -xf <downloaded image.tar> -C <directory to extract image into>

Basic

For convenience, create alias first

$ alias torizoncore-builder='docker run --rm -it -v $(pwd):/workdir -v storage:/storage -v /deploy --net=host -v /var/run/docker.sock:/var/run/docker.sock torizon/torizoncore-builder:1'

TorizonCore Builder tool uses Docker volumes as persistent storage for intermediate outputs of sub-commands.

TorizonCore Builder tool provides different logging levels, which can be set by using --log-level option. There are five logging levels i.e. debug, info, warning, error and critical and default is set to info. Formatted logging can be enabled by using --verbose option.

Sub-commands of TorizonCore Builder tool can be used as follows

unpack

unpack gets the base Toradex Easy Installer image of Torizon and unpacks the rootfs (OSTree filesystem) to Docker volume

$ torizoncore-builder unpack --image-directory <base Toradex Easy Installer image of Torizon>

Isolate

isolate is used to get configuration changes(modifications/addition/deletion) from the Target board. It creates ssh session with Target board to get changes and store them within Docker volume on host.

$ torizoncore-builder isolate --remote-ip <ip of target board> --remote-username <username on target board> --remote-password <password of user on target board> 

Union

union is used to make an OStree commit of all changes in OSTree repo of unpacked base Torizon image.

$ torizoncore-builder union --union-branch <name to be given to the branch containing the changes commit>

Deploy

deploy is used to take branch from union command, make it default deployment, create the new Torizon image, make directory (inside bind mounted working directory ) with user provided name and store Torizon Image in it.

Note: Output directory is not needed to be created.

$ torizoncore-builder deploy --output-directory <name of directory to store Torizon image>

Bundle

bundle is used to bundle the Docker container(s), which is used by combine command to pre-provision container(s) bundle with Torizon Image.

User provides the docker-compose.yml, platform of target board (linux/arm/v7 or linux/arm64) and working directory of host to store data used for DIND (docker in docker) and bundle command stores the Docker container(s) tar archive in directory named “bundle” inside the working directory.

$ torizoncore-builder bundle --file <path to docker-compose.yml> --platform <target board like linux/arm/v7 or linux/arm64> --host-workdir $(pwd)

Combine

combine command is used to create Torizon image by combining user provided Torizon image (either base Torizon Image OR output of deploy;depending on what is needed) with bundled Docker container(s) from bundle command.

$ torizoncore-builder combine --image-directory <Path to Torizon image> --output-directory <Torizon image with pre-provisioned Docker container(s)>

Example

Creating alias

$ alias torizoncore-builder='docker run -it --rm -v $(pwd):/workdir -v storage:/storage -v /deploy --net=host -v /var/run/docker.sock:/var/run/docker.sock torizon/torizoncore-builder:1'

Unpack base Toradex Easy Installer image of Torizon

$ torizoncore-builder unpack --image-directory colibri-imx7
Copying Toradex Easy Installer image.
Unpacking TorizonCore Toradex Easy Installer image.
Unpacked OSTree from oradex Easy Installer image:
Commit ref: 3c5795a0dcd0ceca552c916f6515a84011425443fd14d118caa09b869dbe4804
TorizonCore Version: 4.0.0-devel-20200607+build.117

Get configuration changes from target board

$ torizoncore-builder isolate --remote-ip 192.168.1.146 --remote-username torizon --remote-password 1234 
isolation command completed

Merge changes to base Toradex Easy Installer image of Torizon

$ torizoncore-builder union --union-branch=changes_committed
Commit d77ed9d5ddec41b7b61f9a571e810de2447133391ee19fe5f34d1a81009967d1 has been generated for changes and ready to be deployed. 

Set branch to deploy at boot. Output image can be deployed on target board. Note: deployed_image is not needed to be created

$ torizoncore-builder deploy --output-directory deployed_image --ref=d77ed9d5ddec41b7b61f9a571e810de2447133391ee19fe5f34d1a81009967d1
Running tar command: tar --xattrs --xattrs-include='*' -cf /workdir/deployed_image/torizon-core-docker-colibri-imx7.ota.tar.xz --xz -S -C /deploy -p .
Using unpacked Toradex Easy Installer image as base:
  Commit ref: 3c5795a0dcd0ceca552c916f6515a84011425443fd14d118caa09b869dbe4804
  TorizonCore Version: 4.0.0-devel-20200607+build.117
  Kernel arguments: quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash ostree=/ostree/boot.1/torizon/d103ac800d2c9a67cce7ec68ad1accb836d766ff959895820deea195f16eaeed/0

Deploying commit ref: d77ed9d5ddec41b7b61f9a571e810de2447133391ee19fe5f34d1a81009967d1
Pulling OSTree with ref d77ed9d5ddec41b7b61f9a571e810de2447133391ee19fe5f34d1a81009967d1 from local repository...
Pulling done.
Deploying OSTree with ref d77ed9d5ddec41b7b61f9a571e810de2447133391ee19fe5f34d1a81009967d1
Deploying done.
Copy rootdirs such as /home from original deployment.
Packing rootfs...
Packing rootfs done.

If required, create tar archive of Docker container(s)

$ torizoncore-builder --bundle-directory bundled_containers  bundle -f docker-compose.yml --platform linux/arm/v7 --host-workdir $(pwd)

Create image with pre-provisioned docker containers. --image-directory can be either base image or output from deploy command.

$ mkdir preprov_cont_image
$ torizoncore-builder --bundle-directory bundled_containers combine --image-directory deployed_image --output-directory preprov_cont_image
Creating copy of TorizonCore source image.
Combining TorizonCore image with Docker Container bundle.
Successfully created a TorizonCore image with Docker Containers preprovisioned in preprov_cont_image

Advanced

User can use TorizonCore Builder tool by either providing the working directory (capable of carrying Linux filesystem metadata) or Docker volumes for intermediate data storage. So all commands are explained with both approaches. Commands can be used with mix of Docker volumes and working directory but that will complicate the usage. So it is recommended to use either working directory or Docker volumes for all the commands.

As it is provided in the form of Docker image, any of the following commands (based on working directory or Docker volumes) can be executed to create an alias to be easy to use.

Usage of all commands explained later are based on the following aliases.

With Docker volumes:

$ alias torizoncore-builder='docker run --rm -it -v $(pwd):/workdir -v storage:/storage -v /deploy --net=host -v /var/run/docker.sock:/var/run/docker.sock torizon/torizoncore-builder:1'

Here, two volumes are being created and mounted at /storage and /deploy of container for internal usage and host current directory is bind mounted at /workdir within container.

With user provided working directory (capable of handling Linux filesystem metadata)

$ alias torizoncore-builder='docker run --rm -it -v $(pwd):/workdir --net=host -v /var/run/docker.sock:/var/run/docker.sock torizon/torizoncore-builder:1'

move to the working directory so the required one gets bind mounted when alias is executed

Base Command

Base command is now alias torizoncore-builder and has below optional options

--verbose: enable formatted logs

--log-level: set log level from options debug, info, warning, error and critical
default: info

--log-file: write logs to specified file instead of console

--bundle-directory: path to directory to store output of bundle command
default: creates and use directory with name bundle in working directory

--storage-directory: path to directory to use as intermediate storage for sub-commands.
default: Docker volume mounted at/storage
Note: do not set it if Docker volumes are to be used.

Unpack

Unpack command is used to unpack the ostree filesystem from user provided Toradex Easy Installer image of Torizon base image.

Arguments

--image-directory : path to base Toradex Easy Installer image of Torizon in the working directory

--sysroot-directory (optional): path to where ostree filesystem from Toradex Easy Installer image of Torizon will be unpacked
default: sysroot directory inside either user provided storage_directory using --storage-directory option OR docker volume (mounted at /storage)

Example
With docker volume
$ torizoncore-builder unpack --image-directory <path to base Toradex Easy Installer image of Torizon in the working directory>  
Without Docker Volume

Additional argument --sysroot-directory, specifying path to store unpacked ostree filesystem, is needed in case of user provided working directory (capable of handling Linux filesystem metadata) is to be used for storage of intermediate data.

$ torizoncore-builder unpack --image-directory <path to base Toradex Easy Installer image of Torizon in working directory> --sysroot-directory <path to where ostree filesystem from Toradex Easy Installer image of Torizon will be unpacked>

If there is already an unpacked ostree filesystem, user is prompted to delete it because currently a single ostree filesystem can be handled.

Isolate

Isolate command is used to get the configuration (/etc) changes (modifications, additions and deletions) from user target module. It ignores following files/directories for any change

'gshadow', 'machine-id', 'group', 'shadow', 'systemd/system/sysinit.target.wants/run-postinsts.service', 'ostree/remotes.d/toradex-nightly.conf', 'docker/key.json', '.updated', '.pwd.lock', 'group-', 'gshadow-', 'hostname', 'ssh/ssh_host_rsa_key', 'ssh/ssh_host_rsa_key.pub', 'ssh/ssh_host_ecdsa_key', 'ssh/ssh_host_ecdsa_key.pub','ssh/ssh_host_ed25519_key','ssh/ssh_host_ed25519_key.pub', 'ipk-postinsts', 'fw_env.conf'

Isolate command establishes a ssh session with the target board, gets all the changes and places them in either user provided directory or within docker volumes.

Arguments

--remote-ip: IP of target board

--remote-username: Username of target board to be used for creating ssh session

--remote-password: Password of target board to be used for creating ssh session

--diff-dir (optional): If docker volume is not to be used, directory for storing configuration changes
default: changes directory inside either user provided storage_directory using --storage-directory option OR docker volume (mounted at /storage)

Example
with docker volume
$torizoncore-builder  isolate --remote-ip <target board ip> --remote-username <target board username> --remote-password <taregtboard password>
without docker volume

Directory path is needed (--diff-dir) to store changes within user provided working directory

$ torizoncore-builder  isolate --remote-ip <target board ip> --remote-username <target board username> --remote-password <taregtboard password> --diff-dir <directory to store changes>

Union

It is to be run after isolate command to commit changes to the ostree filesystem. If changes are stored in docker volume, it is to be run with docker volume and if changes are stored in working directory, same directory is needed to be provided.

Arguments

----union-branch: branch name containing the changes committed to ostree repo of unpacked ostree filesystem

--diff-directory(optional): path (within working directory) to configuration changes picked by isolate command.
default: changes directory inside either user provided storage_directory using --storage-directory option OR docker volume (mounted at /storage)

--sysroot-directory(optional): path to unpacked ostree filesystem from Toradex Easy Installer image of Torizon (same as used in unpack command)
default: sysroot directory inside either user provided storage_directory using --storage-directory option OR docker volume (mounted at /storage)

Example
with docker volume
$ torizoncore-builder union --union-branch <name to be given to the branch>
without docker volume

isolate command is used to store changes in a directory within working directory. Same directory is needed here with --diff-directory option.

$ torizoncore-builder union --union-branch <name to be given to the branch> --diff-directory <path to directory containing configuration changes> --sysroot-directory <path to unpacked ostree filesystem from Toradex Easy Installer image of Torizon>  

Deploy

It takes a branch from union command, sets it up for the boot, packs the rootfs and creates the Toradex Easy Installer image of Torizon ready to be deployed (if no pre-install container(s) are needed).

If pre-installed container(s) are needed, “combine command“ is used to create a new Toradex Easy Installer image of Torizon with bundled container(s) from bundle command.

Arguments

--output-directory: directory to store Toradex Easy Installer image of Torizon with deployment set to branch from union command. It is not needed to be created by the user.

--diff-ref: branch name/checksum to be deployed from union command
default: active deployment in ostree repo

--sysroot-directory (optional): path to unpacked ostree filesystem from Toradex Easy Installer image of Torizon
default: sysroot directory inside either user provided storage_directory using --storage-directory option OR docker volume (mounted at /storage)

--deploy-sysroot-directory (optional): directory to store the intermittent deployment sysroot (used internally by tool).
default: docker volume mounted at /deploy

Example
with docker volume
$ torizoncore-builder deploy --output-directory <directory to store Toradex Easy Installer image of Torizon with deployment set to provided reference> --diff-ref <name or checksum of branch within unpacked ostree repo> 
without docker volumes
$ torizoncore-builder deploy --output-directory <directory to store Toradex Easy Installer image of Torizonm image with deployment set to provided reference> --diff-ref <name or checksum of branch within unpacked ostree repo> --sysroot-directory <path to unpacked ostree filesystem from Toradex Easy Installer image of Torizon> --deploy-sysroot-directory <directory to store the intermittent deployment sysroot>

Bundle

It creates a tar archive of bundle container(s) from provided docker-compose file and stores it in ether "bundle" named directory inside working directory or with user provided name and path using --bundle-directory option of torizonbuilder. Docker-compose file and tar archive is later combined with Toradex Easy Installer image of Torizon using combine command to pre-install containers in Toradex Easy Installer image of Torizon. Toradex EasyInstaller tool copies containers data to appropriate places in rootfs during flashing the image on Target board.

Arguments

--compose-file: path to docker-compose file
default: docker-compose.yml in working directory

--platform: specify platform to make sure fetching the correct container image when multi-platform container images are specified (e.g. linux/arm/v7 or linux/arm64).
default: linux/arm/v7

--host-workdir: directory to save some internal information for DIND(Docker in Docker) and output container tar archive.

Examples
$ torizoncore-builder --bundle-directory <path to store docker bundle tar archive> --compose-file <pat to docker compose file> --platform <platform of container> --host-workdir <path to directory to store continaer bundle>

Combine

It is used to create a Toradex Easy Installer image of Torizon by combining tar archive with user provided Toradex Easy Installer image of Torizon. User provided Toradex Easy Installer image of Torizon can be either image created by deploy command or base Toradex Easy Installer image of Torizon as per need.

Arguments

--image-directory: path to Toradex Easy Installer image of Torizon to be combined with docker bundle

--output-directory: path to combined Toradex Easy Installer image of Torizon with bundled containers

--image-name (optional) : image name to be used in Easy Installer image json

--image-description (optional): image description to be used in Easy Installer image json

--image-licence (optional): path to license file which will be shown on image installation

--image-release-notes (optional): path to release notes file which will be shown on image installation

Example
$ torizoncore-builder --bundle-directory <path to docker bundle tar archive> --image-directory <path to Tezi image to be combined with docker bundle> --output-directory <directory to store combined Tezi image> --image-name <name of image to be set in image json file> --image-description <image desription to be set in image json file> --image-licence <path to license file to be shown at time of installation of image> --image-release-notes <path to release notes file which will be shown on image installation>