TorizonCore Builder - Workflow
Introductionβ
TorizonCore Builder Tool is a tool that allows you to easily generate a custom image of Torizon OS with device configuration changes, a custom device tree and overlays, pre-provisioned containers, external kernel modules, and a splash screen. This final image can then be deployed to the board using SSH, Toradex Easy Installer, or Torizon Cloud updates features. The tool is designed to be used in production programming and on development as well.
In this article, the workflow of the tool will be explained in detail. It takes a Toradex image of Torizon OS and source codes of the device tree overlays, the device tree and the kernel modules, a splash screen image, and a docker-compose file of the application containers as input and generates as output an image of Torizon OS with all these customizations.
*OS Customization Files are device tree and overlays source codes, external kernel modules source files, a splash screen image, and directories with the configuration changes captured from the device. They can also be called version-controlled artifacts, as they should be version-controlled.
Approaches to Use TorizonCore Builderβ
There are two approaches to use TorizonCore Builder, which are in many cases interchangeable and have their own advantages and disadvantages, depending on the way TorizonCore Builder is used:
- Using The Build Command
- Using Standalone Commands
The first approach, with the build
command, allows you to implement all customizations in one step, through a configuration YAML file, making it generally the easier approach. Integrating several customizations, such as including a kernel module, a custom device tree, and an overlay in one image, is just a matter of adding all of them to the customization YAML file.
Also, the build command is stateless, meaning that given the same inputs, it will always produce the same output. This makes this approach the recommended way to use on production programming and on CI/CD pipelines. When using the standalone commands, commands depend on a previous state, and they may produce different outputs depending on the order that they are executed. For example, executing dto apply
and them dto remove
will produce a different output compared to when executing dto remove
and them dto apply
.
However, every time the build
command is executed, it runs a full pipeline (cycle) from scratch: it unpacks an input image, applies all the customizations, and creates an output custom image. If you plan to make incremental changes, it may become too slow and possibly annoying to run the full pipeline every time.
With the second approach, using standalone commands, you don't have to run a full pipeline every time you want to apply another customization, like a different splash screen or a new overlay, to your custom Torizon OS image. This can be useful if you are making incremental customizations on testing and development. As it does not execute a full pipeline every time, making different customization can be easier and faster.
For example, given a previous image of Torizon OS with all the wanted customizations, you can change the splash screen by just applying the new one with the splash
command and executing the merge and deploy steps.
As explained in the next section, even with the build
command, in some cases, standalone auxiliary commands are used in order to have all the necessary input artifacts.
Approach 1: Using The Build Commandβ
- Create a configuration file with the desired settings (the build command can even help you with that);
- Run the auxiliary commands where needed, so all local artifacts possibly referenced by the configuration file are available. Also, depending on the input method, download an Easy Installer image of Torizon from Toradex;
- Customize the configuration file in order to select the desired input image, include the desired customization, and get the desired outputs.
- Run the build command;
- Deploy the output image to the board or to the Torizon Cloud server;
Creating a configuration fileβ
The configuration file follows the YAML format and syntax. The tool expects the configuration file to be made up of three sections, represented by the top-level properties (keys): input, customization, and output, as outlined below. The default name of this file is tcbuild.yaml
.
# Configuration file outline:
input:
# Input section items (required).
customization:
# Customization section items.
output:
# Output section items (required).
The contents of the file are organized in a tree structure where the leaves are configurable properties. In order to set a certain property, you need to make sure all its parent properties are also uncommented (by removing the first # character).
Auxiliary Commandsβ
Before running the build
command, it is necessary to have all the input artifacts, accordingly with the desired customizations. This includes:
- A splash screen image.
- The source code of the device tree and the device tree overlays.
- The source code of the external kernel modules.
- The file with the configuration changes captured from the device.
- The docker-compose (YAML) configuration file of the application container or the directory with the container images.
- The Easy Installer image tarball of Torizon from Toradex (depending on the
input
mode) that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.
Some of these artifacts can be easily obtained using auxiliary TorizonCore Builder commands. Such as:
- The file with the configuration changes captured from the device. This can be easily obtained with the
isolate
command.
$ torizoncore-builder isolate --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --changes-directory changes
Customizing the Configuration Fileβ
As explained, the configuration file contains three sections: input
, customization
, and output
.
- The
input
section defines the source Toradex Easy Installer image of Torizon from Toradex, without containers, that will be customized. - The
customization
section determines the modifications to be applied to the input to produce the desired output. - The
output
section provides the details about the output artifact(s) to be produced by thetorizoncore-builder build
command.
To learn more about the structure of the file and see all the possible values for each section, check the "build" command - Detailed Manual.
Below is an example file that:
- As input, specifies:
- a locally downloaded tarball of a Toradex Easy Installer image
- As customizations, applies:
- a splash screen
- a specific device tree
- a display overlay
- a directory containing configuration changes captured from the device
- As output, specifies:
- a docker-compose.yml file to bundle into the generated Toradex Easy Installer image
- the directory name where the generated Toradex Easy Installer image with the containers and customizations is stored
- the parameters defining the ostree commit for the custom image
input:
easy-installer:
local: torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
customization:
splash-screen: splash.png
filesystem:
- changes/
device-tree:
include-dirs:
- linux/include/
custom: device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
overlays:
add:
- device-trees/overlays/colibri-imx6_lcd-edt7_overlay.dts
output:
ostree:
branch: "test-release-1"
commit-subject: "Test Release 1"
commit-body: "This is the first test release for my great product"
easy-installer:
local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
bundle:
compose-file: docker-compose.yml
username: "docker_hub_username"
password: "docker_hub_password"
Note that as a best practice, Toradex suggests you use Variable Substitution to provide the container registry credentials rather than placing them in the file in clear text.
Building The Imageβ
To build this custom Toradex Easy Installer image of Torizon from this configuration file, simply execute the build
command.
$ torizoncore-builder build --file tcbuild.yaml
As tcbuild.yaml
is the default name, the --file
argument is not required in this case and is just included here for demonstration purposes.
Deploying The Imageβ
The output image can be deployed in three different ways to the board:
- Using Toradex Easy Installer.
- Directly on the board through the deploy command.
- Through Torizon Cloud server.
- Upload a new image to the cloud.
- Update from an already deployed image using a Static-Delta.
The custom image created above can be deployed to Torizon Cloud server with the platform push
command using the ostree branch name specified in the tcbuild.yaml
file:
$ torizoncore-builder platform push --credentials credentials.zip test-release-1
Or directly to the board through SSH with the deploy
command:
$ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon test-release-1 --reboot
The SSH and Platform Services ways do not support the deployment of images with pre-provisioned containers. To learn more about the deployment of pre-provisioned containers, please refer to Deploying Container Images to Torizon OS article.
Approach 2: Using The Standalone Commandsβ
- Download an Easy Installer image of Torizon from Toradex (the
images download
command recognizes which is the device and automatically downloads and unpacks the corresponding image of Torizon). - Unpack the downloaded image;
- Run the auxiliary commands where needed, so all local artifacts possibly referenced by the configuration file are available;
- Apply the desired customizations in the unpacked image;
- Merge these customizations;
- (Just when including application containers) Deploy the output image without containers locally and combine it with the container images;
- Deploy the output image to the board or to the Torizon Cloud server;
Downloading And Unpacking The Torizon Image From Toradexβ
There are two different ways to obtain an unpacked image of Torizon from Toradex.
For the first way, it is necessary an Easy Installer image tarball of Torizon from Toradex (depending on the input
mode), that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.
After downloading this image, unpack it with the images unpack
command. As TorizonCore Builder is a container, it will be unpacked in a Docker volume that will be later used by the customization and deployment commands without the need of passing it.
$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
Another way to obtain this unpacked image is with the images download
command. This command connects to the board and downloads the appropriate Easy Installer image of Torizon from Toradex, unpacking the image in the same way as the images unpack
command after the download. In this way, instead of passing the image tarball, the arguments passed are the device's IP address, username, and password.
$ torizoncore-builder images download --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon
Differently from the build
command, unless you want to change the version of Torizon or the directory with the unpacked image after you unpack the image once, it is not necessary to run these steps again. Making a different custom image from the same base image can be done without executing this step again.
Auxiliary Commandsβ
Before applying the customizations to the custom Torizon OS image, it is necessary to have all the input artifacts accordingly to the desired customizations. This includes:
- A splash screen image.
- The source code of the device tree and the device tree overlays.
- The source code of the external kernel modules.
- The file with the configuration changes captured from the device.
- The docker-compose (YAML) configuration file of the application container or the directory with the container images.
- The Easy Installer image tarball of Torizon from Toradex (depending on the
input
mode) that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.
Some of these artifacts can be easily obtained using auxiliary TorizonCore Builder commands. Such as:
- The source code of the device tree and the device tree overlays from Toradex. They can be obtained with the
dt checkout
command.
$ torizoncore-builder dt checkout
- The file with the configuration changes captured from the device. This can be easily obtained with the
isolate
command.
$ torizoncore-builder isolate --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --changes-directory changes
- The directory with the container images. This can be generated from the docker-compose configuration file with the
bundle
command.
$ torizoncore-builder bundle docker-compose.yml --bundle-directory bundle
Applying The Customizationsβ
TorizonCore Builder has standalone commands for each desired customization.
To generate the image exemplified in the approach using the build
command with the same customizations, one should run the following commands:
- Run the
splash
command to apply the splash-screen.
$ torizoncore-builder splash splash.png
- Run the
dt apply
command to compile the device tree and store the generated device tree blob.
$ torizoncore-builder dt apply --include-dir linux/include/ device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
- Run the
dto apply
command to compile the device tree overlay (of the display in this example) and store the generated device tree overlay blob.
$ torizoncore-builder dto apply --include-dir linux/include/ device-trees/overlays/display-edt7_overlay.dts
Merging The Customizations Into The Imageβ
The union
command them merges all these customizations generated by the isolate
, splash
, kernel
, dt
, and dto
commands. It generates a custom image of Torizon with all these customizations included.
$ torizoncore-builder union custom-branch --changes-directory changes
Combining with Containersβ
First, the custom image generated with the union
command needs to be deployed into an output directory with the deploy
command. This output directory will contain a Custom Easy Installer image of Torizon without containers.
$ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
After this, combine this custom image with the container images generated with the bundle
command as explained above. This is done using the combine
command. The second argument passed on the command refers to the IMAGE_DIRECTORY
, and the third argument refers to the OUTPUT_DIRECTORY
.
$ torizoncore-builder combine --bundle-directory bundle torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WITHOUT-CONTAINERS torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WTIH-CONTAINERS
Deploying The Imageβ
The output image can be deployed in three different ways to the board:
- Using Toradex Easy Installer. First, the image generated by the
union
command needs to be deployed locally, with thedeploy
command passing a local directory as an argument.
$ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
- Directly on the board through SSH, with the
deploy
command, passing the device IP address, username, and password as arguments.
$ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot
- Through Torizon Cloud updates server, with
platform push
command.
$ torizoncore-builder platform push --credentials credentials.zip custom-branch
The SSH and Platform Services ways do not support the deployment of images with pre-provisioned containers. To learn more about the deployment of pre-provisioned containers, please refer to Deploying Container Images to Torizon OS article.
Deployment Methods Comparison: Development, Production Programming and In Fieldβ
As described in the deploy sections of each approach, there are three methods to deploy the custom image of Torizon OS to the device. Here is the recommended use of each:
- SSH: to be used when developing and evaluating single customizations and combinations of them. It allows for quick and direct deployment of the generated custom image of Torizon, without the need of using Toradex Easy Installer or the Torizon Cloud.
- Toradex Easy Installer: to be used when you are creating your own custom Torizon OS image for production programming. It is the only method that allows the deployment of images with application containers pre-installed, allowing the board to boot with your containers from the first boot.
- Torizon Cloud: to be used when you want to update the OS or the application on a running fleet of Torizon OS devices (also called In Field).
- It allows you to update your Torizon OS remotely, securely, on multiple devices at once, and without needing to re-flash the board.
- It's also possible to update from the cloud using the static-delta approach, which is more network efficient.
Despite the recommendations, in some cases, it is possible to interchange between the methods. If you want, instead of using the SSH method, you can for example use the Torizon Cloud updates features and the Toradex Easy Installer methods during development.
For example, we have reports of customers with remote teams and contractors who used the Torizon Cloud for collaborative development.