Build TorizonCore from Source With Yocto Project/OpenEmbedded
Introduction
TorizonCore is the base operating system in which Torizon is built. It is a minimal Yocto-based embedded Linux image and, even though customers may usually stick to application development on top of containers, sometimes they may need to customize the base image for their needs.
If you just want to start using TorizonCore as-is with no customization in the original image, see the Quickstart Guide for information on how to install it on your Toradex Computer-on-Module.
Before trying to build TorizonCore using Yocto Project/Open Embedded, briefly read what TorizonCore Builder can do. It's a Toradex-provided tool that can help in customizing your TorizonCore image, without having to deal with Yocto.
The Toradex manifest repository contains a Repo manifest and setup scripts for the TorizonCore build system. If you want to modify, extend or port TorizonCore to a new hardware platform, this is the manifest repository to use.
The build system uses various components from the Yocto Project, most importantly the OpenEmbedded build system, the bitbake task executor and various application and BSP layers.
You have two options to build TorizonCore (and possibly any Yocto-based image):
- Use a native Linux distribution like Ubuntu to build TorizonCore.
- Pros: has the advantage that you can easily understand what is going on.
- Cons: you may be stuck on a certain Distribution Version which is supported by the Yocto Release where TorizonCore is built on; you need to install all dependencies by yourself in every new development machine.
- Use a Docker container and build everything inside that container.
- Pros: it has the advantage that it will work independently of the version of the host distribution; it works under Windows; the commands used inside the container are the exact same as if you would build on the native distribution.
- Cons: it may seem to be more complicated at the first look.
You can build TorizonCore natively under your favorite Linux flavor. However, not all distribution version/Yocto version combos are allowed. If your distribution is not supported you will get a warning when running your first bitbake
command. In such cases, it may happen that your current distribution can't build TorizonCore. You may consider trying to use a Docker container then.
This article complies with the Typographic Conventions for Torizon Documentation.
Common Prerequisites
The following information is something that you should determine before proceeding with either build option.
Manifest Branch
The manifest branch refers to the git branch of the manifest repository that will be used for setup. The available branches related to TorizonCore images are:
TorizonCore version | Branch |
---|---|
6.x.y | kirkstone-6.x.y |
5.x.y | dunfell-5.x.y |
0.0.0 | master |
These branches come from the official Toradex Manifest Repository.
master is maintained to monitor changes in the external layers. Builds can be broken for a longer period of time. This branch is not recommended for productization.
Manifest File
The manifest file refers to the specific file in the manifest repository that will be used for setup. The available files related to Torizon images are:
torizoncore/default.xml
torizoncore/integration.xml
torizoncore/next.xml
It is recommended to use torizoncore/default.xml
. The other two manifest files are used by Toradex to monitor future changes and may not be stable.
TorizonCore Distro
Within the greater Torizon platform, Toradex provides different variations of TorizonCore. In the OpenEmbedded community these are known as "Distros". For TorizonCore, Toradex provides the following Distros:
Distro | Kernel Base | Kernel Config |
---|---|---|
torizon | Downstream kernel from SoC vendor (NXP) (default for iMX8* based modules) | Default (without the PREEMPT_RT patch) |
torizon-upstream | Mainline kernel (default for iMX6/iMX7 based modules) | Default (without the PREEMPT_RT patch) |
torizon-rt | Downstream kernel from SoC vendor (NXP) | Fully preemptive (real-time Linux PREEMPT_RT patch) |
torizon-upstream-rt | Mainline kernel | Fully preemptive (real-time Linux PREEMPT_RT patch) |
Machine
Supported MACHINE targets (officially tested by Toradex):
Machine Name | Corresponding Toradex Module |
---|---|
apalis-imx6 | Apalis iMX6 |
apalis-imx8 | Apalis iMX8 |
colibri-imx7-emmc | Colibri iMX7D 1GB (equipped with eMMC flash) |
colibri-imx6 | Colibri iMX6 |
colibri-imx6ull-emmc | Colibri iMX6ULL 1GB (equipped with eMMC flash) [1] |
colibri-imx8x | Colibri iMX8X V1.0C or newer |
verdin-imx8mm | Verdin iMX8M Mini |
verdin-imx8mp | Verdin iMX8M Plus |
TorizonCore Image
Supported image targets for bitbaking:
Image | Description |
---|---|
torizon-core-docker | Default image supported by Toradex - Network Manager: NetworkManager - Init system: systemd - Container Engine: Docker - OTA Framework: OSTree & Uptane |
torizon-core-podman | Experimental image using Podman instead of Docker Same Features as above except Container Engine is Podman |
Build Process
Choose from the tabs below to either do a native or a containerized build:
Prerequisites
You must have a host PC configured with the following prerequisites:
- Linux host with prerequisites for building a Yocto/OpenEmbedded image.
Download Metadata
To configure the scripts and download the build metadata, do:
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
In addition, make sure you have Git user name and e-mail configured:
$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"
Run repo init
to bring down the latest stable version of Repo. You must
specify a URL for the manifest, which specifies the various repositories that
will be placed within your working directory. Furthermore, a branch must be
specified. Available manifest branches are listed in the above table.
To check out the latest TorizonCore release, including monthly pre-releases:
$ repo init -u git://git.toradex.com/toradex-manifest.git -b kirkstone-6.x.y -m torizoncore/default.xml
Alternatively, if you want to work on a specific version, check out using tags. Find out available tags on the Git repository. See the example below for 6.0.0:
repo init -u git://git.toradex.com/toradex-manifest.git -b refs/tags/6.0.0 -m torizoncore/default.xml
For reproducible builds, you should at least lock in on a specific version. If reproducible builds are critical for your use case, consider fetching all source code and storing it on a local machine after the setup.
A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.
To pull down the metadata sources to your working directory from the repositories as specified in the repo manifest, run:
$ repo sync --no-clone-bundle
When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by repo:
$ export HTTP_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
$ export HTTPS_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
More rarely, Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during "Receiving objects"). It has been reported that tweaking the settings of the TCP/IP stack and using non-parallel commands can improve the situation. You need root access to modify the TCP setting:
$ sudo sysctl -w net.ipv4.tcp_window_scaling=0
$ repo sync -j1
Setup Environment
The default distribution (DISTRO) variable is automatically set based on the chosen MACHINE (either torizon
or torizon-upstream
). To choose another DISTRO
, specify so in the below command, similar to MACHINE
. For more information on available DISTROs please see the previous section TorizonCore Distro.
Setup the work environment by using the setup-environment
script:
$ [MACHINE=<MACHINE>] source setup-environment [BUILDDIR]
If MACHINE is not provided, the script will list all possible machines and force one to be selected. For more information on available machines please consult the Machine list.
Start Building
To build the TorizonCore image:
The above setup script should properly prepare the environment with correct defaults.
Additionally you'll want to modify the conf/local.conf file by adding ACCEPT_FSL_EULA="1"
.
$ bitbake torizon-core-docker
The TorizonCore Build Environment is a project that extends CROPS to build a TorizonCore image. CROPS is a Docker image for building Yocto based distros which is maintained by the Yocto Project.
Prerequisites
The following prerequisites are mandatory:
- Linux / Windows host with Docker installed.
Additionally, the following is required on a Windows Machine only:
- WSL2
- A suitable WSL2 Linux Distribution (Ubuntu 18.04 was tested)
For more information on setting up WSL2 please see the section Install the Windows Subsystem for Linux (WSL2).
Basic Build Configuration
Linux Setup
Create a working directory where the Yocto build will be stored. For example:
$ cd ~
$ mkdir ~/yocto-workdir
Then run the following minimal docker run
command to bring up the build container. Don't forget to substitute <machine>
and <image>
based on the module and TorizonCore image you want to build:
$ docker run --rm -it --name=crops -v ~/yocto-workdir:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:kirkstone-6.x.y startup-tdx.sh
Windows Setup
For Windows due to the heavy Linux filesystem dependencies of OpenEmbedded it will be easier to setup a working directory via WSL2. This working directory will be used to store the resulting build files on your host machine.
First, open a Windows command prompt and execute wsl
. This should put you in a prompt "inside" your installed WSL2 Linux distribution. From here you can setup a folder where you want to store the build files. For example, if you want to set this up inside the home directory then do as follows:
$ cd
$ explorer.exe .
This will open up your WSL2 home directory with Windows File Explorer. From here you can copy the exact file path for later use (i.e. \wsl$\Ubuntu\home\coj
)
Once you've setup your desired working directory you can now run the following minimal docker run
command via Windows command prompt (don't forget to substitute <machine>
and <image>
based on the module and TorizonCore image you want to build):
$ docker run --rm -it --name=crops -v \wsl$\Ubuntu\home\coj:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:kirkstone-6.x.y startup-tdx.sh
Build Information
For either Windows or Linux, once you've ran the appropriate command, this will then start to setup and build a Torizon image of IMAGE
for module MACHINE
. For more information on valid options for these variables please consult the Common Prerequisites at the beginning of this article.
Before the build begins you will be prompted to accept the NXP/Freescale EULA which is required for some packages. Make sure to accept this prompt before leaving to get coffee!
Once the build is complete you can view the build output from either in the container or your host machine's working directory that was setup up prior.
Container Arguments
The above docker run
commands were just minimal commands required to get the container to setup and start a build. However, there are additional arguments that can be passed to either further configure the build or change its behavior. These arguments are passed into the container as environment variables via the -e
flag of docker run
:
-e MACHINE=<machine>
: This is the target machine that the image will be built for. This is required and will not default to any value if it is not set. For a list of all compatible machines see the Machine list.-e DISTRO=<distro>
: This is the Torizon DISTRO that will be built. This will default to eithertorizon
ortorizon-upstream
depending on the chosenMACHINE
if this variable is not explicitly set. For other valid options please see the TorizonCore Distro list.-e BRANCH=<branch>
: This is the manifest repository branch. If not passed it defaults tokirkstone-6.x.y
. For a list of other options see the Manifest Branch.-e MANIFEST=<manifest>
: This is the specific manifest file that is to be used. If not passed it defaults totorizoncore/default.xml
. For other options see the Manifest File.-e IMAGE=<IMAGE>
: This is the Yocto build recipe that is to be built. If not passed the container will simply setup the build environment and then run a shell inside the without starting any build, for if you want to do further configurations before building. For a list of valid image recipes see the TorizonCore Images list.
Customization
The Yocto Project can be quite defying and hard to use. In the following documentation, we cover some basic functionality that is likely you will have to go through during your development:
It does not replace the official Yocto Project documentation, though. At the end of the article Build a Reference Image with Yocto Project, we provide links to it, and you will most likely need to study it to some degree.
Custom Layers
On TorizonCore all your custom layers must be version controlled by Git, due to how we include layer revision information with OSTree. More details are provided in the aforementioned article Hello World integration into OpenEmbedded.
Deploy to Torizon Platform Services
Request access to evaluate the Torizon Platform Services premium features |
To be able to deploy your custom image to Torizon Platform Services, you need to meet the following criteria:
- Maintain the OSTree and Aktualizr references unchanged at your Yocto Build setup. They are responsible for the system update and OTA process.
Then, all you need from Yocto is the resulting image in the Toradex Easy Installer format, which you will use with the TorizonCore Builder Tool to unpack
the image and then push
it to Torizon Platform Services.
Read the article Signing and Pushing TorizonCore Images to Torizon Platform Services for more details about this process.
Keep in mind, though, that once you build your own TorizonCore image you are fully responsible for regularly building and deploying updates as well.