Search by Tags

Build a Reference Image with Yocto Project

 

Article updated at 03 Sep 2020
Compare with Revision




Subscribe for this article updates

Introduction

This hands-on document describes how to build our Reference Images for Yocto Project from scratch using OpenEmbedded-Core (OE-Core). For a conceptual article about the relationship between Toradex' Embedded Linux offerings and the Yocto Project, you must read Yocto Project. For building TorizonCore, additional instructions are provided on Build TorizonCore With Yocto Project.

OpenEmbedded (openembedded.org) is a build framework that creates kernel images, root filesystem images and installable packages from source code. It is used by the Yocto Project to build Embedded Linux images. For the sake of this document OpenEmbedded and Yocto Project are used interchangeably unless otherwise noted.

OpenEmbedded uses meta-information (called recipes) for downloading/compiling/deploying of software packages on a x86/x86_64 Linux build host for a target device. This meta-information is structured into so-called layers. Layers are directory trees with recipes that provide similar functionality, e.g. meta-lxde which provides recipes to build packages for the LXDE desktop.

Versions and Source-Code

Layers from git.toradex.com are used together with a number of other layers. The collection of Toradex and third-party layers used on a specific BSP release are provided in a repo manifest on toradex-manifest, and you will learn more about it later in this article.

There might be several concurrent versions available at a given point in time, learn more about this on the Toradex Embedded Linux Support Strategy. Versions of the Reference Images for Yocto Project and their correspondent distribution, OpenEmbedded codename, and Yocto Project releases are provided in the following table:

Toradex Version Status Distribution OpenEmbedded/Yocto Project Codename Yocto Project Release
2.1 Outdated Ångström v2013.06 dylan 1.4
2.2 Outdated Ångström v2013.12 dora 1.5
2.3 Outdated Ångström v2014.06 daisy 1.6
2.4 Outdated Ångström v2014.12 dizzy 1.7
2.5 Outdated Ångström v2015.06 fido 1.8
2.6 / 2.6.1 Outdated Ångström v2015.12 jethro 2.0
2.7 Outdated Ångström v2016.12 morty 2.2
2.8 LTS Ångström v2017.12 rocko 2.4
3.0 LTS Poky based thud 2.6
4.0.0 (deferred*) Deferred Poky based zeus 3.0
5.0.0 Latest Poky based dunfell 3.1

* 4.0.0 is deferred, learn more on https://www.toradex.com/blog/torizon-yocto-project-long-term-support-alignment

A release matrix that summarizes versions of Yocto/OpenEmbedded, the Linux kernel and U-Boot, and release dates for Toradex Embedded Linux releases is provided in a separate article:

Release Notes

Concise, high-level release notes are published on the News section of toradex.com:

Comprehensive, detailed release notes are maintained on the following pages on the developer website:

Legacy BSPs

At the end of this page, there is a section dedicated to older BSP releases.

Prerequisites

A Yocto Project build requires intensive computing power and specific software packages.

Computer for the Yocto Project Build

To build a Yocto Project image a powerful host machine is highly recommended. If the host system is a multi-core machine, you can configure the Yocto Project build system to significantly decrease the time needed to build images. There should be a minimum of 60 GBytes of free disk space. For some images, a 32-bit host with 4 GBytes of memory is enough, but in order to build applications such as the WebKit web engine, a 64-bit machine with at least 8 GBytes of RAM is recommended.

Operating System and Build Dependencies

As for the operating environment, a supported Linux distribution is required, and you must install Yocto build dependencies. It is already clearly documented by the Yocto Project, therefore we simply point you to their documentation.

Attention: make sure to only follow the steps on the Yocto Mega Manual to install the build dependencies, do not go any further into the environment setup.

Should you choose Ubuntu or another distro that uses dash as the default shell we recommend changing this to bash. (Google for 'ubuntu bash update-alternatives')

Choose the BSP version from the tabs below:

Virtual Machines, Containers and Cloud Builds

Toradex does not go on about doing a Yocto Project build on a VM, Container or a public cloud provider as AWS or Azure. Nevertheless, it is possible, therefore if you feel like doing it you may consider studying your options. Find some generic remarks for a simplified overview:

  • VM: Since a Yocto Project build is resource-intensive, it may not be a good idea to use a VM.
  • Container: it has a negligible performance penalty. Pay attention to using bind-mounts or volumes to keep persistent data, otherwise you may lose time on every build. Containers theoretically allow you to do a Yocto Project build from a Windows PC.
  • Cloud: the cloud has powerful machines that make it possible to run a full Yocto Project build in less than half an hour if you disregard setup and pre-requisites installation time. Similar to containers, you should plan how to store the output of a build. In addition, don't forget to evaluate the cost of this approach!

Repo and Git

Since OpenEmbedded requires several git repositories to be available to build our images, starting with V2.1 images we are using a utility called 'repo'. The repo manifest manages the various git repositories and their branches. (more on repo: http://code.google.com/p/git-repo/)

Install the repo bootstrap binary:

$  mkdir ~/bin
$  export PATH=~/bin:$PATH
$  curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$  chmod a+x ~/bin/repo

Alternatively, in Ubuntu, 'repo' may be installed as part of the 'phablet-tools' package.

Repo uses Git. Make sure you have it installed and user name and e-mail configured. Below is an example for Debian-based systems:

$ sudo apt install git
$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

Installation

This section has two major sub-sections, you have to choose:

  • First-time Configuration: if you are doing a build for the first time.
  • Update an Existing Configuration: if you have a build already set up and want to update it.

First-time Configuration

Create a directory for your OE-Core setup to live in and clone the meta-information:

mkdir ${HOME}/oe-core
cd ${HOME}/oe-core

Choose the BSP version from the tabs below:

$ repo init -u https://git.toradex.com/toradex-manifest.git -b dunfell-5.x.y -m tdxref/default.xml
$ repo sync

Deferred release. See more at https://www.toradex.com/blog/torizon-yocto-project-long-term-support-alignment

$ repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImage4.0 -m default.xml
$ repo sync
$ repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImage3.0 -m default.xml
$ repo sync
$ repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImageV2.8
$ repo sync

Source the file 'export' to setup the environment. On the first invocation, this also copies a sample configuration to build/conf/*.conf.

$ . export

Note: Sourcing 'export' configures the shell environment for the current shell session. It must be entered whenever a new shell session is opened for use with OpenEmbedded.

local.conf

Adapt build/conf/local.conf to your needs.

The MACHINE variable specifies the target device for which the image is built. Set this variable to the module type you are planning to build for

Our BSP layers provide the following machines:

Machine Name
apalis-imx6
apalis-imx8
apalis-imx8x (for Apalis iMX8X V1.0B)
apalis-imx8x-v11a (for Apalis iMX8X V1.1A and newer)
apalis-t30 (In BSP releases prior to 3.0)
apalis-tk1
apalis-tk1-mainline
colibri-imx6
colibri-imx6ull
colibri-imx7
colibri-imx7-emmc
colibri-imx8x (for Colibri iMX8X V1.0C and newer)
colibri-imx8x-v10b (for Colibri iMX8X V1.0B)
colibri-t20 (In BSP releases prior to 3.0)
colibri-t30 (In BSP releases prior to 3.0)
colibri-vf (In BSP releases prior to 3.0)
verdin-imx8mm

e.g. set in local.conf

MACHINE ?= "colibri-imx6"

Note: You can explicitly override the MACHINE setting on the cmdline, set the variable MACHINE when calling the bitbake command (e.g. MACHINE=apalis-imx6 bitbake...)

If you have lots of disk space and want to browse the unpacked sources, consider commenting the 'INHERIT += "rm_work"' line.

If you already have a download directory with sources from a previous OE setup consider moving that directory into oe-core/build/download or change local.conf to point to your common download location. DL_DIR.

If you want to build for a machine based on an NXP based SoM some downloads require you to read and accept the NXP®/Freescale EULA available in layers/meta-freescale/EULA. You have to state your acceptance by adding the following line to your local.conf file: ACCEPT_FSL_EULA = "1" If this line is missing the relevant packages will produce an ERROR similar to: ERROR: To use 'imx-vpu' you need to accept the NXP®/Freescale EULA at 'layers/meta-freescale/EULA'. Please read it and in case you accept it, write: ACCEPT_FSL_EULA = "1" in your local.conf.

Setup is finished, continue with the Building chapter.

Update an Existing Configuration

To update the installation you first update the repo manifest to the version you want to work with and then sync all layers. Check the known issues section below before starting the update.

If you did local changes to the individual layers merge conflicts may occur which you will have to resolve in the respective layer.

Update to the HEAD Revision

Choose from the tabs below:

Note: You cannot update from BSP 4.0.0 or older to 5.0.0 or newer, because a different Git repository is used!

A list of available branches can be found in the repository: http://git.toradex.com/cgit/toradex-manifest.git/refs/heads

repo init -b dunfell-5.x.y
repo sync

Note: e.g. use LinuxImage4.0 to update to the BSP 4.0 image version. Use LinuxImageV2.x e.g. LinuxImageV2.8 in BSP releases prior to 3.0).

A list of available branches can be found in the repository: https://git.toradex.com/cgit/toradex-bsp-platform.git/refs/heads

repo init -b LinuxImage4.0
repo sync

Update to a Specific Version by Using its Tag

Choose from the tabs below:

A list of available tags can be found in the manifest repository: https://git.toradex.com/cgit/toradex-manifest.git/refs/tags

repo init -b refs/tags/Apalis-iMX8_Console-Image_3.0b1.40-20190612
repo sync

A list of available tags can be found in the manifest repository: https://git.toradex.com/cgit/toradex-bsp-platform.git/refs/tags

repo init -b refs/tags/Apalis-iMX8_Console-Image_3.0b1.40-20190612
repo sync

Known Issues During repo Setup

If your repo command succeeds, you can directly proceed with the instructions in the chapter Building below. In addition the known issues on this section, there are more known issues for older versions at the end of this page.

Repo Sync Issues
Bitbake Build Issues

Building

This section goes on about build steps.

Setup the Environment

Every time you open a new terminal, go to the OpenEmbedded directory and setup the environment:

$ cd
$ cd oe-core
$ . export

Build the image, bitbake automatically logs console output to timestamped files in build/tmp/log/cooker/$MACHINE/.

Note: With OE-Core you need to be in the directory 'build' when you execute bitbake. Note: This will at first build take hours, download lots of stuff and fill 'build/tmp', so have at least 60 GByte free. Note: Check available demo image recipes in layers/meta-toradex-demos/recipes-images/images/

Directory Structure

OE-Core installation, configuration, and build will setup the following directory structure:

oe-core/
+-- build
¦   +-- conf
¦   +-- downloads
¦   +-- tmp
¦   +-- sstate-cache
¦   +-- deploy
+-- layers
    +-- meta-browser
    +-- meta-freescale
    (... other layers)
    +-- openembedded-core
  • layers: stores metadata, as machine and distro information, but also the recipes for building all components of an embedded Linux image.
  • build: keeps build configuration, downloaded source codes, intermediate build output, generated packages (IPK, DEB, RPM), SDKs.
    • conf: this is the only subdirectory that you must preserve under build, all other subdirectories are regenerated when you start a new build if they don't exist yet.
    • deploy: the final images, ready to install (flash) into a computer on module.

Choose an Image

Toradex provides reference images. You can use them as a base for your own distribution, just don't use them as-is because they are not ready to be deployed in production (not hardened, etc.). The available image recipes are located in layers/meta-toradex-demos/recipes-images/images/, also in our Git repository - just make sure to select the correct branch!

Choose the BSP version from the tabs below:

From BSP 5.0 onwards, there will be two reference images:

Image Description
tdx-reference-minimal-image A minimal reference image that you can use to create your own custom distribution from
tdx-reference-multimedia-image A reference image with multimedia capabilities that you can use to create your own custom distribution from

On BSPs 3.0 and 4.0.0, you are recommended to use the following image:

Image Description
console-tdx-image A minimal reference image that you can use to create your own custom distribution from

On BSPs 3.0 and 4.0.0, you are recommended to use the following image:

Image Description
console-tdx-image A minimal reference image that you can use to create your own custom distribution from

On BSP 2.8 and older, you are recommended to use the following image:

Image Description
angstrom-lxde-image A minimal reference image that you can use to create your own custom distribution from

Choose the Distro

The DISTRO defines a distribution which configuration file has the same name with the extension .conf. See the configuration files on the layers/meta-toradex-distro/conf/distro/ directory. You can set this variable on the export file located at the root directory of your OE setup.

Toradex provides a real-time kernel recipe for use in an OpenEmbedded-Core (Yocto Project) build, and corresponding distros that use it. See the Real-Time Linux article for more information about the PREEMPT_RT patch.

The following are possible options of distribution:

Distro Configuration file Description
tdx-xwayland ./layers/meta-toradex-distro/conf/distro/tdx-xwayland.conf Support for Wayland included
tdx-xwayland-rt ./layers/meta-toradex-distro/conf/distro/tdx-xwayland-rt.conf Support for Wayland included and Real-time Linux (PREEMPT_RT patch) enabled
tdx-x11 ./layers/meta-toradex-distro/conf/distro/tdx-x11.conf Support for X11 included
tdx-x11-rt ./layers/meta-toradex-distro/conf/distro/tdx-x11-rt.conf Support for X11 included and Real-time Linux (PREEMPT_RT patch) enabled

Build with bitbake

This section contains the command for building an image, but also other useful bitbake commands. You must follow at least the sub-section Build an Image.

Build an Image

After you choose the image from the previous section, you can build the image. If you do an unattended build, -k tells bitbake to keep going with independent packages after an error occurred:

$ bitbake -k <image>

If you don't want the build to proceed after an error, run without -k:

$ bitbake <image>

Build a Single Package

Build a single package and all the things it depends on.

 bitbake samba

Build an SDK

Building an SDK for your image, pass the parameter -c populate_sdk (See also Linux SDKs)

bitbake <image> -c populate_sdk

Build the Linux kernel

There is a virtual recipe for building the Linux kernel for your machine regardless of its version:

$ bitbake virtual/kernel

Additional Logs

Warning: the amount of logs printed to stdout is huge!

If you need to dig into what recipes and settings are used the following command outputs a bunch of environment settings and info about what is going on, use the parameter -e:

$ bitbake -e virtual/kernel
$ bitbake -e virtual/kernel | grep ^WORKDIR

Available Tasks for a Recipe

If you want to see all tasks which exist for a given recipe, use the parameter -c listtasks:

$ bitbake -c listtasks virtual/kernel

Build Output

The output can be found here:

  • For images, u-boot, uImage, rootfs, deployable tarball: build/deploy/images/${MACHINE}/
  • For SDKs: build/deploy/sdk/
  • For ipk packages: build/deploy/ipk//*.ipk

Update

Get the deployable tarball image from the directory pointed in the previous section.

Toradex Easy Installer Image Format

The image tarball, (e.g. Colibri-iMX6_LXDE-Image-Tezi_2.7-20180104.tar) has the same structure as our pre-built ones we provide for download. To update your computer on a module with a newly built image follow the update procedure documented in the Toradex Easy Installer article.

Legacy Image Format

At the end of this page, there is a section dedicated to flashing Toradex modules using the legacy flashing procedure that preceded the existence of the Toradex Easy Installer. You should only use it if you really need to.

First Steps Into Your Custom Build

The Yocto Project can be quite defying and hard to use. In the following documentation, we cover some basic functionality that is likely you will have to go through during your development:

It does not replace the official Yocto Project documentation, though. At the end of this page, we provide links to it and you will most likely need to study it to some degree.

Failing Builds

An OpenEmbedded build sometimes fails.

This is what I normally do when a task fails:

  • Reading through the error messages.
  • Restarting bitbake without deleting anything and see if it happens again.
  • Clean the task which failed and then restarts building the image, e.g. if python-native fails

    $ bitbake -c clean python-native $ bitbake tdx-reference-minimal-image

  • Clean the task which failed, including deleting cached output and then restart building the image

    $ bitbake -c cleansstate python-native $ bitbake tdx-reference-minimal-image

  • Clean the task which failed, including deleting cached output and the downloaded files and then restart building the image

    $ bitbake -c cleanall python-native $ bitbake tdx-reference-minimal-image

  • Check the layer which provides the recipe if a newer commit exists which addresses the problem.

  • If there are a lot of issues with fetching sources you could try to first only download all needed sources.

    bitbake -c fetchall tdx-reference-minimal-image

Check the legacy section at the end of this page to find out about failing builds on older BSP versions.

Additional Layers

Recipes for even more software are available in additional layers. e.g. The Java layer provides recipes related to Java. Most layers are registered at openembedded.org. They provide a web interface to find layers or individual recipes: http://layers.openembedded.org/layerindex/

Note that just because a layer or a recipe exists, that does not mean that these recipes will compile/execute. The layers/recipe might have a version conflict with what we use or might have never been used with or intended for the Arm architecture.

Refer to the following sections to see examples of how to add a new layer to our setup.

Adding the Qt Layer

Please refer to How to set up Qt Creator to cross-compile for embedded Linux.

Adding the Java Layer

Please refer to Java Virtual Machine.

Working with the Package Manager

Sometimes you may want to build a single package with OpenEmbedded and install it on a running system, instead of building and re-flashing the entire image. A package manager makes it possible and our reference images come with Opkg, a lightweight package manager when compared to those often used in desktop distros, like apt-get for example.

Note: Starting with BSP 3.0 we dropped the Ångström Distribution in favour of a Poky based distribution. So for BSP 3.0 and newer based images you cannot install packages from an online feed, only packages you build. However, if you use TorizonCore you can use a container that makes it possible to use feeds from Debian.

Install a Package

To deploy ipk packages into a running image copy the relevant ipk files onto the module and use the opkg package manager to install it. If the package has dependencies that are not yet installed you will have to install them as well.

opkg install file1.ipk file2.ipk

What Package Provides a File

The following script can be used on a module to search what package provides a given file.

#!/bin/sh
FILE=vpu_fw_imx6q.bin
PKGS=`opkg list-installed | sed 's/^\(.*\)\s-\s.*/\1/g'`
for pkg in $PKGS
do
        CNT=`opkg files $pkg | grep -c $FILE`
        if [ $CNT -ne 0 ]
        then
                echo $pkg
        fi
done

Further Reading

The Yocto Project Documentation, particularly the Complete Documentation Set and the Bitbake User Manual provides a lot of information. Two books we can recommend are: Embedded Linux Development with Yocto Project Embedded Linux Projects Using Yocto Project Cookbook

Webinar: Building Embedded Linux Images with the Yocto Project



Legacy

For our Linux image starting from V2.0 we use OpenEmbedded-Core. Older images used OpenEmbedded (classic). For the early V2.0 images and information on how to setup OpenEmbedded for them, see OpenEmbedded-Core Configuration for V2.0 Images.

The directory structure presented on the previous section has changed over time:

  • The layers directory was called stuff.
  • The tmp directory was called tmp-glibc or out-glibc or out-eglibc.
  • The deploy directory was a subdirectory of oe-core or out-glibc.

Yocto Build Dependencies

Toradex BSP (meta-toradex) builds require some additional packages, mainly to compile the flashing utilities as 32-bit executables.

Attention: the Yocto Project already documents the required dependencies and supported OS for a specific Yocto version. We stopped to document it ourselves and instead we point to the official Yocto docs. This section is kept solely for reference, but it is not updated and you are recommended to not follow it.

Fedora 28
Fedora 24 and later for releases prior to V2․7
Fedora 22 to 28
Fedora 20 and 21
Ubuntu 18․04 64-bit
Ubuntu 16․04 64-bit
Ubuntu 15․10 64-bit
Ubuntu 15․10 32-bit
Ubuntu 14․04 64-bit
Ubuntu 14․04 32-bit

Known Issues During repo Setup

If your repo command succeeds, you can directly proceed with the instructions in the chapter Building below.

Updates from older versions
Bitbake Build Issues

Legacy Image Format

Attention: We dropped building legacy images with BSP 3.0.

The image tarball, (e.g. Colibri-iMX6_LXDE-Image_2.7-20180104.tar.bz2) has the same structure as our pre-built ones we provide for download. To update your Colibri module with a newly built image follow the update procedures documented in the flashing articles for i.MX6 , i.MX7 , Tegra and Vybrid.

If you built an alternative image target like core-image-minimal one can just replace the rootfs from e.g. our Colibri_T20_LinuxImageV2.7Beta1_20170112.tar.bz2 tarball with the one from your core-image-minimal build and update your module as usual. Note however that Toradex images newer than 03.2013 determine the module type from the rootfs/etc/issue file. On newer images use 'update.sh -h' and then '-m' parameter to force a module type. For older images add a line so that our update.sh script can determine the correct module type e.g.:

sudo sh -c 'echo "" >> etc/issue'
sudo sh -c 'echo "Colibri_T20" >> etc/issue'

Use one of the following strings: "Apalis_iMX6", "Apalis_T30", "Apalis_TK1", "Colibri_iMX6", "Colibri_iMX7", "Colibri_T20", "Colibri_T30", "Colibri_VF"

Failing Builds

For builds, before V2.1 I had at least one installation where bitbake consistently failed on some task because there was garbage in the ccache. Deleting $HOME/.ccache helped, alternatively you can disable the feature by commenting the CCACHE line in build/conf/local.conf. Also deleting the parser cache helps sometimes. $OEHOME/output/cache

The native readline build fails on some machines. An issue for which I do not have a clean solution. The workaround which works is to install readline on the build machine and add an entry to build/conf/local.conf that it need not be built: ASSUME_PROVIDED += "readline-native"

Working with the Package Manager

Using Ångström Distribution Feeds - Up To BSP 2.8

Note: Starting with BSP 3.0 we dropped the Ångström Distribution in favour of a Poky based distribution. So for BSP 3.0 based images, no feeds are available.

The Ångström Distribution provides packages for certain architectures and versions of OpenEmbedded which you can test and use as follows.

Download the list of packages in the available feeds, if a configured feed is not available it will print an error message:

opkg update

Listing available packages, search the list:

opkg list
opkg list | grep jpeg

Install a package and its dependencies:

opkg install nano

Note that using 'opkg upgrade' likely leads to an unusable system. Due to the closed source X11 drivers the whole of X11 must use a matching version. opkg upgrade will update some X11 components to a non-compatible version resulting in X11 unable to start.

Note: Starting with V2.7 beta 1 we modified libc to work on Apalis/Colibri T20/T30 despite us relying on an ancient Linux kernel version from NVIDIA's L4T. Please make sure to hold on to our libc version as follows to avoid it getting replaced leading to FATAL: kernel too old:

root@colibri-t30:~# opkg flag hold libc6
Setting flags for package libc6 to hold.