Search by Tags

Linux - Booting

 

Compare with Revision




Subscribe for this article updates

Boot Sequence

After reset the Tegra T20/T30 internal boot ROM code goes trough the following steps:

  • The CPU internal boot ROM reads and latches the values present at GPIO's with strapping functionality.
  • The state of the strapping pins and CPU internal fuse settings are used to determine the boot device used for booting.
  • The Boot Configuration Table (BCT) is read from the start of the boot device. The BCT contains the needed DDR RAM configuration and optimised timings for the boot device.
  • The RAM gets initialized.
  • The boot loader is read from the boot device and control is passed to the boot loader. This is the earliest where customised code can be executed.

We use an extended U-Boot as boot loader. (U-Boot)

  • U-Boot reads its environment variable from the boot device. If these data is corrupted it loads initial settings.
  • U-Boot analyses the environment variables to find out where the kernel and the rootfs are stored and what kernel command line is requested.
  • U-Boot auto detects the available RAM size and NAND Flash size and properties.
  • U-Boot sets the Ethernet MAC address and configures the hardware in preparation for the Linux kernel.
  • U-Boot loads the kernel into RAM and passes control to the kernel.
  • The kernel initializes Linux, mounts the rootfs and starts 'init' to initialize the Linux user space.

Boot Scenarios

  • Boot device is the NAND flash provided on the Colibri T20 or eMMC provided on T30 modules. U-Boot, kernel and rootfs are stored on the boot medium. The U-Boot environment in our images is set for this scenarios.
  • Boot device is a SD card. U-Boot, kernel and rootfs are stored on the boot medium. The strapping pins have to be set accordingly (Colibri T20 only).
  • By changing the U-Boot environment the storage location of the kernel and rootfs can be changed to something different than the boot device.
  • The kernel can be loaded from NAND/eMMC, SD card, USB memory stick or a TFTP server.
  • The rootfs can be on NAND/eMMC, SD card, USB memory stick or a NFS server.

Common Combinations

  • Everything on NAND/eMMC –> Standard, for production and testing.
  • Everything on SD card -> When you want to change between different OS', i.e. WinCE on NAND flash, Linux on SD card (Colibri T20 only).
  • U-Boot on NAND/eMMC, kernel and rootfs on SD card -> When you want to change between different kernels / rootfs'.
  • U-Boot on NAND/eMMC, kernel from TFTP and rootfs from NFS -> During kernel and application development and debugging.

See the release notes of our Linux image for instructions on how to activate NAND Flash or SD card boot on a Colibri T20. For the other scenarios the U-Boot environment of a module that either does NAND/eMMC or SD card boot is modified as described below.

U-Boot

Console

U-Boot's console is available on the primary aka Full Function (FF) UART aka UART_A.

While U-Boot's console output is also visible on the parallel RGB display (and with the carrier boards RAMDAC on VGA) a USB keyboard directly connected to the module does not work as of yet.

Environment Variables

The following listing shows the content of the U-Boot environment as present in our T20_LinuxImageV2.0 for NAND flash boot.

Tegra2 # printenv
baudrate=115200
bootcmd=run flashboot; run nfsboot
bootdelay=5
defargs=video=tegrafb vmalloc=128M usb_high_speed=1
ethaddr=00:14:2d:48:8a:58
fdtaddr=182ac8
flashargs=ip=off root=/dev/mtdblock0 rw rootfstype=yaffs2
flashboot=run setup; setenv bootargs ${defargs} ${flashargs} ${mtdparts} ${setupargs}; echo Booting from NAND...; nboot ${loadaddr} 0 ${lnxoffset} && bootm
ipaddr=192.168.10.2
lnxoffset=0xc80000
loadaddr=0x408000
memargs=mem=372M@0M fbmem=12M@372M nvmem=128M@384M
mmcboot=echo Loading RAM disk and kernel from MMC/SD card...; mmc init && fatload mmc 0:1 0xC08000 rootfs-ext2.img.gz && fatload mmc 0:1 ${loadaddr} uImage;run ramboot
mtdparts=mtdparts=tegra_nand:1023744K@23808K(USR),3072K@0K(BCT),256K@4096K(PT),2048K@5376K(EBT),256K@8448K(BMP),2048K@9728K(ENV),8192K@12800K(LNX),256K@22272K(ARG)
netmask=255.255.255.0
nfsargs=ip=:::::eth0:on root=/dev/nfs rw netdevwait
nfsboot=run setup; setenv bootargs ${defargs} ${nfsargs} ${mtdparts} ${setupargs}; echo Booting from NFS...; usb start; dhcp; bootm
ramargs=initrd=0xA1800000,32M ramdisk_size=32768 root=/dev/ram0 rw
ramboot=run setup; setenv bootargs ${defargs} ${ramargs} ${mtdparts} ${setupargs}; echo Booting from RAM...; bootm
sdargs=root=/dev/mmcblk0p1 ip=:::::eth0:off rw,noatime rootfstype=ext3 rootwait gpt gpt_sector=18945
sdboot=run setup; setenv bootargs ${defargs} ${sdargs} ${mtdparts} ${setupargs}; echo Booting from MMC/SD card...; mmc read 0 ${loadaddr} 0x2a00 0x4000; bootm
serverip=192.168.10.1
setup=setenv setupargs asix_mac=${ethaddr} no_console_suspend=1 console=tty1 console=ttyS0,${baudrate}n8 debug_uartport=lsport,0 ${memargs}
stderr=serial,lcd
stdin=serial,usbkbd
stdout=serial,lcd
ubiargs=ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs
ubiboot=run setup; setenv bootargs ${defargs} ${ubiargs} ${mtdparts} ${setupargs}; echo Booting from NAND...; ubi part kernel-ubi && ubi read ${loadaddr} kernel; bootm
usbboot=echo Loading RAM disk and kernel from USB stick...; usb start && fatload usb 0:1 0xC08000 rootfs-ext2.img.gz && fatload usb 0:1 ${loadaddr} uImage;run ramboot

Environment size: 2122/4092 bytes
Tegra2 #

Resetting the Environment to the Defaults

Tegra2 # env default -f
## Resetting to default environment
Tegra2 # saveenv

Creating a New or Changing an Existing Variable

Creating a new variable or changing an existing one is done using the setenv command. You must enclose the argument with ' ' if you don't want to expand variables or multiple statements are given. To make changes permanent they need to be stored in flash with saveenv:

Tegra2 # setenv defargs 'video=tegrafb vmalloc=248M usb_high_speed=1'
Tegra2 # saveenv
Saving Environment to NAND...
Erasing Nand...
Erasing at 0xe00000 -- 100% complete.
Writing to Nand... done

Booting Methods

U-Boot on NAND Flash, Kernel and Rootfs on SD Card

Please use this method only during development. Normal SD cards aren't reliable for long term usage.

Legacy Bring-up

This section provides information related to the legacy bring-up aka flashing resp. updating of Embedded Linux.

Note: Please try our new Toradex Easy Installer which is a Linux-based application providing a graphical selection of operating system images that users may choose to install onto the internal flash memory of Toradex modules in an extremely simple way.

Note: Please note that the Toradex Easy Installer will be the default pre-installed software for Apalis TK1, Apalis/Colibri iMX6, Colibri iMX7 and all newer modules in the future.

Note: Only the latest U-Boot (June 2013) for Colibri T30 is able to read from SD card.

#create on your Linux PC a SD card with a GPT partition formatted for ext3
$ sudo parted /dev/sd? mklabel gpt
$ sudo parted /dev/sd? mkpart "colibri ext3 1 -1"
$ sudo mkfs.ext3 /dev/sd?1

#copy the rootfs & kernel to the SD card
$ sudo mkdir /mnt/sd
$ sudo mount /dev/sd?1 /mnt/sd
$ sudo cp -Ppr /path to image/rootfs/* /mnt/sd/
$ sudo cp /path to kernel/uImage /mnt/sd/boot/uImage
$ sudo umount /mnt/sd

#put the card into your Colibri T20 system, connect a terminal and boot to the U-Boot prompt
#change the U-Boot environment so that it loads kernel and rootfs from SD card.
#check the blockdevice name by booting from internal memory first. It may change between image versions. 
Tegra2 # setenv mmcargs 'root=/dev/mmcblk0p1 ip=:::::eth0:off rw,noatime rootfstype=ext3 rootwait'
Tegra2 # setenv mmcboot 'run setup; setenv bootargs ${defargs} ${mmcargs} ${mtdparts} ${setupargs} ${vidargs}; echo Loading kernel and rootfs from MMC/SD card...; mmc dev 0 && mmc part && ext2load mmc 0:1 ${loadaddr} /boot/uImage; bootm'
Tegra2 # saveenv
Tegra2 # run mmcboot

#put the card into your Apalis/Colibri T30 system, connect a terminal and boot to the U-Boot prompt (Below example uses 8-bit slot on Apalis T30, use 'mmc dev 2 && ext2load mmc 2:1 ${loadaddr} /boot/uImage' for 4-bit slot e.g. on Ixora)
#change the U-Boot environment so that it loads kernel and rootfs from MMC/SD card.
#check the blockdevice name by booting from internal memory first. It may change between image versions.
Apalis T30 # setenv sdargs 'ip=off root=/dev/mmcblk1p1 rw,noatime rootfstype=ext3 rootwait'
Apalis T30 # setenv sdboot 'run setup; setenv bootargs ${defargs} ${sdargs} ${setupargs} ${vidargs}; echo Loading kernel and rootfs from MMC/SD Card...; mmc dev 1 && ext2load mmc 1:1 ${loadaddr} /boot/uImage; bootm ${loadaddr}'
Apalis T30 # saveenv
Apalis T30 # run sdboot

Note: For versions prior to V2.1Beta3_20140318 one has to replace above 'mmc dev [dev]' command by 'mmc part [dev]' command as U-Boot changed in that respect.

Toradex Easy Installer Bring-up

This section provides information related to the Toradex Easy Installer bring-up.

Make sure you have installed the latest U-Boot by installing the latest Version of Toradex Easy Installer.

Download a Toaradex Easy Installer image, here we assume it is an image for Apalis iMX6.

On a Linux PC run the following commands:

# untar the Toradex Easy Installer image
tar -xf apalis-imx6_lxde-image-tezi_2.8b3.111-20180626.tar
cd Apalis-iMX6_LXDE-Image-Tezi_2.8b3.111/

# Delete the SD card current partition table 
sudo dd if=/dev/zero of=/dev/sda bs=1k count=2
# Create new partition table
sudo parted /dev/sda mktable msdos
# Create partitions
sudo parted /dev/sda mkpart primary ext4 1 100M
sudo parted /dev/sda mkpart primary ext4 100 100%
# Format the partitions
sudo mkfs.ext4 -L boot /dev/sda1
sudo mkfs.ext4 -L root /dev/sda2

mkdir /tmp/boot
mkdir /tmp/root
sudo mount /dev/sda1 /tmp/boot
sudo mount /dev/sda1 /tmp/root

# Untar images to SD card
sudo tar -xf Apalis-iMX6_LXDE-Image.bootfs.tar.xz -C /tmp/boot/
sudo tar -xf Apalis-iMX6_LXDE-Image.rootfs.tar.xz -C /tmp/root/

sudo umount /tmp/boot
sudo umount /tmp/root
rm -r /tmp/boot
rm -r /tmp/root

Under U-Boot run the following command:

run sdboot

U-Boot on NAND Flash, Kernel from TFTP and Rootfs from NFS

We recommend using a second interface card on your development PC to build a dedicated network to your Colibri.


Install and configure a DHCP, TFTP and NFS server on your development PC.

  • U-Boot obtains the IP address and the name of the file containing the kernel from the DHCP server.
  • U-Boot loads the kernel from the TFTP server with 'serverip' address.
  • U-Boot sets the kernel's commandline so that the kernel mounts the rootfs from a NFS server.

  • Linux uses DHCP again to get an IP address and get the root-path arguments

  • Linux tries to mount the root file system using the root-path

Please consult your distributions documentation on what server packages are needed and how they get configured.
Setup your distributions DHCP, TFTP and NFS server. Maybe you additionally need to allow these protocols to pass through by your firewall rules. The following sample configuration may (or may not) fit your distribution.

After configuration changes and when changing the served NFS files the servers need to be restarted:
e.g. on Ubuntu: mk@ubuntu: ~ $ sudo service isc-dhcp-server restart; sudo service tftpd-hpa restart; sudo service nfs-kernel-server restart
e.g. on Fedora: [root@vm_one ~]# systemctl restart dhcpd.service; systemctl restart nfs-server.service

Sample DHCP Configuration

# dhcpd.conf
option domain-name "colibri.net";
option domain-name-servers ns1.example.org;

default-lease-time 600;
max-lease-time 7200;

# Use this to enable / disable dynamic dns updates globally.
ddns-update-style none;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

subnet 192.168.10.0 netmask 255.255.255.0 {
        default-lease-time              86400;
        max-lease-time                  86400;
        option broadcast-address        192.168.10.255;
        option domain-name              "colibri.net";
        option domain-name-servers      ns1.example.org;
        option ip-forwarding            off;
        option routers                  192.168.10.1;
        option subnet-mask              255.255.255.0;
        interface                       eth1;
        range                           192.168.10.32 192.168.10.254;
}
#MAC address dependent IP assignment, used for the colibri target device
host eval {
        filename                        "uImage";
        fixed-address                   192.168.10.2;
        hardware ethernet               00:14:2d:48:8a:58;
        next-server                     192.168.10.1;
        option host-name                "colibri";
        option root-path                "192.168.10.1:/srv/nfs/rootfs,wsize=1024,rsize=1024,v3";
}

Sample root-path for NFSv4

        option root-path                "rootfs,v4,tcp,clientaddr=0.0.0.0";

Note: For NFSv4, rootfs as given above is relative to the root path which is where exports has the option "fsid=root" set (see below).


Take care to only answer DHCP requests originating from your development network card. You won't make friends by providing addresses to clients on your corporate network!

Then, start/restart the relevant services e.g. on Ubuntu:

mk@ubuntu: ~ $ sudo service isc-dhcp-server restart

e.g. on Fedora:

[root@vm_one ~]# systemctl restart dhcpd.service

Sample TFTP Configuration

Configure the server to serve /srv/tftp.
e.g. on a recent Ubuntu: /etc/default/tftpd-hpa
e.g. on a recent Fedora: /etc/xinitd.d/tftp

Copy the kernel to the tftp directory

sudo cp -Ppr /path to kernel/uImage* /srv/tftp/

Sample NFS Configuration

Copy the rootfs to /srv/nfs/rootfs (while preserving symlinks, timestamps..)

sudo cp -Ppr /path to image/rootfs/* /srv/nfs/rootfs
NFSv3 Configuration

Create an export configuration:

#/etc/exports
/srv/nfs/rootfs 192.168.10.2(no_root_squash,no_subtree_check,rw)
NFSv4 Configuration

Create an export configuration:

#/etc/exports
/srv/nfs 192.168.10.1/24(no_root_squash,no_subtree_check,rw,fsid=root)
Apply Configuration

Start/restart the relevant daemons e.g. on Ubuntu:

mk@ubuntu: ~ $sudo service nfs-kernel-server restart

e.g. on Fedora:

[root@vm_one ~]# systemctl restart nfs-server.service

U-Boot Variables for TFTP/NFS Boot

See note about the MAC address (ethaddr) at the end of the page.

The Colibri IP address used for TFTP client, the IP address of the TFTP server, the MAC address:

ipaddr=192.168.10.2
serverip=192.168.10.1
ethaddr=00:14:2d:48:8a:58

To debug mounting root file system via NFS:

defargs=nfsrootdebug

Booting

stop in U-Boot and then:

Tegra2 # run nfsboot

to make this the default boot option:

Tegra2 # setenv bootcmd 'run nfsboot'
Tegra2 # saveenv

NFS and connman

In our images we use connman to manage network connections.
When using an NFS mounted rootfs connman is not started during boot. This is because connman takes an already configured NIC down before bringing it up again. In the case of a NFS boot this unmounts the rootfs and makes the boot fail.
Should you require connman when using NFS, e.g. to do tests with wireless connectivity, you could change the connman.service file. Make sure to change eth0 to the name of the used NIC on Apalis T30. This is likely enp7s0.

--- lib/systemd/system/connman.service~ 2014-05-02 17:01:49.000000000 +0200
+++ lib/systemd/system/connman.service  2014-06-16 09:48:08.454798634 +0200
@@ -3,14 +3,15 @@
 After=syslog.target
 Before=remote-fs.target
 # only if not NFS mount, connman will disconnect your rootfs otherwise!
-ConditionKernelCommandLine=!root=/dev/nfs
+#ConditionKernelCommandLine=!root=/dev/nfs
 
 [Service]
 Type=dbus
 BusName=net.connman
 Restart=on-failure
 ExecStartPre=-/usr/lib/connman/wired-setup
-ExecStart=/usr/sbin/connmand -n
+#ExecStart=/usr/sbin/connmand -n
+ExecStart=/usr/sbin/connmand -n -I eth0
 StandardOutput=null
 
 [Install]

Various Variables

Default Boot Scenario

After a timeout U-Boot automatically executes the statements stored in the environment variable bootcmd. By default this runs flashboot, should that fail a fall back to nfsboot is programmed. To make booting from SD card the default boot scenario do the following:

Tegra2 # printenv bootcmd
bootcmd=run flashboot; run nfsboot
Tegra2 # setenv bootcmd 'run mmcboot'
Tegra2 # saveenv

USB High Speed

To enable USB high speed have a look here: USB 2.0 High Speed (480Mbps)

Ethernet MAC Address

If no environment variable 'ethaddr' exists the Ethernet MAC address is read from the 'config block'. If you want to force an Ethernet MAC address add or change the ethaddr environment variable. Config block is a data structure shared with WinCE. It contains information written during manufacturing and configuration for the WinCE boot loader. See Config Block

Tegra2 # setenv ethaddr 00:01:02:03:04:05
Tegra2 # saveenv