Instructions for building and booting Linux
These instructions explain how to build a toolchain, root filesystem and kernel for Linux running on an Xtensa processor. These instructions apply specifically to the XTAV60 (LX60) board.
Some general notes on these instructions:
- They are a work in progress. If you are using the latest development sources, the build process may change, so you should be sure to get the latest version of this document at http://wiki.linux-xtensa.org/index.php/Buildroot_Build_Instructions. If you are using one of the development snapshot releases, you should use the version of the build instructions included with that snapshot.
- The following was tested on x86 machines running RedHat Enterprise Linux 4 (RHEL4) and Fedora Core 3 (FC3). It is expected to work on newer Fedora releases (e.g., was tested on FC6) but to maintain host compatibility with Xtensa Tools, it is best to avoid Fedora releases beyond FC5. Other host distributions likely work but have not been tested.
- NOTE: this assumes the default selection of packages in buildroot. Selecting extra packages may require a more recent host OS or more recent version of certain tools. For example, the default version of find on RHEL4 is too old to allow building all X11 packages in buildroot, so it is necessary to upgrade the host version of findutils on RHEL4 to build these packages.
- Lines prefixed with "sudo" need to be executed as root. The rest is best executed as a non-root user. (It is possible to install git as a non-root user if root access is an issue. How to do this isn't shown here.)
- Lines that set environment variables assume a Bourne compatible shell (e.g., /bin/sh or bash), but are easily adapted to other shells.
- Instructions are assumed executed all in order (e.g., commands assume current directory and environment variables set earlier).
Setup your host system
If you are using one of the snapshot releases, you can skip this step.
If you want to work with the latest development sources, you will need git installed on your system.
Download buildroot and Linux kernel sources
The following section describes how to download stable versions of buildroot, and of the Linux kernel, for Xtensa cores.
First, create an empty directory <workdir> on a disk with at least 4 GB available space. After the download operations below, you'll end up with the following file structure, which is assumed by the rest of this document:
<workdir>/
/buildroot
/linux
First download the buildroot sources. The current stable source repository and branch have strange names, bear with me, they're the correct ones to use. Note: If you happen to have write access to the repository, use git+ssh: instead of git: in the clone command below, so that you can later easily push changes back to the repository. For more details, see the repository access page.
Download buildroot as follows:
$ cd <workdir> $ git clone git://git.linux-xtensa.org/git/buildroot/buildroot-xtensa-HiFi2-Snapshot.git buildroot $ cd buildroot $ git branch --track snapshot_2+SMP origin/snapshot_2+SMP $ git checkout snapshot_2+SMP
then the Linux kernel sources:
$ cd <workdir> $ git clone git://git.linux-xtensa.org/git/kernel/xtensa-2.6.29-smp.git linux
Note how in the first case (buildroot), we use a non default branch, so the last two commands setup tracking for that remote branch and does a check-out of that branch in the working tree. In the second case (linux), we use the default branch (master branch) which gets checked out by default.
Major development updates to buildroot for Xtensa are expected to happen in the following repository, mostly in the master branch. However, although it has some more recent packages, this repository is often in a state of development and not quite as stable as the above. So, DON'T DO THIS STEP unless you know what you are doing, and want to experiment with this version of buildroot instead of the above.
$ cd <workdir> $ git clone git://git.linux-xtensa.org/git/buildroot/buildroot-xtensa.git buildroot
Install any required processor specific overlay
This step is required if you are targeting a custom Xtensa processor. If you use a Diamond 232L Standard Core, you can skip this step.
To configure buildroot and the kernel to use a custom Xtensa processor configuration, you need to overlay a set of files customized for your processor configuration onto toolchain sources (gcc, binutils, gdb, etc) and the Linux kernel. This process has been automated using a script located in the buildroot source tree. For usage information, you can invoke it without arguments:
$ cd <workdir> $ ./buildroot/target/xtensa/xt-buildroot-overlay-install
which displays a help message similar to the following:
xt-buildroot-overlay-install version 1.3
Usage: xt-buildroot-overlay-install <parameters> [<options>]
Where <parameters> are:
-t file.tgz Specify path to the Xtensa Linux overlay tarball, typically
<xtensa_root>/src/xtensa-config-overlay.tar.gz
-b dir Path to the base of the buildroot source tree, in which
package specific overlay tarballs get installed.
-k dir Path to the base of the Linux kernel source tree, in which
the Linux kernel specific overlay gets installed.
-c config_name Name for the Xtensa processor configuration as it will be
known to the open source community. Must be a lowercase
identifier, starting with a letter, consisting of letters
and numbers and underscores, not ending with underscore
and not containing consecutive underscores. For examples:
dc232b , dc232b_be , mmubasele , fsf , s5000 .
-l long_name Long name for the Xtensa processor configuration, human-
readable with spaces etc allowed (must be quoted).
For example: 'Diamond 232L Standard Core Rev.B (LE)'
Try to keep it within approximately 40 characters.
And <options> are:
-f If package specific overlay tarballs already exist in
the destination source tree, overwrite them without asking.
--help Show this usage message.
Note that the Xtensa Linux overlay tarball MUST be obtained from an Xtensa core package built using release RB-2008.3 or later.
Here is an example invocation of the overlay installation script:
$ ./buildroot/target/xtensa/xt-buildroot-overlay-install -t blinkcore-config-overlay.tar.gz \
-b ./buildroot -k ./linux -c superzip -l "ChipCorp SuperZIP Blink Accelerator Core"
The script extracts some relevant information from the overlay tarball, presents it along with relevant parameters to the user, and prompts for confirmation before proceeding.
The chosen processor name (-c option) must be unique among known Xtensa processors. It is also probably wise to avoid names confusingly similar to other (non-Xtensa) processors. If you expect at some point to contribute support for your custom processor to the open-source community, this name will likely be long-lived. (One way to contribute support for a custom processor might be to submit the overlay tarball or files to the linux-xtensa.org maintainers.)
In the steps below, buildroot and the Linux kernel must be configured
(in their respective make menuconfig steps) to select
the custom Xtensa processor option.
This presents an extra configuration menu entry where you must provide
the name of the Xtensa processor. Set it to the same name
as was specified in the -c option above.
Build a toolchain and root filesystem using buildroot
Note: for more details on generic buildroot topics not covered in this simple how-to, see buildroot.uclibc.org.
Configure buildroot
The first time you configure buildroot, start with defaults for your Xtensa processor:
$ cd <workdir>/buildroot $ ./target/xtensa/setup-config <corename>
where <corename> is the overlay name of your selected Xtensa or Diamond core (e.g., dc233c). If you omit <corename>, a help message and a list of currently installed Tensilica core overlays is displayed.
Optionally, you can then customize your buildroot configuration, such as selecting additional packages, setting various parameters, and so on. The following make targets provide a curses (text-based graphical) interface for configuring buildroot, the uClibc C library, and busybox, respectively. (Note: You need the TERM environment variable properly set for these curses based tools to work.)
$ make menuconfig $ make uclibc-menuconfig $ make busybox-menuconfig
Build buildroot
Just type:
$ make
THIS WILL TAKE A LONG TIME (from about 30 minutes with defaults on a 3 GHz Pentium 4, to perhaps an hour or many depending on your system and on whether you enabled extra packages in your buildroot configuration).
Wait patiently for the build to complete. This builds an entire toolchain as well as some basic packages, and constructs a root filesystem. It might build many more packages if you selected them earlier with menuconfig.
If Something Goes Wrong
Perhaps you missed some step above and the build fails. Or you're just switching to a different processor configuration, or making some change with unknown dependencies. Either way, you'll probably need to fix the error and/or make the change, and redo the whole build. The above make command usually works fine if you just added a package using menuconfig, but otherwise does not check many dependencies, so it will usually NOT rebuild things that depend on whatever you have fixed or changed. Here's one way to retry the make from scratch without having to wipe out the buildroot directory tree completely and start again from the git cloning.
$ rm -rf *build_xtensa* binaries $ make
It is not necessary to empty the 'dl' subdirectory, which contains tarballs downloaded from the Internet, nor the .config file, which contains the buildroot configuration. Customizations to the uClibc and busybox configurations should have ended up getting saved in target/xtensa/uClibc.config and target/xtensa/busybox-config, respectively, so hopefully are preserved by the above sequence.
Build and Run a Linux Kernel on the XTAV60, AXAV110, or XTAV200 Boards
References to the XTAV60 board refer to the combination of the Avnet LX60 (Xilinx) Development Board and a Tensilica-provided FPGA bitstream containing a configured Tensilica processor and basic peripheral IP. For more details, please refer to the Tensilica Avnet LX60 (XT-AV60) Board User's Guide.
Similarly, references to the XTAV200 board refers the combination of the Avnet LX200 (Xilinx) Development Board and a Tensilica-provided FPGA bitstream containing a configured Tensilica processor and basic peripheral IP. For more details, please refer to the Tensilica Avnet LX200 (XT-AV200) Board User's Guide.
Configure the kernel
The kernel build is a two-step process: configure the kernel, and build it. First, before we can do anything with the kernel, set your PATH to point to the toolchain built using buildroot.
$ export PATH="<workdir>/buildroot/build_xtensa_<cname>/staging_dir/usr/bin:$PATH"
where <cname> is the name of the Xtensa processor configuration you selected
when building the toolchain (e.g., dc232b).
Now setup a destination for kernel builds. We usually like to have the build directory separate from the source directory, so we use 'O=</destination/path>' in kernel make commands. If you'll be routinely building multiple configurations of the Linux kernel, it's good to have some naming conventions for the build directory. For example, to experiment with multiple target platforms but only one processor configuration, we include just the platform name here:
$mkdir build-xtav60
Let's start with the default kernel configuration for the Avnet board (which works on both XTAV60 (LX60), XTAV110 (LX110), and XTAV200 (LX200) boards) as follows:
$ cd linux $ make O=../build-xtav60 ARCH=xtensa KBUILD_DEFCONFIG=lx60_defconfig defconfig
This configures the kernel using the default configuration found in arch/xtensa/configs/lx60_defconfig.
NOTE: This kernel configuration has nothing to do with Xtensa processor configuration. That comes next.
Let's now customize this a little bit to select the correct Xtensa processor configuration, and to bundle the root filesystem into the kernel:
$ make O=../build-xtav60 ARCH=xtensa menuconfig
Under General setup, make sure the Initial RAM filesystem and RAM disk (initramfs/initrd) support entry is enabled. Underneath that, edit Initramfs source file(s), and enter the path to the cpio formatted root filesystem generated by buildroot:
<workdir>/buildroot/binaries/uclibc/rootfs.xtensa_<cname>.cpio.gz
(Don't forget to replace <workdir> with the appropriate full path and <cname> with your Xtensa processor configuration name.)
Under Processor type and features, make sure the Xtensa Processor Configuration is correct and matches the processor configuration name you used to build the toolchain with buildroot (for example, dc232b for Diamond 232L Rev.B). The kernel build selects a toolchain and processor variant specific headers based on this selection. If the desired processor configuration does not appear explicitly in the list, simply select Custom Xtensa processor configuration and enter the processor configuration name (all lowercase) into the next menu parameter, Xtensa Processor Custom Variant Name.
Back in the main menu, under Platform options, make sure the CPU clock rate entry is set correctly according to the following table:
| Board | Frequency selected in XPG (MHz) |
CPU clock rate setting (kHz) |
|---|---|---|
| XTAV60 (LX60) | 30 | 33333 |
| 40 | 40000 | |
| 50 | 50000 | |
| XTAV110 (LX110) | 30 | 33333 |
| 40 | 40000 | |
| 50 | 50000 | |
| XTAV200 (LX200) | 30 | 31250 |
| 40 | 41667 | |
| 50 | 50000 |
Exit menuconfig, saving your configuration changes.
Backup your configuration outside the build directory. For example:
$ cp ../build-xtav60/.config config.xtav60.saved
If you do a clean rebuild of the kernel (e.g., "rm -rf ../build-xtav60") you can now configure it with simply:
$ mkdir ../build-xtav60 ; cp config.xtav60.saved ../build-xtav60/.config
Of course if you update the kernel, you'll probably again have to do:
$ make O=../build-xtav60 ARCH=xtensa menuconfig
and save a new copy of your configuration.
Build the kernel
$ make O=../build-xtav60 ARCH=xtensa
The build will issue a few warnings which are normal. Once complete, the bootable image is in:
<workdir>/build-xtav60/arch/xtensa/boot/Image.elf
and the uncompressed ELF file containing kernel symbols is in:
<workdir>/build-xtav60/vmlinux
There is also a RedBoot bootable image (if you use RedBoot) in:
<workdir>/build-xtav60/arch/xtensa/boot/zImage.redboot
Install Xtensa Tools for the selected core
If you opt to download the kernel over OCD, you need xt-gdb which is part of Tensilica's Xtensa Tools package. This must be obtained from Tensilica, and is not the same toolchain as built by buildroot. You also need to install the corresponding Tensilica core package, so that the Xtensa Tools know about your particular configured Tensilica core.
The toolchain built using buildroot can be configured to include GDB, but that version of GDB does not support communicating with Tensilica's Xtensa OCD Daemon, and thus cannot be used to download the kernel over OCD.
In this example, it is assumed that the machine hosting the Xtensa Tools has access to the <workdir> directory tree.
Install and setup the Xtensa OCD Daemon
Setup and connect your JTAG probe, etc.
Connect a terminal server
At 38400 bps 8N1 no flow control to the XTAV60 serial port.
Optionally, setup networking
This step is optional.
Setup the board's MAC address using dipswitches (see XTAV60 or XTAV200 board docs as appropriate) and connect the board to a network that has a DHCP server that will respond to that MAC address. If you don't do this step, you'll simply not have network access, and the boot process will take a minute or so longer while the kernel times out waiting for a BOOTP response. You can edit the kernel configuration and rebuild the kernel to avoid using bootp (in particular, remove "ip=bootp" from the kernel cmdline).
Download and run the kernel
Reset the board (see board documentation).
Using Xtensa Tools, invoke:
$ xt-gdb <workdir>/build-xtav60/arch/xtensa/boot/Image.elf
(xt-gdb) target remote <ocdhost>:20000 0 (xt-gdb) reset (xt-gdb) load (xt-gdb) set $pc = &_ResetVector (xt-gdb) symbol-file <workdir>/build-xtav60/vmlinux (xt-gdb) c
where <ocdhost> is the IP address or DNS name of the machine running the Xtensa OCD daemon. The Linux kernel should start booting as soon as 'c' (continue) is executed.
You should eventually get a login prompt. Just login as root (no password).
Try various Linux commands. Look at /bin, /sbin, /usr/bin, etc to see what's available.
Note: no need to setup an NFS or TFTP server. The filesystem is contained within the kernel image. You may be able to mount other filesystems over NFS though, if you wish, after booting.
Recently the MMU was modified to come up uninitialized and it's necessary for the Linux kernel to map the MMU prior to running. This means you can't place breakpoints in the kernel until the mapping has been completed. It's likely worth while looking at the example .xt-gdbinit file in the Kernel Xtensa Documentation:
xtensa-2.6.29-smp-xcc-O3/Documentation/xtensa/gdbmacros/xt-gdbinit
When running on Avnet boards it's necessary to use a HardWare Breakpoint at the kernel symbol set_breakpoints which is near _startup and set your early kernel breakpoints once you get to this hardware breakpoint.
With ISS simulation normal breakpoints must be used.
Below is an example snippet from the sample xt-gdbinit script:
if $debug_hw_breakpoints_supported
hbreak set_breakpoints
else
break set_breakpoints
end
set var $_startup = $bpnum
commands $_startup
set_breakpoints
delete $_startup
info breakpoints
set var $doing_commands = 0
end
Mounting the Root Filesystem Over NFS
If you build more than the minimal set of packages using buildroot, the root filesystem can easily grow too large to fit comfortably (or at all) inside the kernel. In this case, rather than follow the instructions exactly as above, you'll want to mount the root filesystem over NFS.
You may also want to mount the root filesystem over NFS to make changes to the filesystem persistent across runs and accessible from your development host.
First you need to install the root filesystem generated using buildroot, to a machine that can serve it over NFS. Assuming this machine runs Linux, one way to install it is as follows:
$ mkdir <exportpath> $ cd <exportpath> $ sudo cpio -i -d -m -F <workdir>/buildroot/binaries/uclibc/rootfs.xtensa_<cname>.cpio --no-absolute-filenames
You then need to enable the NFS server on your system (if not already active), and export this filesystem. How you do this depends on your host Linux distribution. For example, on some Fedora releases, one might:
- add the export path to /etc/exports using a line such as
<exportpath> *(rw,sync,no_root_squash,no_all_squash,no_wdelay)
- turn on the NFS server if needed:
$ sudo /sbin/chkconfig nfs on
- tell the NFS server about the new exported filesystem
$ sudo /usr/sbin/exportfs -a $ sudo /usr/sbin/exportfs -r
Second, follow the instructions in the previous sections, then configure the kernel again as follows.
Under General setup, make sure the Initial RAM filesystem and RAM disk (initramfs/initrd) support entry is disabled (rather than enabled as done in previous sections).
Under Platform options, under the line Default bootloader kernel arguments, select the boot parameters line and set it to something like this:
console=ttyS0,38400 ip=dhcp root=/dev/nfs rw nfsroot=<server_ip>:<exportpath>
where <server_ip> is the IP address of your NFS server, and <exportpath> is the exported filesystem path.
Another approach is to just specify the ip addresses directly in the CMDLINE
CONFIG_CMDLINE="console=ttyS0,38400 ip=192.168.11.95:192.168.11.220:192.168.11.1:255.255.255.0:HiFi-2 root=/dev/nfs rw nfsroot=192.168.11.55:/exports/LINUX_ROOT.DC_C_233L debug coredump_filter=0xff"
Where the IP addresses are in the form:
${ipaddr}:${nfsroot_server}:${gatewayip}:${netmask}:${hostname}
Save this configuration, rebuild the kernel, and follow the instructions in previous sections to download and run the kernel. If everything goes well and is setup correctly, your kernel should now boot with a root filesystem mounted over NFS.
Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)
You can build and run a Linux kernel in the Xtensa Instruction Set Simulator (ISS). The following instructions have only been tried with an ISS from the RB-2008.3 release of Xtensa Tools. Bear in mind that the XTAV60 port is likely much more stable than the ISS port of Linux at this point in time.
Note: Depending on your host OS version, you may need to install Xtensa Tools on a separate machine.
Configure and Build the Kernel for ISS
Repeat all the same instructions as above for building the kernel for an Avnet board, except that when initially configuring the kernel, start from a ISS platform template instead. For example for the DC233C Variant:
$ mkdir build-iss $ cd linux $ make O=../build-iss ARCH=xtensa KBUILD_DEFCONFIG=iss_dc233c_defconfig defconfig
The above make example configures the kernel using the default configuration found in arch/xtensa/configs/iss_dc233_defconfig. Most important to note is that the root file system mounted when Linux boots comes via the targets device /dev/simdisk0 which is defined in the kernel .config file. For example:
CONFIG_SIMDISK0_FILENAME="/export2/DC_C_233L/LINUX_ROOT.ext2"
The SIMDISK file name and VARIANT name are typically configured when running make menuconfig; Ex:
$ make O=../build-iss ARCH=xtensa menuconfig
and selecting
Xtensa Platform options ---> Host filename for a simulated device
and entering the pathname to the .ext2 file there. Ex: /export2/DC_C_233L/LINUX_ROOT.ext2
You often want to change the VARIANT to the one you desire, Ex: FSF. You do this via
the top level menu:
Xtensa Processor type and features ---> Xtensa Processor Custom Core Variant Name
With the new SIMDISK approach you copy the EXT2 Filesystem file generated by Buildroot
to a location to be accessed by Xtensa Linux. Ex:
$ cp /export2/DC_C_233L/src/buildroot-xtensa-HiFi2-Snapshot/binaries/dc233c/rootfs.xtensa_dc233c.ext2 /export2/DC_C_233L/LINUX_ROOT.ext2
NOTE: There is an old iss_defconfig that tries to use a tuntap Ethernet interface that currently isn't supported. Previously with this old iss_defconfig using the tuntap Ethernet interface it was suggested to set up the initramfs filesystem, and do the following. Under Bus Options, deselect PCI support. Under Platform Options, deselect Default bootloader kernel arguments. Hopefully we will someday be able to support the use of a network interface with ISS simulation and then be able to mount NFS root partitions again. So I'm leaving this note for future exploration of this mechanism; Hope you don't mind the clutter.
Continue configuring and building the kernel as usual.
Run the Kernel on ISS
Using Xtensa Tools, invoke:
$ xt-gdb <workdir>/build-iss/arch/xtensa/boot/Image.elf (xt-gdb) target sim --turbo --memlimit=128 (xt-gdb) symbol-file <workdir>/build-iss/vmlinux (xt-gdb) run
Wait patiently while Linux boots ... (maybe a minute). The setting of the memlimit needs to match the PLATFORM_DEFAULT_MEM_SIZE defined in /arch/xtensa/platforms/iss/include/platform/hardware.h. Currently this limit us 128 MegaBytes:
#define PLATFORM_DEFAULT_MEM_SIZE 0x08000000 /* 13,4217,728 128M */
and we will be extending this for the DC233L to take advantage of it's V3 MMU.
Login as root (no password).
Note: Input via ISS is cooked, so everything typed gets echoed. Also, time as reported by the kernel does not progress according to true wall-clock time: it currently depends on a simulated processor clock, which advances at various speeds according to load.