Linux/Xtensa - User contributions [en]

Linux Kernel

2022-04-21T11:22:15Z

Jcmvbkbc: update features by release section

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see kernel .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.19 ==
* support coprocessors on SMP.
* support protection faults on noMMU.
* support full tickless operation.
* KCSAN.
* hibernation.

== Linux 5.18 ==
* support compiler plugins (with gcc-12 and newer).
* support for kernel ABI selection.
* removed dependency on compiler's libgcc.

== Linux 5.16 ==
* support for cores without windowed registers.
* call0 ABI kernel.

== Linux 5.13 ==
* GDBIO semihosting.

== Linux 5.9 ==
* syscall audit.
* seccomp filter.

== Linux 5.6 ==
* exception vectors can be attached to the rest of the kernel or placed at specific address.

== Linux 5.5 ==
* eXecute In Place (XIP) kernel.

== Linux 5.4 ==
* support for call0 ABI userspace.

== Linux 5.3 ==
* 'virt' board (sim with PCI, corresponding board model available in QEMU since v4.2).

== Linux 5.2 ==
* support for cores with exclusive access option.
* basic support for cores with MPU.

== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump labels.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2021-11-08T05:17:58Z

Jcmvbkbc: update features by release section

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see kernel .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.16 ==
* support for cores without windowed registers
* call0 ABI kernel

== Linux 5.13 ==
* GDBIO semihosting

== Linux 5.9 ==
* syscall audit.
* seccomp filter.

== Linux 5.6 ==
* exception vectors can be attached to the rest of the kernel or placed at specific address.

== Linux 5.5 ==
* eXecute In Place (XIP) kernel.

== Linux 5.4 ==
* support for call0 ABI userspace.

== Linux 5.3 ==
* 'virt' board (sim with PCI, corresponding board model available in QEMU since v4.2).

== Linux 5.2 ==
* support for cores with exclusive access option.
* basic support for cores with MPU.

== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump labels.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2020-08-06T18:31:18Z

Jcmvbkbc: fix typo

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see kernel .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.9 ==
* syscall audit.
* seccomp filter.

== Linux 5.6 ==
* exception vectors can be attached to the rest of the kernel or placed at specific address.

== Linux 5.5 ==
* eXecute In Place (XIP) kernel.

== Linux 5.4 ==
* support for call0 ABI userspace.

== Linux 5.3 ==
* 'virt' board (sim with PCI, corresponding board model available in QEMU since v4.2).

== Linux 5.2 ==
* support for cores with exclusive access option.
* basic support for cores with MPU.

== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump labels.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2020-08-06T18:09:45Z

Jcmvbkbc: update features by release section

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see kernel .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.9 ==
* syscall audit.
* seccomp filter.

== Linux 5.6 ==
* exception vectors can be attached to the rest of the kernel or placed at specific address.

== Linux 5.5 ==
* eXecute In Place (XIP) kernel.

== Linux 5.4 ==
* support for call0 ABI userspace.

== Linux 5.3 ==
* 'virt' board (sim with PCI, corresponding board model available in QEMU since v4.2).

== Linux 5.2 ==
* support for cores with exclusive access option.
* basic support for cores with MPU.

== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump lables.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Instructions for building and booting Linux

2019-08-30T01:21:05Z

Jcmvbkbc: building kernel on its own

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa cores supported by Linux ==

Official requirements for Xtensa core to run Linux are listed in the Xtensa OSKit Guide document.
The absolute minimum set of options required to run Linux is windowed registers, exception option and one level-1 or medium level timer interrupt.
* Full MMU option is required for full featured Linux kernel that includes process address space isolation, virtual memory and fork syscall.
* Full MMU option, thread pointer option and conditional store option are required for running userspace with NPTL or TLS.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Overlay file for custom configuration ' to the location of the overlay archive (it may be a path to a local file or an URL).

Select toolchain options, packages to be built and final root file system image type. Select whether the kernel need to be built, kernel version and configuration.

Build the toolchain, kernel and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== The kernel may be built on its own ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Instructions for building and booting Linux

2019-08-30T01:19:13Z

Jcmvbkbc: building kernel inside the buildroot

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa cores supported by Linux ==

Official requirements for Xtensa core to run Linux are listed in the Xtensa OSKit Guide document.
The absolute minimum set of options required to run Linux is windowed registers, exception option and one level-1 or medium level timer interrupt.
* Full MMU option is required for full featured Linux kernel that includes process address space isolation, virtual memory and fork syscall.
* Full MMU option, thread pointer option and conditional store option are required for running userspace with NPTL or TLS.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Overlay file for custom configuration ' to the location of the overlay archive (it may be a path to a local file or an URL).

Select toolchain options, packages to be built and final root file system image type. Select whether the kernel need to be built, kernel version and configuration.

Build the toolchain, kernel and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Linux Kernel

2019-08-30T00:52:42Z

Jcmvbkbc:

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see kernel .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump lables.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2019-08-30T00:52:08Z

Jcmvbkbc: update Rootfs in initial RAMFS

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==

Initial RAMFS is a CPIO archive that the kernel may use to populate a tmpfs and then use as a root file system.
It may be integrated into the kernel image (see .config variable CONFIG_INITRAMFS_SOURCE) or loaded into memory separately by the bootloader. In the latter case its load address is passed to the kernel in the boot parameters block.

When Buildroot is used to generate root file system it can produce cpio image.
When Buildroot is also used to build the kernel it can be directed to build the root file system into the kernel.
See Buildroot .config symbols BR2_TARGET_ROOTFS_CPIO and BR2_TARGET_ROOTFS_INITRAMFS.

== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump lables.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2019-03-29T22:07:39Z

Jcmvbkbc: update xtensa status

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 5.1 ==
* generic spinlock/rwlock.

== Linux 5.0 ==
* memtest.
* Jump lables.
* tracehook.
* syscall tracepoints.
* multiple coprocessor fixes.
* SMP fixes.

== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

ABI Interface

2018-11-10T01:17:58Z

Jcmvbkbc: fix ordinary call return value description

== Argument Passing ==

Arguments are passed in both registers and memory. The first six incoming arguments are stored in registers a2 through a7, and additional arguments are stored on the stack starting at the current stack pointer a1. Because Xtensa uses register windows that rotate during a function call, outgoing arguments that will become the incoming arguments must be stored to different register numbers. Depending on the call instruction and, thus, the rotation of the register window, the arguments are passed starting starting with register a(2+N), where N is the size of the window rotation. Therefore, the first argument in case of a <code>call4</code> instruction is placed into a6, and for a <code>call8</code> instruction into a10. Large arguments (8-bytes) are always passed in an even/odd register pair even if that means to omit a register for alignment. The return values are stored in a2 through a5 (so a function with return value occupying more than 2 registers may not be called with call12).

return addr stack ptr arg0, arg1, arg2, arg3, arg4, arg5
----------- --------- ----------------------------------
a0 a1 a2, a3, a4, a5, a6, a7

call4 a4 a5 a6, a7, a8, a9, a10, a11
call8 a8 a9 a10, a11, a12, a13, a14, a15
call12 a12 a13 a14, a15 --- --- --- ---

== Syscall ABI ==

Linux takes system-call arguments in registers. The ABI and Xtensa software conventions require the system-call number in a2. For improved efficiency, we try not to shift all parameters
one register up to maintain the original order. Register a2 is, therefore, moved to a6, a6 to a8, and a7 to a9, if the system call requires these arguments.

syscall number arg0, arg1, arg2, arg3, arg4, arg5
-------------- ----------------------------------
a2 a6, a3, a4, a5, a8, a9

Upon return, a2 contains the return value or error code. All other registers are preserved.

== TLS and NPTL implementation ==

This is a description of the Xtensa-specific aspects of supporting thread-local storage (TLS). Originally developed by Sun, the implementation concept for TLS has been modified and generalized by Ulrich Drepper for Linux for a variety of platforms. [http://people.redhat.com/drepper/tls.pdf "ELF Handling for Thread-Local Storage"] provides background information and descriptions of the machine-independent aspects. Alex Oliva later developed an alternative and heavily optimized version for a Linux port to the FR-V processor and subsequently ported it to x86, x86_64, and ARM Linux systems. See also Oliva's implementation details for [http://www.lsd.ic.unicamp.br/~oliva/writeups/TLS/RFC-TLSDESC-x86.txt X86] and [http://www.lsd.ic.unicamp.br/~oliva/writeups/TLS/RFC-TLSDESC-ARM.txt ARM]. The current status of this alternative approach is somewhat ambiguous: it has been accepted into GCC and Binutils but the glibc patches have not made it into the mainline tree, yet. As a new port, however, Xtensa can uses this optimized implementation without affecting legacy software. Although it would be convenient to share some of the platform-independent code, it is not required and will work regardless of whether those patches are ever accepted.

== Run-Time Handling of TLS ==

The __tls_get_addr function has a different prototype with this approach:

<code>extern void *__tls_get_addr (tls_index *ti);</code>

(This is actually the same prototype for __tls_get_addr that is used for most processors with the standard approach.)

== TLS Access Models ==

The Initial Exec and Local Exec models are the same as with the standard approach and are not shown here.

=== General Dynamic Model ===

For the general dynamic model, the code sequence loads a function pointer and a single argument from the GOT and calls the function with that argument. If the thread-local symbol is in the static TLS, the runtime linker will set the argument to the TLS offset and the function to a short routine that returns the offset plus the thread pointer. Otherwise, the function is set to __tls_get_addr and the argument is a pointer to the tls_index structure holding the offset and module values. This tls_index structure is dynamically allocated in a hash table by the runtime linker.

This approach allows lazy relocation processing for TLS references, which should improve the start-up times. The runtime linker can initialize the TLS function pointer in the GOT to a resolver function. The details of this are presumably similar to other processors and are not yet specified for Xtensa.

{| border='1' cellspacing='0' width='100%'
|-
! Location !! Code Sequence !! Initial Relocations !! Dynamic Relocations
|-
|
.literal
 
|
.LC0
.LC1
|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
|-
|
.text
 
 
|
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
|
R_XTENSA_TLS_FUNC (x)
R_XTENSA_TLS_ARG (x)
R_XTENSA_TLS_CALL (x)
|
 
 
 
|-
|}

The assembly syntax for this instruction sequence is:

movi a8, x@TLSFUNC
movi a10, x@TLSARG
callx8.tls a8, x@TLSCALL

This relies on the assembler to relax the <code>movi</code> instructions to <code>l32r</code> instructions, so that both the literals and the instructions get the appropriate relocations. The <code>callx8.tls</code> assembler macro generates a <code>callx8</code> instruction with an extra relocation specified by its second operand.

=== Local Dynamic Model ===

This model uses the same relocations and functions as the general dynamic model except that it references a special linker-defined _TLS_MODULE_BASE_ symbol which is set to the start of the local TLS space.

{| border='1' cellspacing='0' width='100%'
|-
! Location !! Code Sequence !! Initial Relocations !! Dynamic Relocations
|-
|
.literal
 
 
 
|
.LC0
.LC1
.LC2
.LC3
|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_DTPOFF (x)
R_XTENSA_TLS_DTPOFF (y)
|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
 
 
|-
|
.text
 
 
 
 
 
 
 
 
|
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
...
0x0f l32r a13, .LC3
0x12 add a13, a13, a10
|
R_XTENSA_TLS_FUNC (_TLS_MODULE_BASE_)
R_XTENSA_TLS_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_CALL (_TLS_MODULE_BASE_)
 
 
 
 
 
 
|
 
 
 
 
 
 
 
 
 
|-
|}

The assembly syntax for this code sequence is:

movi a8, _TLS_MODULE_BASE_@TLSFUNC
movi a10, _TLS_MODULE_BASE_@TLSARG
callx8.tls a8, _TLS_MODULE_BASE_@TLSCALL
...
movi a12, x@DTPOFF
add a12, a12, a10
...
movi a13, y@DTPOFF
add a13, a13, a10

== Linker Optimizations ==

=== General Dynamic -> Initial Exec ===

{| border='1' cellspacing='0' width='100%'
|-
|width='30%'|
.literal .LC0
.literal .LC1
 
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
|width='18%'|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
 
R_XTENSA_TLS_FUNC (x)
R_XTENSA_TLS_ARG (x)
R_XTENSA_TLS_CALL (x)
| ->
|width='30%'|
 
.literal .LC1
 
0x00 rur a8, THREADPTR
0x03 l32r a10, .LC1
0x06 add a10, a10, a8
|width='18%'|
 
R_XTENSA_TLS_TPOFF (x)
 
 
 
 
|-
|}

=== Local Dynamic -> Local Exec ===

{| border='1' cellspacing='0' width='100%'
|-
|width='30%'|
.literal .LC0
.literal .LC1
.literal .LC2
 
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
|width='18%'|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_DTPOFF (x)
 
R_XTENSA_TLS_FUNC (_TLS_MODULE_BASE_)
R_XTENSA_TLS_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_CALL (_TLS_MODULE_BASE_)
 
 
 
| ->
|width='30%'|
 
 
.literal .LC2
 
0x00 nop
0x03 nop
0x06 rur a10, THREADPTR
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
|width='18%'|
 
 
R_XTENSA_TLS_TPOFF (x)
 
 
 
 
 
 
 
|-
|}

== New Xtensa ELF Definitions ==

.literal
R_XTENSA_TLSDESC_FN
R_XTENSA_TLSDESC_ARG
R_XTENSA_TLS_DTPOFF
R_XTENSA_TLS_TPOFF

.text
R_XTENSA_TLS_FUNC
R_XTENSA_TLS_ARG
R_XTENSA_TLS_CALL

ABI Interface

2018-11-09T23:46:57Z

Jcmvbkbc: Fix syscall return value description.

== Argument Passing ==

Arguments are passed in both registers and memory. The first six incoming arguments are stored in registers a2 through a7, and additional arguments are stored on the stack starting at the current stack pointer a1. Because Xtensa uses register windows that rotate during a function call, outgoing arguments that will become the incoming arguments must be stored to different register numbers. Depending on the call instruction and, thus, the rotation of the register window, the arguments are passed starting starting with register a(2+N), where N is the size of the window rotation. Therefore, the first argument in case of a <code>call4</code> instruction is placed into a6, and for a <code>call8</code> instruction into a10. Large arguments (8-bytes) are always passed in an even/odd register pair even if that means to omit a register for alignment. The return values are stored in a2 through a7.

return addr stack ptr arg0, arg1, arg2, arg3, arg4, arg5
----------- --------- ----------------------------------
a0 a1 a2, a3, a4, a5, a6, a7

call4 a4 a5 a6, a7, a8, a9, a10, a11
call8 a8 a9 a10, a11, a12, a13, a14, a15
call12 a12 a13 a14, a15 --- --- --- ---

== Syscall ABI ==

Linux takes system-call arguments in registers. The ABI and Xtensa software conventions require the system-call number in a2. For improved efficiency, we try not to shift all parameters
one register up to maintain the original order. Register a2 is, therefore, moved to a6, a6 to a8, and a7 to a9, if the system call requires these arguments.

syscall number arg0, arg1, arg2, arg3, arg4, arg5
-------------- ----------------------------------
a2 a6, a3, a4, a5, a8, a9

Upon return, a2 contains the return value or error code. All other registers are preserved.

== TLS and NPTL implementation ==

This is a description of the Xtensa-specific aspects of supporting thread-local storage (TLS). Originally developed by Sun, the implementation concept for TLS has been modified and generalized by Ulrich Drepper for Linux for a variety of platforms. [http://people.redhat.com/drepper/tls.pdf "ELF Handling for Thread-Local Storage"] provides background information and descriptions of the machine-independent aspects. Alex Oliva later developed an alternative and heavily optimized version for a Linux port to the FR-V processor and subsequently ported it to x86, x86_64, and ARM Linux systems. See also Oliva's implementation details for [http://www.lsd.ic.unicamp.br/~oliva/writeups/TLS/RFC-TLSDESC-x86.txt X86] and [http://www.lsd.ic.unicamp.br/~oliva/writeups/TLS/RFC-TLSDESC-ARM.txt ARM]. The current status of this alternative approach is somewhat ambiguous: it has been accepted into GCC and Binutils but the glibc patches have not made it into the mainline tree, yet. As a new port, however, Xtensa can uses this optimized implementation without affecting legacy software. Although it would be convenient to share some of the platform-independent code, it is not required and will work regardless of whether those patches are ever accepted.

== Run-Time Handling of TLS ==

The __tls_get_addr function has a different prototype with this approach:

<code>extern void *__tls_get_addr (tls_index *ti);</code>

(This is actually the same prototype for __tls_get_addr that is used for most processors with the standard approach.)

== TLS Access Models ==

The Initial Exec and Local Exec models are the same as with the standard approach and are not shown here.

=== General Dynamic Model ===

For the general dynamic model, the code sequence loads a function pointer and a single argument from the GOT and calls the function with that argument. If the thread-local symbol is in the static TLS, the runtime linker will set the argument to the TLS offset and the function to a short routine that returns the offset plus the thread pointer. Otherwise, the function is set to __tls_get_addr and the argument is a pointer to the tls_index structure holding the offset and module values. This tls_index structure is dynamically allocated in a hash table by the runtime linker.

This approach allows lazy relocation processing for TLS references, which should improve the start-up times. The runtime linker can initialize the TLS function pointer in the GOT to a resolver function. The details of this are presumably similar to other processors and are not yet specified for Xtensa.

{| border='1' cellspacing='0' width='100%'
|-
! Location !! Code Sequence !! Initial Relocations !! Dynamic Relocations
|-
|
.literal
 
|
.LC0
.LC1
|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
|-
|
.text
 
 
|
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
|
R_XTENSA_TLS_FUNC (x)
R_XTENSA_TLS_ARG (x)
R_XTENSA_TLS_CALL (x)
|
 
 
 
|-
|}

The assembly syntax for this instruction sequence is:

movi a8, x@TLSFUNC
movi a10, x@TLSARG
callx8.tls a8, x@TLSCALL

This relies on the assembler to relax the <code>movi</code> instructions to <code>l32r</code> instructions, so that both the literals and the instructions get the appropriate relocations. The <code>callx8.tls</code> assembler macro generates a <code>callx8</code> instruction with an extra relocation specified by its second operand.

=== Local Dynamic Model ===

This model uses the same relocations and functions as the general dynamic model except that it references a special linker-defined _TLS_MODULE_BASE_ symbol which is set to the start of the local TLS space.

{| border='1' cellspacing='0' width='100%'
|-
! Location !! Code Sequence !! Initial Relocations !! Dynamic Relocations
|-
|
.literal
 
 
 
|
.LC0
.LC1
.LC2
.LC3
|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_DTPOFF (x)
R_XTENSA_TLS_DTPOFF (y)
|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
 
 
|-
|
.text
 
 
 
 
 
 
 
 
|
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
...
0x0f l32r a13, .LC3
0x12 add a13, a13, a10
|
R_XTENSA_TLS_FUNC (_TLS_MODULE_BASE_)
R_XTENSA_TLS_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_CALL (_TLS_MODULE_BASE_)
 
 
 
 
 
 
|
 
 
 
 
 
 
 
 
 
|-
|}

The assembly syntax for this code sequence is:

movi a8, _TLS_MODULE_BASE_@TLSFUNC
movi a10, _TLS_MODULE_BASE_@TLSARG
callx8.tls a8, _TLS_MODULE_BASE_@TLSCALL
...
movi a12, x@DTPOFF
add a12, a12, a10
...
movi a13, y@DTPOFF
add a13, a13, a10

== Linker Optimizations ==

=== General Dynamic -> Initial Exec ===

{| border='1' cellspacing='0' width='100%'
|-
|width='30%'|
.literal .LC0
.literal .LC1
 
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
|width='18%'|
R_XTENSA_TLSDESC_FN (x)
R_XTENSA_TLSDESC_ARG (x)
 
R_XTENSA_TLS_FUNC (x)
R_XTENSA_TLS_ARG (x)
R_XTENSA_TLS_CALL (x)
| ->
|width='30%'|
 
.literal .LC1
 
0x00 rur a8, THREADPTR
0x03 l32r a10, .LC1
0x06 add a10, a10, a8
|width='18%'|
 
R_XTENSA_TLS_TPOFF (x)
 
 
 
 
|-
|}

=== Local Dynamic -> Local Exec ===

{| border='1' cellspacing='0' width='100%'
|-
|width='30%'|
.literal .LC0
.literal .LC1
.literal .LC2
 
0x00 l32r a8, .LC0
0x03 l32r a10, .LC1
0x06 callx8 a8
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
|width='18%'|
R_XTENSA_TLSDESC_FN (_TLS_MODULE_BASE_)
R_XTENSA_TLSDESC_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_DTPOFF (x)
 
R_XTENSA_TLS_FUNC (_TLS_MODULE_BASE_)
R_XTENSA_TLS_ARG (_TLS_MODULE_BASE_)
R_XTENSA_TLS_CALL (_TLS_MODULE_BASE_)
 
 
 
| ->
|width='30%'|
 
 
.literal .LC2
 
0x00 nop
0x03 nop
0x06 rur a10, THREADPTR
...
0x09 l32r a12, .LC2
0x0c add a12, a12, a10
|width='18%'|
 
 
R_XTENSA_TLS_TPOFF (x)
 
 
 
 
 
 
 
|-
|}

== New Xtensa ELF Definitions ==

.literal
R_XTENSA_TLSDESC_FN
R_XTENSA_TLSDESC_ARG
R_XTENSA_TLS_DTPOFF
R_XTENSA_TLS_TPOFF

.text
R_XTENSA_TLS_FUNC
R_XTENSA_TLS_ARG
R_XTENSA_TLS_CALL

UClinux

2018-04-24T20:56:00Z

Jcmvbkbc: uclinux support status update

= Toolchain with uClinux support =

Xtensa uClinux userspace is based on bFLT executable file format.

* gcc supports xtensa*-*-uclinux* targets starting with version 6.1.

= Linux kernel with noMMU support =

Linux kernel mainline supports noMMU configurations starting with version 3.19.

= Buildroot with noMMU support =

Buildroot mainline supports noMMU configurations starting with version 2016.05.

= QEMU =

QEMU supports nommu variants of all XTFPGA machines starting with version 2.12.

Linux Kernel

2018-04-24T20:39:55Z

Jcmvbkbc: drop special uclinux case

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Buildroot

2018-04-24T20:38:56Z

Jcmvbkbc: move modern buildroot xtensa overlay configuration option to the top

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain, a root filesystem, linux kernel and the U-Boot bootloader.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** release 2017.08 and newer:
*** BR2_XTENSA_OVERLAY_FILE: path to the overlay file. May be relative or absolute, may also be an URL.
** release 2017.05 and older:
*** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
*** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2017.08 ==
* overlay overhaul: now overlay may be compressed, may have arbitrary name and may be specified as an URL
* overlay is applied to linux
* overlay is applied to U-Boot

== 2017.05 ==
* build xtensa linux kernel uImage

== 2016.05 ==
* support noMMU xtensa cores

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Buildroot

2018-04-24T20:35:17Z

Jcmvbkbc: drop special uclinux case

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain, a root filesystem, linux kernel and the U-Boot bootloader.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** release 2017.05 and older:
*** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
*** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
** release 2017.08 and newer:
*** BR2_XTENSA_OVERLAY_FILE: path to the overlay file. May be relative or absolute, may also be an URL.
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2017.08 ==
* overlay overhaul: now overlay may be compressed, may have arbitrary name and may be specified as an URL
* overlay is applied to linux
* overlay is applied to U-Boot

== 2017.05 ==
* build xtensa linux kernel uImage

== 2016.05 ==
* support noMMU xtensa cores

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Instructions for building and booting Linux

2018-04-24T20:33:45Z

Jcmvbkbc: drop special uclinux case

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa cores supported by Linux ==

Official requirements for Xtensa core to run Linux are listed in the Xtensa OSKit Guide document.
The absolute minimum set of options required to run Linux is windowed registers, exception option and one level-1 or medium level timer interrupt.
* Full MMU option is required for full featured Linux kernel that includes process address space isolation, virtual memory and fork syscall.
* Full MMU option, thread pointer option and conditional store option are required for running userspace with NPTL or TLS.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Linux Kernel

2018-04-02T16:04:49Z

Jcmvbkbc: add new features in 4.16

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.16 ==
* KASAN.
* SSP.
* allocation of DMA buffers from high memory.
* support DMA_ATTR_NO_KERNEL_MAPPING.
* kernel is built with -mtext-section-literals and -mno-serialize-volatile.

== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Xtensa on QEMU

2018-03-09T17:55:32Z

Jcmvbkbc: update core import script location

== Setting up and running Xtensa simulation on QEMU ==

[http://wiki.qemu.org QEMU] is a free processor/machine emulator and virtualizer.

Starting with version 1.0 QEMU offers [http://wiki.qemu.org/Features/Xtensa Xtensa architecture emulation].
DC232B, DC233C (LE) and fsf (BE) cores are supported on sim and LX60/LX200/ML605/KC705 platforms.

== Getting and building QEMU ==
QEMU is developed using git. The main tree is located at git://git.qemu.org/qemu.git. Release tarballs and guest systems images are available at http://wiki.qemu.org/Download.

This will make a copy of master QEMU repository, configure, build and install QEMU binaries for little-endian and big-endian targets:
$ '''git clone git://git.qemu.org/qemu.git'''
$ '''mkdir qemu-xtensa ; cd qemu-xtensa'''
$ '''../qemu/configure --prefix=`pwd`/root --target-list=xtensa-softmmu,xtensaeb-softmmu'''
$ '''make install'''

== QEMU options ==
{| border="1"
|-
! Option !! Meaning !! Example
|-
| -M <machine-name>
| Emulate specific platform type. Use '?' to list all supported platform types.
| -M lx60
|-
| -cpu <core-name>
| Emulate specific core type. Use '?' to list all supported core types.
| -cpu dc232b
|-
| -m <memory size>
| Set emulated RAM size. M and G suffixes for mega- and gigabytes.
| -m 128M
|-
| -semihosting
| Enable SIMCALL instruction for the protection ring 0.
| -semihosting
|-
| -kernel <elf-image/uImage>
| Load elf-image or uImage and start emulation from its entry point. Note that image is loaded to virtual addresses it is linked for. For XTFPGA this option also chooses 'boot from SRAM' mode, mapping SRAM to the System ROM region at physical address 0xFE000000.
| -kernel xtensa-2.6.29-smp/arch/xtensa/boot/Image.elf
|-
| -append <kernel arguments>
| (Not for sim) Wrap kernel arguments into the bootparameters block and pass pointer to it to the elf-image in a2.
| -append 'console=ttyS0 ip=dhcp root=/dev/nfs rw'
|-
| -initrd <initrd-image>
| (Not for sim) Load initial RAM disk image and pass its location to kernel in the bootparameters block.
| -initrd buildroot-build/images/rootfs.cpio
|-
| -dtb <Device-Tree-Blob>
| (Not for sim) Load DTB image and pass its location to kernel in the bootparameters block.
| -dtb xtensa-build/arch/xtensa/boot/dts/kc705.dtb
|-
| -pflash <flash-image>
| (Not for sim) Mount flash-image as a system FLASH. Note that image size must match platform FLASH size (4MB for LX60, 16MB for LX200 and ML605, 128MB for KC705).
| -pflash u-boot.dc232b_xtav200.flash
|-
| -net nic,<nic-parameters>
| (Not for sim) Set network interface parameters.
| -net nic,vlan=0
|-
| -net user,<SLIRP parameters>
| (Not for sim) Use the user mode network stack. SLIRP parameters may be used to tune built-in DHCP and TFTP servers.
| -net user,vlan=0,tftp=.,hostname=dc232b-lx200
|-
|}

== sim platform ==

The sim platform is modelled after Tensilica ISS. It may run ELF images built for ISS, see e.g. [[Buildroot_Build_Instructions#Build_and_Run_a_Linux_Kernel_on_the_Instruction_Set_Simulator_(ISS)|Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)]].
There's only SIMCALL external interface available for the sim machine. To avoid console interference with QEMU monitor, monitor should be disabled with -monitor null or moved to network interface with e.g. -monitor tcp::1236,telnet,server,nowait.

$ '''qemu-system-xtensa -cpu dc232b -M sim -m 128M -semihosting -nographic -monitor null \'''
'''-kernel sim-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Since the QEMU doesn't manage terminal settings in semihosting mode, ^C and other terminal control codes interrupt QEMU instead of its guest. '''stty -echo -icanon -isig''' may be used to set up terminal to not echo input characters and not terminate QEMU.

== XTFPGA (LX60/LX200/ML605/KC705/...) platforms ==

XTFPGA platform emulates system RAM, board-specific FPGA, UART, OpenCores 10/100 Mbit Ethernet MAC, National Semiconductors DP83848C 10/100 PHY and parallel FLASH.
The platform may boot from either SRAM (when -kernel option is specified, SRAM is mapped to the System ROM region and ELF image is loaded) or from FLASH.

NFS root export record for user network (there's no way to specify root-path option for the QEMU builtin DHCP server. Unless kernel parameter 'nfsroot' is given, you'll have to put the rootfs to /tftpboot/<hostname>):
/tftpboot/ 127.0.0.1(rw,no_root_squash,insecure)

Booting linux image from SRAM (FLASH not mapped):
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel lx60-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Booting u-boot image from SRAM (empty FLASH image is created and mapped):
$ '''head -c 16M /dev/zero > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200'''
In this scenario binary u-boot image may be downloaded from TFTP and copied to the FLASH as described [[U-Boot|here]].

Alternatively FLASH image containing u-boot may be preformatted and QEMU may be booted from it:
$ '''cat u-boot-xtensa.git/u-boot.bin.dc232b_xtav200 /dev/zero | head -c 16M > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash'''

(Note that KC705 expects bootloader to be at the offset 0x06000000 in the FLASH, unlike the older boards that boot at offset 0).

== Debugging with QEMU ==
=== GDB ===
There's builtin GDB server in the QEMU. -s command line switch activates it on the address 0.0.0.0:1234 (-gdb tcp:<ip>:<port> may be used instead of -s to change address and port to listen), -S switch stops QEMU at the first guest instruction waiting for 'continue' command from gdb.

$ '''qemu-system-xtensa -M lx200 -nographic -pflash lx200.flash \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200 -s -S'''

By default xtensa gdb may only access unprivileged core registers. To allow access to all available registers ./gdb/xtensa-config.c needs to be modified, marking privileged registers as non-privileged. If [https://github.com/jcmvbkbc/xtensa-toolchain-build these scripts] are used to build the toolchain, the fixup-gdb.sh step does that.

GDB breakpoints, watchpoints, single-stepping and other debugging features work as usual.

$ '''xtensa-dc232b-elf-gdb'''
GNU gdb (GDB) 7.3.50.20110709-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=xtensa-dc232b-elf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) '''target remote 127.0.0.1:1234'''
Remote debugging using 127.0.0.1:1234
0xfe000000 in ?? ()
(gdb) '''symbol-file u-boot-xtensa.git/u-boot.dc232b_xtav200'''
Reading symbols from u-boot-xtensa.git/u-boot.dc232b_xtav200...done.
(gdb) '''x/10i $pc'''
=> 0xfe000000 <_ResetVector>: j 0xfe00000c <_ResetVector+12>
0xfe000003 <_ResetVector+3>: break 0, 0
0xfe000006 <_ResetVector+6>: excw
0xfe000009 <_ResetVector+9>: .byte 0xff
0xfe00000a <_ResetVector+10>: .byte 0xff
0xfe00000b <_ResetVector+11>: .byte 0xff
0xfe00000c <_ResetVector+12>: l32r a2, 0xfe000004 <_ResetVector+4>
0xfe00000f <_ResetVector+15>: jx a2
0xfe000012 <_ResetVector+18>: ill
0xfe000015 <_ResetVector+21>: subx2 a0, a0, a0
(gdb) '''si'''
69 1: l32r a2, 2b
(gdb)

QEMU monitor commands may be invoked via gdb monitor command:

(gdb) '''monitor info mtree'''
memory
00000000-fffffffe (prio 0): system
00000000-07ffffff (prio 0): lx60.dram
f0000000-fdffffff (prio 0): lx60.io
fd020000-fd02ffff (prio 0): lx60.fpga
fd030000-fd030053 (prio 0): open_eth.regs
fd030400-fd0307ff (prio 0): open_eth.desc
fd050020-fd05003f (prio 0): serial
fd800000-fd803fff (prio 0): open_eth.ram
f8000000-f83fffff (prio 0): lx60.io.flash
fe000000-fe01ffff (prio 0): lx60.sram

=== Logging ===

QEMU can log guest CPU state/opcode translation during execution. Invalid, reserved and not implemented opcodes, invalid conditions for certain opcodes, reserved/not implemented SR access, page table lookups, interrupt requests are all logged with this method. Logging may be enabled with -d<logging types> command line option, or with monitor command 'log <logging types>'. By default logs go to the file /tmp/qemu.log. Verbose logging slows down execution significantly, it may be useful to control logging from GDB, enabling it via monitor command for the specific code blocks.

{| border="1"
|-
! Logging type !! Meaning
|-
| none
| disable logging
|-
| out_asm
| show generated host assembly code for each compiled TB
|-
| in_asm
| show target assembly code for each compiled TB (not implemented for Xtensa)
|-
| op
| show micro ops for each compiled TB
|-
| op_opt
| show micro ops after liveness analysis
|-
| int
| show interrupts/exceptions in short format
|-
| exec
| show trace before each executed TB
|-
| cpu
| show CPU state before block translation
|-
|}

== Extending QEMU ==

Starting with QEMU-2.2 there's a script [https://github.com/OSLL/qemu-xtensa/blob/master/target/xtensa/import_core.sh qemu/target-xtensa/import_core.sh] that adds new core to QEMU from the xtensa configuration overlay.

Setting up U-Boot

2018-03-02T23:31:27Z

Jcmvbkbc: Blanked the page

U-Boot

2018-03-02T23:30:13Z

Jcmvbkbc:

Under Construction - This page will describe the steps required to support a Xtensa processor configuration in U-Boot.

== U-Boot Installation to XTFPGA boards (LX60, LX200, ML605, KC705) ==

=== Loading initial image through xocd + xt-gdb ===

$ '''xt-gdb -n u-boot'''
(xt-gdb) '''target remote localhost:20000 0'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''continue'''

=== Flashing U-Boot image ===

U-Boot> '''protect off ...'''
U-Boot> '''erase ...'''
U-Boot> '''tftp u-boot.bin'''
U-Boot> '''cp.b 2000000 F8000000 20000'''

Linux Kernel

2018-03-02T23:09:54Z

Jcmvbkbc: fix U-Boot page link

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[U-Boot|U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Instructions for building and booting Linux

2018-03-02T23:09:01Z

Jcmvbkbc: fix U-Boot page link

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa cores supported by Linux ==

Official requirements for Xtensa core to run Linux are listed in the Xtensa OSKit Guide document.
The absolute minimum set of options required to run Linux is full MMU, debug option, windowed registers, one level-1 or medium level timer interrupt.
* See [[uClinux]] for experimental support of Xtensa cores without full MMU.
* Thread pointer option and conditional store option are required for running userspace with NPTL or TLS.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Xtensa on QEMU

2018-03-02T23:07:20Z

Jcmvbkbc: fix u-boot page link and 'extending qemu' section

== Setting up and running Xtensa simulation on QEMU ==

[http://wiki.qemu.org QEMU] is a free processor/machine emulator and virtualizer.

Starting with version 1.0 QEMU offers [http://wiki.qemu.org/Features/Xtensa Xtensa architecture emulation].
DC232B, DC233C (LE) and fsf (BE) cores are supported on sim and LX60/LX200/ML605/KC705 platforms.

== Getting and building QEMU ==
QEMU is developed using git. The main tree is located at git://git.qemu.org/qemu.git. Release tarballs and guest systems images are available at http://wiki.qemu.org/Download.

This will make a copy of master QEMU repository, configure, build and install QEMU binaries for little-endian and big-endian targets:
$ '''git clone git://git.qemu.org/qemu.git'''
$ '''mkdir qemu-xtensa ; cd qemu-xtensa'''
$ '''../qemu/configure --prefix=`pwd`/root --target-list=xtensa-softmmu,xtensaeb-softmmu'''
$ '''make install'''

== QEMU options ==
{| border="1"
|-
! Option !! Meaning !! Example
|-
| -M <machine-name>
| Emulate specific platform type. Use '?' to list all supported platform types.
| -M lx60
|-
| -cpu <core-name>
| Emulate specific core type. Use '?' to list all supported core types.
| -cpu dc232b
|-
| -m <memory size>
| Set emulated RAM size. M and G suffixes for mega- and gigabytes.
| -m 128M
|-
| -semihosting
| Enable SIMCALL instruction for the protection ring 0.
| -semihosting
|-
| -kernel <elf-image/uImage>
| Load elf-image or uImage and start emulation from its entry point. Note that image is loaded to virtual addresses it is linked for. For XTFPGA this option also chooses 'boot from SRAM' mode, mapping SRAM to the System ROM region at physical address 0xFE000000.
| -kernel xtensa-2.6.29-smp/arch/xtensa/boot/Image.elf
|-
| -append <kernel arguments>
| (Not for sim) Wrap kernel arguments into the bootparameters block and pass pointer to it to the elf-image in a2.
| -append 'console=ttyS0 ip=dhcp root=/dev/nfs rw'
|-
| -initrd <initrd-image>
| (Not for sim) Load initial RAM disk image and pass its location to kernel in the bootparameters block.
| -initrd buildroot-build/images/rootfs.cpio
|-
| -dtb <Device-Tree-Blob>
| (Not for sim) Load DTB image and pass its location to kernel in the bootparameters block.
| -dtb xtensa-build/arch/xtensa/boot/dts/kc705.dtb
|-
| -pflash <flash-image>
| (Not for sim) Mount flash-image as a system FLASH. Note that image size must match platform FLASH size (4MB for LX60, 16MB for LX200 and ML605, 128MB for KC705).
| -pflash u-boot.dc232b_xtav200.flash
|-
| -net nic,<nic-parameters>
| (Not for sim) Set network interface parameters.
| -net nic,vlan=0
|-
| -net user,<SLIRP parameters>
| (Not for sim) Use the user mode network stack. SLIRP parameters may be used to tune built-in DHCP and TFTP servers.
| -net user,vlan=0,tftp=.,hostname=dc232b-lx200
|-
|}

== sim platform ==

The sim platform is modelled after Tensilica ISS. It may run ELF images built for ISS, see e.g. [[Buildroot_Build_Instructions#Build_and_Run_a_Linux_Kernel_on_the_Instruction_Set_Simulator_(ISS)|Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)]].
There's only SIMCALL external interface available for the sim machine. To avoid console interference with QEMU monitor, monitor should be disabled with -monitor null or moved to network interface with e.g. -monitor tcp::1236,telnet,server,nowait.

$ '''qemu-system-xtensa -cpu dc232b -M sim -m 128M -semihosting -nographic -monitor null \'''
'''-kernel sim-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Since the QEMU doesn't manage terminal settings in semihosting mode, ^C and other terminal control codes interrupt QEMU instead of its guest. '''stty -echo -icanon -isig''' may be used to set up terminal to not echo input characters and not terminate QEMU.

== XTFPGA (LX60/LX200/ML605/KC705/...) platforms ==

XTFPGA platform emulates system RAM, board-specific FPGA, UART, OpenCores 10/100 Mbit Ethernet MAC, National Semiconductors DP83848C 10/100 PHY and parallel FLASH.
The platform may boot from either SRAM (when -kernel option is specified, SRAM is mapped to the System ROM region and ELF image is loaded) or from FLASH.

NFS root export record for user network (there's no way to specify root-path option for the QEMU builtin DHCP server. Unless kernel parameter 'nfsroot' is given, you'll have to put the rootfs to /tftpboot/<hostname>):
/tftpboot/ 127.0.0.1(rw,no_root_squash,insecure)

Booting linux image from SRAM (FLASH not mapped):
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel lx60-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Booting u-boot image from SRAM (empty FLASH image is created and mapped):
$ '''head -c 16M /dev/zero > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200'''
In this scenario binary u-boot image may be downloaded from TFTP and copied to the FLASH as described [[U-Boot|here]].

Alternatively FLASH image containing u-boot may be preformatted and QEMU may be booted from it:
$ '''cat u-boot-xtensa.git/u-boot.bin.dc232b_xtav200 /dev/zero | head -c 16M > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash'''

(Note that KC705 expects bootloader to be at the offset 0x06000000 in the FLASH, unlike the older boards that boot at offset 0).

== Debugging with QEMU ==
=== GDB ===
There's builtin GDB server in the QEMU. -s command line switch activates it on the address 0.0.0.0:1234 (-gdb tcp:<ip>:<port> may be used instead of -s to change address and port to listen), -S switch stops QEMU at the first guest instruction waiting for 'continue' command from gdb.

$ '''qemu-system-xtensa -M lx200 -nographic -pflash lx200.flash \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200 -s -S'''

By default xtensa gdb may only access unprivileged core registers. To allow access to all available registers ./gdb/xtensa-config.c needs to be modified, marking privileged registers as non-privileged. If [https://github.com/jcmvbkbc/xtensa-toolchain-build these scripts] are used to build the toolchain, the fixup-gdb.sh step does that.

GDB breakpoints, watchpoints, single-stepping and other debugging features work as usual.

$ '''xtensa-dc232b-elf-gdb'''
GNU gdb (GDB) 7.3.50.20110709-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=xtensa-dc232b-elf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) '''target remote 127.0.0.1:1234'''
Remote debugging using 127.0.0.1:1234
0xfe000000 in ?? ()
(gdb) '''symbol-file u-boot-xtensa.git/u-boot.dc232b_xtav200'''
Reading symbols from u-boot-xtensa.git/u-boot.dc232b_xtav200...done.
(gdb) '''x/10i $pc'''
=> 0xfe000000 <_ResetVector>: j 0xfe00000c <_ResetVector+12>
0xfe000003 <_ResetVector+3>: break 0, 0
0xfe000006 <_ResetVector+6>: excw
0xfe000009 <_ResetVector+9>: .byte 0xff
0xfe00000a <_ResetVector+10>: .byte 0xff
0xfe00000b <_ResetVector+11>: .byte 0xff
0xfe00000c <_ResetVector+12>: l32r a2, 0xfe000004 <_ResetVector+4>
0xfe00000f <_ResetVector+15>: jx a2
0xfe000012 <_ResetVector+18>: ill
0xfe000015 <_ResetVector+21>: subx2 a0, a0, a0
(gdb) '''si'''
69 1: l32r a2, 2b
(gdb)

QEMU monitor commands may be invoked via gdb monitor command:

(gdb) '''monitor info mtree'''
memory
00000000-fffffffe (prio 0): system
00000000-07ffffff (prio 0): lx60.dram
f0000000-fdffffff (prio 0): lx60.io
fd020000-fd02ffff (prio 0): lx60.fpga
fd030000-fd030053 (prio 0): open_eth.regs
fd030400-fd0307ff (prio 0): open_eth.desc
fd050020-fd05003f (prio 0): serial
fd800000-fd803fff (prio 0): open_eth.ram
f8000000-f83fffff (prio 0): lx60.io.flash
fe000000-fe01ffff (prio 0): lx60.sram

=== Logging ===

QEMU can log guest CPU state/opcode translation during execution. Invalid, reserved and not implemented opcodes, invalid conditions for certain opcodes, reserved/not implemented SR access, page table lookups, interrupt requests are all logged with this method. Logging may be enabled with -d<logging types> command line option, or with monitor command 'log <logging types>'. By default logs go to the file /tmp/qemu.log. Verbose logging slows down execution significantly, it may be useful to control logging from GDB, enabling it via monitor command for the specific code blocks.

{| border="1"
|-
! Logging type !! Meaning
|-
| none
| disable logging
|-
| out_asm
| show generated host assembly code for each compiled TB
|-
| in_asm
| show target assembly code for each compiled TB (not implemented for Xtensa)
|-
| op
| show micro ops for each compiled TB
|-
| op_opt
| show micro ops after liveness analysis
|-
| int
| show interrupts/exceptions in short format
|-
| exec
| show trace before each executed TB
|-
| cpu
| show CPU state before block translation
|-
|}

== Extending QEMU ==

Starting with QEMU-2.2 there's a script [https://github.com/OSLL/qemu-xtensa/blob/master/target-xtensa/import_core.sh qemu/target-xtensa/import_core.sh] that adds new core to QEMU from the xtensa configuration overlay.

Setting up U-Boot

2018-03-02T18:58:04Z

Jcmvbkbc: Replaced content with "== LX60, LX200, ML605, KC705 U-Boot Installation == === Loading initial image through xocd + xt-gdb === $ '''xt-gdb -n u-boot''' (xt-gdb) '''target remote localhos..."

== LX60, LX200, ML605, KC705 U-Boot Installation ==

=== Loading initial image through xocd + xt-gdb ===

$ '''xt-gdb -n u-boot'''
(xt-gdb) '''target remote localhost:20000 0'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''continue'''

=== Flashing U-Boot image ===

U-Boot> '''protect off ...'''
U-Boot> '''erase ...'''
U-Boot> '''tftp u-boot.bin'''
U-Boot> '''cp.b 2000000 F8000000 20000'''

Buildroot

2017-07-28T21:51:38Z

Jcmvbkbc: add latest features and document changed overlay location specifier

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain, a root filesystem, linux kernel and the U-Boot bootloader.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>
* for the buildroot with uClinux support see [[uClinux]]

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** release 2017.05 and older:
*** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
*** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
** release 2017.08 and newer:
*** BR2_XTENSA_OVERLAY_FILE: path to the overlay file. May be relative or absolute, may also be an URL.
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2017.08 ==
* overlay overhaul: now overlay may be compressed, may have arbitrary name and may be specified as an URL
* overlay is applied to linux
* overlay is applied to U-Boot

== 2017.05 ==
* build xtensa linux kernel uImage

== 2016.05 ==
* support noMMU xtensa cores

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Buildroot

2017-07-27T00:40:00Z

Jcmvbkbc: We now can build linux and U-Boot as a part of buildroot

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain, a root filesystem, linux kernel and the U-Boot bootloader.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>
* for the buildroot with uClinux support see [[uClinux]]

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Toolchain Overlay File

2017-07-26T21:23:04Z

Jcmvbkbc: Update conversion script

Linux Kernel

2016-10-11T00:37:42Z

Jcmvbkbc:

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable KSEG physical memory address and kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Linux Kernel

2016-10-11T00:05:19Z

Jcmvbkbc: update kernel features for 4.9

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.9 ==
* New kernel memory layouts: with 256MBytes and 512MBytes of low memory.
* Configurable kernel load address.
* Custom sysmem boot time memory allocator is replaced with memblock.
* KMEMLEAK.
* reserved-memory DT node.
* Cadence Configurable System Platform (CSP).

== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Boot Loader

2016-09-16T01:11:18Z

Jcmvbkbc: update status of U-Boot port

== Boot Loaders ==

The [http://en.wikipedia.org/wiki/Bootloader bootstrap loader or boot loader] is a small program running on a computer or device after power-on to load the operating system. Most boot loaders today provide additional features that allow to upgrade the firmware and access the system for troubleshooting.

== U-Boot ([[U-Boot|build instructions for Xtensa]]) ==

U-Boot is one of the most popular boot loaders for Linux. It has a very complete [http://www.denx.de/wiki/U-Boot Project Page] as well as an nice introduction available [http://www.linuxfordevices.com/c/a/Linux-For-Devices-Articles/Introduction-to-Das-UBoot-the-universal-open-source-bootloader/ here]. A very complete U-Boot manual is also available [http://www.denx.de/wiki/DULG/Manual here].

As of release v2016.09 xtensa support is available in the U-Boot mainline.

== RedBoot ==

Tensilica has also ported [http://sourceware.org/redboot/ RedBoot] for the Xtensa architecture. A RedBoot binary targeting the XTAV60 (LX60) or XTAV200 (LX200) emulation board is usually provided after generating an FPGA bitstream for one of these boards. See the relevant board documentation from Tensilica for more details such as usage and supported features. For more information and to request sources, please contact [http://www.tensilica.com/ Tensilica] directly. For an example of how to boot the Linux kernel using RedBoot on the LX60 or LX200 boards, see [[RedBoot Example]].

A source tarball for RedBoot with support for the Xtensa architecture is available [http://linux-xtensa.org/pub/redboot/redboot1.3.1-RB2007.2.tgz here]. For more details, see the tarball's [http://linux-xtensa.org/pub/redboot/redboot1.3.1-RB2007.2/README.txt README.txt] file.

== Host-Side Loading via OCD ==

An alternative to boot loaders is an external (e.g. ''"host-side"'') loading mechanism.

One common way to load a Linux kernel is to use debugging interfaces such as On-Chip Debug (OCD). For example, Tensilica Tools GDB (xt-gdb) can connect to a target Xtensa processor using OCD via the Xtensa OCD Daemon. It can then load and execute a Linux RAM image (Image.elf or vmlinux) onto the target using standard GDB commands. See [[Buildroot_Build_Instructions#Download_and_run_the_kernel|Download and Run the Kernel]] for an example.

== Linux Kernel Self-Boot Image ==

It is possible to adapt the kernel build scripts for the Xtensa architecture to create a self-booting image. That is, a Linux kernel image that includes a Reset handler and can be either burned in ROM or Flash or downloaded to the target using another mechanism such as OCD. '''''Note: This has not yet been implemented.'''''

Toolchain and Embedded Distributions

2016-09-07T07:56:15Z

Jcmvbkbc: update status of xtensa support in crosstool-NG

== [[Toolchain Overlay File]] ==

Because Xtensa is a configurable and extensible processor architecture that allows to define additional registers and instructions, the toolchain is the main software component that usually needs to be adapted for a specific Xtensa processor configuration. Tensilica provides a small set of configuration files in form of an ''overlay'' that replace the versions in the toolchain sources. The [[Toolchain Overlay File]] is a version of that ''overlay'' provided by Tensilica with a specific format required for crosstool-NG and buildroot. A script to convert the ''overlay'' provided by Tensilica can be found under [[Toolchain Overlay File]]

== Start from scratch with FSF/OSS sources ==

Creating a toolchain and bootable root file system starting from sources is certainly the most challenging approach. It usually requires additional patches and sometimes matching versions of sources. While support for Xtensa is integrated in most toolchain tools, they must be patches for a specific Xtensa processor configuration. The ''overlay'' metioned above includes a necessary set of configuration files together with information explaining the process to patch the toolchain sources.

'''Warning:''' cross compilers running on 64bit hosts are known to produce broken code. For some more details see the [http://lists.linux-xtensa.org/pipermail/linux-xtensa/Week-of-Mon-20130715/001140.html explanation] of Chris Zankel.

== [http://crosstool-ng.org/ crosstool-NG] ([[crosstool-NG|build instructions for Xtensa]]) ==

crosstool-NG (http://crosstool-ng.org/) provides a system to build a complete toolchain. It supports various C-libraries (Multilib, uClibc, glibc, and eglibc), but can also be used to build a 'bare' toolchain. Starting with release 1.22.0 the mainline crosstool-NG repository includes support for Xtensa. The process of patching the toolchain and other architecture specific software is fully automated using the [[Toolchain Overlay File]]. More information for building a toolchain for Xtensa using crosstool-NG can be found [[crosstool-NG|here]].

== [http://www.buildroot.org BuildRoot] ([[Buildroot|build instructions for Xtensa]]) ==

Buildroot is a set of Makefiles and patches that makes it easy to generate a cross-compilation toolchain and root file system for your target Linux system. The Xtensa architecture is fully supported in the official tree, including support for specific processor configuration using the [[Toolchain Overlay File]]. For build instructions, see [[Buildroot]].

== [http://www.openadk.org OpenADK] ==

OpenADK is a set of Makefiles and patches that makes it easy to generate a cross-compilation toolchain and root file system for your target Linux system. The Xtensa architecture is fully supported, including support for specific processor configuration using the [[Toolchain Overlay File]]. At the moment only Qemu emulation (dc232b,dc233c) targets are available. New configuration can be simply added.

Linux Kernel

2016-03-27T01:18:37Z

Jcmvbkbc: Update Features by release for 4.4 and 4.6

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =
== Linux 4.6 ==
* Hardware breakpoints/watchpoints.
* ioremap for memory outside KIO region.
* Big endian: serial, network, i2c work.

== Linux 4.4 ==
* DMA to high memory.

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Toolchain Overlay File

2015-12-18T15:43:55Z

Jcmvbkbc: Patch gdb/xtensa-config.c file during overlay repacking. See https://sourceware.org/ml/gdb-patches/2015-04/msg00698.html for the reference

== Introduction ==

The ''Overlay'' is a tar file that includes modified versions of processor configuration dependent files that are required by various toolchain tools, such as binutils, gcc, and gdb. The overlay described here has a specific layout and is intended for toolchain generators, such as buildroot, OpenADK or crosstool-NG.

== Creating the overlay ==

The overlay file shipped by Tensilica uses a flat directory structure, which is not suitable for buildroot or crosstool-NG at the moment. The file is part of the generated processor package, and can be found in the XtDevTools directory:

XtDevTools/install/build/$VERSION/$XTENSA_CORE/src/xtensa-config-overlay.tar.gz

The following script converts this file and writes the modified version to xtensa-overlay.tar. Simply provide the path and name of the overlay provided by Tensilica when running the script:

./script xtensa-config-overlay.tar.gz

The file xtensa-overlay.tar created by the script can now be used to build buildroot or crosstool-NG. For more information how to configure those generators, please see [[Buildroot]] or [[crosstool-NG]].

(Note that there is also a similar overlay file in the generated processor package provided by Tensilica, but it has a different directory layout and is not suitable for the script provided below. It can be found under .../$XTENSA_CORE/xtensa-elf/src/linux/misc/linux-overlay.tar.gz)

== Conversion Script ==

#!/bin/sh
if [ $# -ne 1 ]; then
echo "Error: No overlay file specified"
exit 1
fi
if [ $1 == "xtensa-overlay.tar" ]; then
echo "Error: Input file must be named other than 'xtensa-overlay.tar'"
exit 1
fi
if [ ! -f $1 ]; then
echo "Error: Overlay file $1 not found"
exit 1
fi
TMPDIR=`mktemp -d`
tar xzf $1 -C $TMPDIR
mkdir -p $TMPDIR/overlay/binutils/bfd $TMPDIR/overlay/binutils/ld \
$TMPDIR/overlay/binutils/include $TMPDIR/overlay/gcc/include \
$TMPDIR/overlay/gdb/bfd $TMPDIR/overlay/gdb/gdb/regformats \
$TMPDIR/overlay/gdb/gdb/gdbserver $TMPDIR/overlay/gdb/include
mv $TMPDIR/binutils/xtensa-config.h $TMPDIR/overlay/binutils/include
mv $TMPDIR/binutils/xtensa-modules.c $TMPDIR/overlay/binutils/bfd
mv $TMPDIR/gcc/xtensa-config.h $TMPDIR/overlay/gcc/include
sed -e '0,/#include/s/#include/#include "defs.h"\n#include/' -i $TMPDIR/gdb/xtensa-config.c
mv $TMPDIR/gdb/xtensa-config.c $TMPDIR/overlay/gdb/gdb
mv $TMPDIR/gdb/xtensa-config.h $TMPDIR/overlay/gdb/include
mv $TMPDIR/gdb/xtensa-modules.c $TMPDIR/overlay/gdb/bfd
mv $TMPDIR/gdb/xtensa-xtregs.c $TMPDIR/overlay/gdb/gdb/gdbserver
mv $TMPDIR/gdb/reg-xtensa.dat $TMPDIR/overlay/gdb/gdb/regformats
tar cf xtensa-overlay.tar -C $TMPDIR/overlay/ binutils gcc gdb
rm -fr $TMPDIR

== Example Files ==

The following files are the original and generated overlay file for the DC_D_233L processor.

* http://www.linux-xtensa.org/pub/xtensa-config-overlay.tar.gz - Original overlay file provided by Tensilica
* http://www.linux-xtensa.org/pub/xtensa-overlay.tgz - : Generated overlay file, applicable to buildroot and crosstool-NG

== Ready to go overlay archives (can be used with buildroot and OpenADK) ==

https://github.com/jcmvbkbc/xtensa-toolchain-build/tree/master/overlays

Xtensa on QEMU

2015-11-26T13:10:30Z

Jcmvbkbc: specify better stty options for sim

== Setting up and running Xtensa simulation on QEMU ==

[http://wiki.qemu.org QEMU] is a free processor/machine emulator and virtualizer.

Starting with version 1.0 QEMU offers [http://wiki.qemu.org/Features/Xtensa Xtensa architecture emulation].
DC232B, DC233C (LE) and fsf (BE) cores are supported on sim and LX60/LX200/ML605/KC705 platforms.

== Getting and building QEMU ==
QEMU is developed using git. The main tree is located at git://git.qemu.org/qemu.git. Release tarballs and guest systems images are available at http://wiki.qemu.org/Download.

This will make a copy of master QEMU repository, configure, build and install QEMU binaries for little-endian and big-endian targets:
$ '''git clone git://git.qemu.org/qemu.git'''
$ '''mkdir qemu-xtensa ; cd qemu-xtensa'''
$ '''../qemu/configure --prefix=`pwd`/root --target-list=xtensa-softmmu,xtensaeb-softmmu'''
$ '''make install'''

== QEMU options ==
{| border="1"
|-
! Option !! Meaning !! Example
|-
| -M <machine-name>
| Emulate specific platform type. Use '?' to list all supported platform types.
| -M lx60
|-
| -cpu <core-name>
| Emulate specific core type. Use '?' to list all supported core types.
| -cpu dc232b
|-
| -m <memory size>
| Set emulated RAM size. M and G suffixes for mega- and gigabytes.
| -m 128M
|-
| -semihosting
| Enable SIMCALL instruction for the protection ring 0.
| -semihosting
|-
| -kernel <elf-image/uImage>
| Load elf-image or uImage and start emulation from its entry point. Note that image is loaded to virtual addresses it is linked for. For XTFPGA this option also chooses 'boot from SRAM' mode, mapping SRAM to the System ROM region at physical address 0xFE000000.
| -kernel xtensa-2.6.29-smp/arch/xtensa/boot/Image.elf
|-
| -append <kernel arguments>
| (Not for sim) Wrap kernel arguments into the bootparameters block and pass pointer to it to the elf-image in a2.
| -append 'console=ttyS0 ip=dhcp root=/dev/nfs rw'
|-
| -initrd <initrd-image>
| (Not for sim) Load initial RAM disk image and pass its location to kernel in the bootparameters block.
| -initrd buildroot-build/images/rootfs.cpio
|-
| -dtb <Device-Tree-Blob>
| (Not for sim) Load DTB image and pass its location to kernel in the bootparameters block.
| -dtb xtensa-build/arch/xtensa/boot/dts/kc705.dtb
|-
| -pflash <flash-image>
| (Not for sim) Mount flash-image as a system FLASH. Note that image size must match platform FLASH size (4MB for LX60, 16MB for LX200 and ML605, 128MB for KC705).
| -pflash u-boot.dc232b_xtav200.flash
|-
| -net nic,<nic-parameters>
| (Not for sim) Set network interface parameters.
| -net nic,vlan=0
|-
| -net user,<SLIRP parameters>
| (Not for sim) Use the user mode network stack. SLIRP parameters may be used to tune built-in DHCP and TFTP servers.
| -net user,vlan=0,tftp=.,hostname=dc232b-lx200
|-
|}

== sim platform ==

The sim platform is modelled after Tensilica ISS. It may run ELF images built for ISS, see e.g. [[Buildroot_Build_Instructions#Build_and_Run_a_Linux_Kernel_on_the_Instruction_Set_Simulator_(ISS)|Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)]].
There's only SIMCALL external interface available for the sim machine. To avoid console interference with QEMU monitor, monitor should be disabled with -monitor null or moved to network interface with e.g. -monitor tcp::1236,telnet,server,nowait.

$ '''qemu-system-xtensa -cpu dc232b -M sim -m 128M -semihosting -nographic -monitor null \'''
'''-kernel sim-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Since the QEMU doesn't manage terminal settings in semihosting mode, ^C and other terminal control codes interrupt QEMU instead of its guest. '''stty -echo -icanon -isig''' may be used to set up terminal to not echo input characters and not terminate QEMU.

== XTFPGA (LX60/LX200/ML605/KC705/...) platforms ==

XTFPGA platform emulates system RAM, board-specific FPGA, UART, OpenCores 10/100 Mbit Ethernet MAC, National Semiconductors DP83848C 10/100 PHY and parallel FLASH.
The platform may boot from either SRAM (when -kernel option is specified, SRAM is mapped to the System ROM region and ELF image is loaded) or from FLASH.

NFS root export record for user network (there's no way to specify root-path option for the QEMU builtin DHCP server. Unless kernel parameter 'nfsroot' is given, you'll have to put the rootfs to /tftpboot/<hostname>):
/tftpboot/ 127.0.0.1(rw,no_root_squash,insecure)

Booting linux image from SRAM (FLASH not mapped):
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel lx60-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Booting u-boot image from SRAM (empty FLASH image is created and mapped):
$ '''head -c 16M /dev/zero > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200'''
In this scenario binary u-boot image may be downloaded from TFTP and copied to the FLASH as described [[Setting_up_U-Boot|here]].

Alternatively FLASH image containing u-boot may be preformatted and QEMU may be booted from it:
$ '''cat u-boot-xtensa.git/u-boot.bin.dc232b_xtav200 /dev/zero | head -c 16M > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash'''

(Note that KC705 expects bootloader to be at the offset 0x06000000 in the FLASH, unlike the older boards that boot at offset 0).

== Debugging with QEMU ==
=== GDB ===
There's builtin GDB server in the QEMU. -s command line switch activates it on the address 0.0.0.0:1234 (-gdb tcp:<ip>:<port> may be used instead of -s to change address and port to listen), -S switch stops QEMU at the first guest instruction waiting for 'continue' command from gdb.

$ '''qemu-system-xtensa -M lx200 -nographic -pflash lx200.flash \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200 -s -S'''

By default xtensa gdb may only access unprivileged core registers. To allow access to all available registers ./gdb/xtensa-config.c needs to be modified, marking privileged registers as non-privileged. If [https://github.com/jcmvbkbc/xtensa-toolchain-build these scripts] are used to build the toolchain, the fixup-gdb.sh step does that.

GDB breakpoints, watchpoints, single-stepping and other debugging features work as usual.

$ '''xtensa-dc232b-elf-gdb'''
GNU gdb (GDB) 7.3.50.20110709-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=xtensa-dc232b-elf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) '''target remote 127.0.0.1:1234'''
Remote debugging using 127.0.0.1:1234
0xfe000000 in ?? ()
(gdb) '''symbol-file u-boot-xtensa.git/u-boot.dc232b_xtav200'''
Reading symbols from u-boot-xtensa.git/u-boot.dc232b_xtav200...done.
(gdb) '''x/10i $pc'''
=> 0xfe000000 <_ResetVector>: j 0xfe00000c <_ResetVector+12>
0xfe000003 <_ResetVector+3>: break 0, 0
0xfe000006 <_ResetVector+6>: excw
0xfe000009 <_ResetVector+9>: .byte 0xff
0xfe00000a <_ResetVector+10>: .byte 0xff
0xfe00000b <_ResetVector+11>: .byte 0xff
0xfe00000c <_ResetVector+12>: l32r a2, 0xfe000004 <_ResetVector+4>
0xfe00000f <_ResetVector+15>: jx a2
0xfe000012 <_ResetVector+18>: ill
0xfe000015 <_ResetVector+21>: subx2 a0, a0, a0
(gdb) '''si'''
69 1: l32r a2, 2b
(gdb)

QEMU monitor commands may be invoked via gdb monitor command:

(gdb) '''monitor info mtree'''
memory
00000000-fffffffe (prio 0): system
00000000-07ffffff (prio 0): lx60.dram
f0000000-fdffffff (prio 0): lx60.io
fd020000-fd02ffff (prio 0): lx60.fpga
fd030000-fd030053 (prio 0): open_eth.regs
fd030400-fd0307ff (prio 0): open_eth.desc
fd050020-fd05003f (prio 0): serial
fd800000-fd803fff (prio 0): open_eth.ram
f8000000-f83fffff (prio 0): lx60.io.flash
fe000000-fe01ffff (prio 0): lx60.sram

=== Logging ===

QEMU can log guest CPU state/opcode translation during execution. Invalid, reserved and not implemented opcodes, invalid conditions for certain opcodes, reserved/not implemented SR access, page table lookups, interrupt requests are all logged with this method. Logging may be enabled with -d<logging types> command line option, or with monitor command 'log <logging types>'. By default logs go to the file /tmp/qemu.log. Verbose logging slows down execution significantly, it may be useful to control logging from GDB, enabling it via monitor command for the specific code blocks.

{| border="1"
|-
! Logging type !! Meaning
|-
| none
| disable logging
|-
| out_asm
| show generated host assembly code for each compiled TB
|-
| in_asm
| show target assembly code for each compiled TB (not implemented for Xtensa)
|-
| op
| show micro ops for each compiled TB
|-
| op_opt
| show micro ops after liveness analysis
|-
| int
| show interrupts/exceptions in short format
|-
| exec
| show trace before each executed TB
|-
| cpu
| show CPU state before block translation
|-
|}

== Extending QEMU ==

Starting with QEMU-2.2 there's a script [https://github.com/OSLL/qemu-xtensa/blob/master/target-xtensa/import_core.sh qemu/target-xtensa/import_core.sh] that adds new core to QEMU from the xtensa configuration overlay.

To manually extend QEMU with new xtensa core an appropriate XtensaConfig structure must be registered. This structure may be mostly automatically filled in from the overlay data, see mainline commits [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=53add759be1a39063cfc52ca2a6d0fcdd1091697 53add759be1a39063cfc52ca2a6d0fcdd1091697] (target-xtensa: add dc232b core) and [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=935f7a2b428f71a40e985f09955bccaaf09518fa 935f7a2b428f71a40e985f09955bccaaf09518fa] (target-xtensa: add fsf core) for example:
* target-xtensa/core-<core name>/core-isa.h is a copy of config/core.h from xtensa-config-overlay.tar.gz;
* target-xtensa/core-<core name>/gdb-config.c is made from gdb/xtensa-config.c from xtensa-config-overlay.tar.gz, only rmap entries for PC, A*, AR*, SR*, UR* are retained;
* XtensaConfig::gdb_regmap::num_regs is the total number of rmap entries in the gdb-config.c;
* XtensaConfig::gdb_regmap::num_core_regs is the number of contiguous non-privileged ((flags & 1) == 0) rmap entries in the gdb-config.c.

Crosstool-NG

2015-11-19T03:04:00Z

Jcmvbkbc: update ct-ng status

== Overview ==

Crosstool-NG provides a system to build a complete toolchain. It supports various C-libraries (newlib, uClibc, glibc, and eglibc), but can also be used to build a toolchain without a C-library. The following paragraphs describe the steps to build a toolchain tailored for an Xtensa configuration.

== Configure and build Crosstool-NG ==

Starting with release 1.22.0 the mainline crosstool-NG repository includes support for Xtensa. Using tip of the cloned git tree for work is not always a good idea, unless you need the latest mainline changes using official releases (marked by tags) would give more predictable results.

git clone https://github.com/crosstool-ng/crosstool-ng.git
git checkout crosstool-ng-1.22.0

=== Build ct-ng ===

ct-ng is similar to the command 'make' for other projects, and is used to configure and build the toolchain. Since we cloned it from the repository, its build system is not configured yet, use ./bootstrap script to create it:

./bootstrap

The next step is to configure ct-ng for your system. The default installation path is /usr/local/ but can be overwritten with the --prefix option. It is also possible to configure ct-ng for use without installation with --enable-local configure option:

./configure --enable-local

Finally, run 'make' to build ct-ng (and 'make install' to install it if needed):

make

=== Xtensa core overlay file ===
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by the crosstool-NG. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of crosstool-NG requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

== Building a 'bare' toolchain without a C-library ==

=== Configure the toolchain ===

The first step is to configure the toolchain:

./ct-ng menuconfig

Make the following changes (---> means to enter the sub-menu). We are assuming that the overlay tar image (xtensa_<CORE_NAME>.tar) has been copied to ABSOLUTE_PATH_TO_OVERLAY.

Target options --->
Target Architecture (xtensa)
Target Architecture Variant (Custom Xtensa processor configuration)
(CORE_NAME) Custom Xtensa process configuration file name
(ABSOLUTE_PATH_TO_OVERLAY) Full path to custom Xtensa processor configurations

C-library --->
C library (none)

The default directory for the created toolchain is $HOME/x-tools, but can be configured under 'Paths and misc options':

Paths and misc options --->
(${HOME}/x-tools/${CT_TARGET}) Prefix directory

=== Build and install the toolchain ===

Finally, build and install the toolchain.

./ct-ng build

=== Test it ===

export PATH=$PATH:$HOME/x-tools/xtensa-unknown-linux-gnu/bin
xtensa-unknown-linux-gnu-gcc t.c

== Building uClibc-ng based toolchain ==

=== Configure the toolchain ===

Similar to the 'bare' version above, the first step is to configure crosstool-NG. Simply run

./ct-ng menuconfig

and make the following changes (---> means to enter the sub-menu).

Paths and misc options --->
[*] Try features marked as EXPERIMENTAL

Target options --->
Target Architecture (xtensa)
Target Architecture Variant (Custom Xtensa processor configuration)
(CORE_NAME) Custom Xtensa process configuration name
(ABSOLUTE_PATH_TO_OVERLAY) Full path to custom Xtensa processor configurations

Operating system --->
Target OS (linux)

C-library --->
C library (uClibc)

=== Build and install the toolchain ===

Finally, build and install the toolchain.

./ct-ng build

== Building a 'glibc' based toolchain ''(experimental)'' ==

NOTE THAT THE REPOSITORY IS CURRENTLY NOT AVAILABLE.

The glibc port is experimental at the moment and also not part of the main repository yet. A version for Xtensa can be found on linux-xtensa.org, so you can clone it from there:

git clone git://.../git/glibc

=== Configure the toolchain ===

Similar to the 'bare' version above, the first step is to configure crosstool-NG. Simply run

./ct-ng menuconfig

and make the following changes (---> means to enter the sub-menu).

Paths and misc options --->
[*] Try features marked as EXPERIMENTAL

Target options --->
Target Architecture (xtensa)
Target Architecture Variant (Custom Xtensa processor configuration)
(CORE_NAME) Custom Xtensa process configuration name
(ABSOLUTE_PATH_TO_OVERLAY) Full path to custom Xtensa processor configurations

C-library --->
C library (glibc)
glibc version (Custom glibc)
(ABSOLUTE_PATH_TO_glibc-2.17) Full path to custom glibc source
(ports) Extra addons

=== Build and install the toolchain ===

Finally, build and install the toolchain.

./ct-ng build

UClinux

2015-10-06T03:39:07Z

Jcmvbkbc: add clone commands

= Toolchain with uClinux support =

Xtensa uClinux userspace is based on bFLT executable file format.

* gcc patch that adds xtensa*-*-uclinux* is [https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=e3c15dbe1c4cd63d906eb67c1c607cc4fab6b329 committed] into gcc-6.0 development branch and will be first released with gcc-6.1.
* elf2flt patch that adds Xtensa support is [http://cgit.openadk.org/cgi/cgit/elf2flt.git/commit/?id=17a2cf8c28cf13b85939b2553b8a87b16d88c94e committed into this] elf2flt repository.

= Linux kernel with noMMU support =

Linux branch with noMMU support patches, sample core and sample configuration: https://github.com/jcmvbkbc/linux-xtensa/commits/xtensa-nommu

The following command may be used to clone that tree and switch to xtensa-nommu branch: <code>git clone -b xtensa-nommu git://github.com/jcmvbkbc/linux-xtensa.git</code>

The patches will likely appear in Linux v4.4

= Buildroot with noMMU support =

Buildroot branch with gcc and elf2flt patches, sample overlay and configuration: https://github.com/jcmvbkbc/buildroot/commits/xtensa-nommu

The following command may be used to clone that tree and switch to xtensa-nommu branch: <code>git clone -b xtensa-nommu git://github.com/jcmvbkbc/buildroot.git</code>

= uClinux distribution for Xtensa =

In progress

Buildroot

2015-10-06T03:34:35Z

Jcmvbkbc: add uclinux link

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain and a root filesystem. Building linux kernel and bootloader for arbitrary Xtensa core with the Buildroot is currently not supported.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>
* for the buildroot with uClinux support see [[uClinux]]

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Linux Kernel

2015-10-06T03:30:51Z

Jcmvbkbc: add uclinux link

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.
* for experimental uClinux code see [[uClinux]].

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

UClinux

2015-10-06T03:28:20Z

Jcmvbkbc: create page

= Experimental uClinux support =

== Toolchain with uClinux support ==

Xtensa uClinux userspace is based on bFLT executable file format.

* gcc patch that adds xtensa*-*-uclinux* is [https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=e3c15dbe1c4cd63d906eb67c1c607cc4fab6b329 committed] into gcc-6.0 development branch and will be first released with gcc-6.1.
* elf2flt patch that adds Xtensa support is [http://cgit.openadk.org/cgi/cgit/elf2flt.git/commit/?id=17a2cf8c28cf13b85939b2553b8a87b16d88c94e committed into this] elf2flt repository.

== Linux kernel with noMMU support ==

Linux branch with noMMU support patches, sample core and sample configuration: https://github.com/jcmvbkbc/linux-xtensa/commits/xtensa-nommu

The patches will likely appear in Linux v4.4

== Buildroot with noMMU support ==

Buildroot branch with gcc and elf2flt patches, sample overlay and configuration: https://github.com/jcmvbkbc/buildroot/commits/xtensa-nommu

== uClinux distribution for Xtensa ==

In progress

Instructions for building and booting Linux

2015-10-06T03:17:50Z

Jcmvbkbc: add 'cores supported by linux'

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa cores supported by Linux ==

Official requirements for Xtensa core to run Linux are listed in the Xtensa OSKit Guide document.
The absolute minimum set of options required to run Linux is full MMU, debug option, windowed registers, one level-1 or medium level timer interrupt.
* See [[uClinux]] for experimental support of Xtensa cores without full MMU.
* Thread pointer option and conditional store option are required for running userspace with NPTL or TLS.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[Setting_up_U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Status of the Xtensa Port

2015-10-06T02:00:13Z

Jcmvbkbc: add ltrace to debug section

This page provides additional information for software support for the Xtensa architecture. While most applications are written architecture independent, development tools and operating systems must be ported to a new processor architecture. Support for the Xtensa architecture has already been integrated in most Open Source software projects. Development trees that feed to the respective mainline repository and for new development can be found in local [http://git.linux-xtensa.org GIT] repositories.

=== Toolchain: Compiler (GCC) and various binary utilities (binutils) ===

All GNU toolchain tools are maintained in OpenSource and can be downloaded from the respective locations.
* Binutils: http://www.gnu.org/software/binutils/
* GCC: http://gcc.gnu.org/

Additional information to patch these sources for a specific Xtensa processor configuration can be found under [[Toolchain and Embedded Distributions]].

=== C-Libraries: Glibc, uClibc and uClibc-ng ===

Xtensa has long been part of uClibc, however, there are a lot of bug fixes pending for the next release (0.9.33.3). Support for the Native Posix Thread Library (NTPL) is under development and is expected to be part of the following release. The spin-off uClibc-ng project already contains the experimental NPTL support. Development for Glibc with NPTL (older thread libraries are not supported by Glibc anymore) is also current under way. A development tree can be found in this [https://github.com/czankel/xtensa-glibc GIT] repository.
* uClibc: http://www.uclibc.org/
* GlibC: http://www.gnu.org/software/libc/
* uClibc-ng: http://www.uclibc-ng.org/

=== Debugging ===

GDB is maintained in OpenSource and support for Xtensa in other debugging tools was maintained as patches in local buildroot repositories, and are currently updated and pushed to the respective main repositories.
* GDB: http://www.gnu.org/software/gdb/
* strace: http://sourceforge.net/projects/strace/
* ltrace: http://ltrace.alioth.debian.org/

=== Linux kernel ===

Xtensa has been part of the official Linux kernel tree since 2.6.13, however, a lot of patches were maintained in a local stable tree for a long time, and are currently ported to the latest kernel tree. Features, such as large memory support and cache aliasing should be expected to hit the kernel in the 3.10/3.11 time frame. The latest version can be found at https://www.kernel.org/.

=== Toolchain creation tools and embedded distributions (crosstool-NG, buildroot, etc) ===

Xtensa is supported by buildroot, which can be downloaded from http://www.buildroot.net. Support for crosstool-NG has not been pushed into the official tree yet. A version can be downloaded from local [http://git.linux-xtensa.org GIT] repositories. Additional information to configure these tools for a specific Xtensa processor configuration can be found under [[Toolchain and Embedded Distributions]].
* Buildroot: http://www.buildroot.net
* Crosstool-NG: http://crosstool-ng.org/
* OpenADK: http://www.openadk.org/

=== Other Projects ===

The following list are additional projects that are architecture specific, or that have architecture specific components with additional information how to compile them for Xtensa.

* [[LibFFI]] - LibFFI is a library that provides an interface to execute low-level functions. It is mostly used by scripting or higher-level languages, such as Python, Dalvik, etc.

Linux Kernel

2015-09-25T17:39:41Z

Jcmvbkbc: update linux features list

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =

== Linux 4.3 ==
* Hardware perf counters and perf support.

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Xtensa on QEMU

2015-07-20T21:39:33Z

Jcmvbkbc: add ML605/KC705 to the list of platforms

== Setting up and running Xtensa simulation on QEMU ==

[http://wiki.qemu.org QEMU] is a free processor/machine emulator and virtualizer.

Starting with version 1.0 QEMU offers [http://wiki.qemu.org/Features/Xtensa Xtensa architecture emulation].
DC232B, DC233C (LE) and fsf (BE) cores are supported on sim and LX60/LX200/ML605/KC705 platforms.

== Getting and building QEMU ==
QEMU is developed using git. The main tree is located at git://git.qemu.org/qemu.git. Release tarballs and guest systems images are available at http://wiki.qemu.org/Download.

This will make a copy of master QEMU repository, configure, build and install QEMU binaries for little-endian and big-endian targets:
$ '''git clone git://git.qemu.org/qemu.git'''
$ '''mkdir qemu-xtensa ; cd qemu-xtensa'''
$ '''../qemu/configure --prefix=`pwd`/root --target-list=xtensa-softmmu,xtensaeb-softmmu'''
$ '''make install'''

== QEMU options ==
{| border="1"
|-
! Option !! Meaning !! Example
|-
| -M <machine-name>
| Emulate specific platform type. Use '?' to list all supported platform types.
| -M lx60
|-
| -cpu <core-name>
| Emulate specific core type. Use '?' to list all supported core types.
| -cpu dc232b
|-
| -m <memory size>
| Set emulated RAM size. M and G suffixes for mega- and gigabytes.
| -m 128M
|-
| -semihosting
| Enable SIMCALL instruction for the protection ring 0.
| -semihosting
|-
| -kernel <elf-image/uImage>
| Load elf-image or uImage and start emulation from its entry point. Note that image is loaded to virtual addresses it is linked for. For XTFPGA this option also chooses 'boot from SRAM' mode, mapping SRAM to the System ROM region at physical address 0xFE000000.
| -kernel xtensa-2.6.29-smp/arch/xtensa/boot/Image.elf
|-
| -append <kernel arguments>
| (Not for sim) Wrap kernel arguments into the bootparameters block and pass pointer to it to the elf-image in a2.
| -append 'console=ttyS0 ip=dhcp root=/dev/nfs rw'
|-
| -initrd <initrd-image>
| (Not for sim) Load initial RAM disk image and pass its location to kernel in the bootparameters block.
| -initrd buildroot-build/images/rootfs.cpio
|-
| -dtb <Device-Tree-Blob>
| (Not for sim) Load DTB image and pass its location to kernel in the bootparameters block.
| -dtb xtensa-build/arch/xtensa/boot/dts/kc705.dtb
|-
| -pflash <flash-image>
| (Not for sim) Mount flash-image as a system FLASH. Note that image size must match platform FLASH size (4MB for LX60, 16MB for LX200 and ML605, 128MB for KC705).
| -pflash u-boot.dc232b_xtav200.flash
|-
| -net nic,<nic-parameters>
| (Not for sim) Set network interface parameters.
| -net nic,vlan=0
|-
| -net user,<SLIRP parameters>
| (Not for sim) Use the user mode network stack. SLIRP parameters may be used to tune built-in DHCP and TFTP servers.
| -net user,vlan=0,tftp=.,hostname=dc232b-lx200
|-
|}

== sim platform ==

The sim platform is modelled after Tensilica ISS. It may run ELF images built for ISS, see e.g. [[Buildroot_Build_Instructions#Build_and_Run_a_Linux_Kernel_on_the_Instruction_Set_Simulator_(ISS)|Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)]].
There's only SIMCALL external interface available for the sim machine. To avoid console interference with QEMU monitor, monitor should be disabled with -monitor null or moved to network interface with e.g. -monitor tcp::1236,telnet,server,nowait.

$ '''qemu-system-xtensa -cpu dc232b -M sim -m 128M -semihosting -nographic -monitor null \'''
'''-kernel sim-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Since the QEMU doesn't manage terminal settings in semihosting mode, ^C and other terminal control codes interrupt QEMU instead of its guest. stty raw may be used to set up terminal not to terminate QEMU.

== XTFPGA (LX60/LX200/ML605/KC705/...) platforms ==

XTFPGA platform emulates system RAM, board-specific FPGA, UART, OpenCores 10/100 Mbit Ethernet MAC, National Semiconductors DP83848C 10/100 PHY and parallel FLASH.
The platform may boot from either SRAM (when -kernel option is specified, SRAM is mapped to the System ROM region and ELF image is loaded) or from FLASH.

NFS root export record for user network (there's no way to specify root-path option for the QEMU builtin DHCP server. Unless kernel parameter 'nfsroot' is given, you'll have to put the rootfs to /tftpboot/<hostname>):
/tftpboot/ 127.0.0.1(rw,no_root_squash,insecure)

Booting linux image from SRAM (FLASH not mapped):
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel lx60-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Booting u-boot image from SRAM (empty FLASH image is created and mapped):
$ '''head -c 16M /dev/zero > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200'''
In this scenario binary u-boot image may be downloaded from TFTP and copied to the FLASH as described [[Setting_up_U-Boot|here]].

Alternatively FLASH image containing u-boot may be preformatted and QEMU may be booted from it:
$ '''cat u-boot-xtensa.git/u-boot.bin.dc232b_xtav200 /dev/zero | head -c 16M > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash'''

(Note that KC705 expects bootloader to be at the offset 0x06000000 in the FLASH, unlike the older boards that boot at offset 0).

== Debugging with QEMU ==
=== GDB ===
There's builtin GDB server in the QEMU. -s command line switch activates it on the address 0.0.0.0:1234 (-gdb tcp:<ip>:<port> may be used instead of -s to change address and port to listen), -S switch stops QEMU at the first guest instruction waiting for 'continue' command from gdb.

$ '''qemu-system-xtensa -M lx200 -nographic -pflash lx200.flash \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200 -s -S'''

By default xtensa gdb may only access unprivileged core registers. To allow access to all available registers ./gdb/xtensa-config.c needs to be modified, marking privileged registers as non-privileged. If [https://github.com/jcmvbkbc/xtensa-toolchain-build these scripts] are used to build the toolchain, the fixup-gdb.sh step does that.

GDB breakpoints, watchpoints, single-stepping and other debugging features work as usual.

$ '''xtensa-dc232b-elf-gdb'''
GNU gdb (GDB) 7.3.50.20110709-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=xtensa-dc232b-elf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) '''target remote 127.0.0.1:1234'''
Remote debugging using 127.0.0.1:1234
0xfe000000 in ?? ()
(gdb) '''symbol-file u-boot-xtensa.git/u-boot.dc232b_xtav200'''
Reading symbols from u-boot-xtensa.git/u-boot.dc232b_xtav200...done.
(gdb) '''x/10i $pc'''
=> 0xfe000000 <_ResetVector>: j 0xfe00000c <_ResetVector+12>
0xfe000003 <_ResetVector+3>: break 0, 0
0xfe000006 <_ResetVector+6>: excw
0xfe000009 <_ResetVector+9>: .byte 0xff
0xfe00000a <_ResetVector+10>: .byte 0xff
0xfe00000b <_ResetVector+11>: .byte 0xff
0xfe00000c <_ResetVector+12>: l32r a2, 0xfe000004 <_ResetVector+4>
0xfe00000f <_ResetVector+15>: jx a2
0xfe000012 <_ResetVector+18>: ill
0xfe000015 <_ResetVector+21>: subx2 a0, a0, a0
(gdb) '''si'''
69 1: l32r a2, 2b
(gdb)

QEMU monitor commands may be invoked via gdb monitor command:

(gdb) '''monitor info mtree'''
memory
00000000-fffffffe (prio 0): system
00000000-07ffffff (prio 0): lx60.dram
f0000000-fdffffff (prio 0): lx60.io
fd020000-fd02ffff (prio 0): lx60.fpga
fd030000-fd030053 (prio 0): open_eth.regs
fd030400-fd0307ff (prio 0): open_eth.desc
fd050020-fd05003f (prio 0): serial
fd800000-fd803fff (prio 0): open_eth.ram
f8000000-f83fffff (prio 0): lx60.io.flash
fe000000-fe01ffff (prio 0): lx60.sram

=== Logging ===

QEMU can log guest CPU state/opcode translation during execution. Invalid, reserved and not implemented opcodes, invalid conditions for certain opcodes, reserved/not implemented SR access, page table lookups, interrupt requests are all logged with this method. Logging may be enabled with -d<logging types> command line option, or with monitor command 'log <logging types>'. By default logs go to the file /tmp/qemu.log. Verbose logging slows down execution significantly, it may be useful to control logging from GDB, enabling it via monitor command for the specific code blocks.

{| border="1"
|-
! Logging type !! Meaning
|-
| none
| disable logging
|-
| out_asm
| show generated host assembly code for each compiled TB
|-
| in_asm
| show target assembly code for each compiled TB (not implemented for Xtensa)
|-
| op
| show micro ops for each compiled TB
|-
| op_opt
| show micro ops after liveness analysis
|-
| int
| show interrupts/exceptions in short format
|-
| exec
| show trace before each executed TB
|-
| cpu
| show CPU state before block translation
|-
|}

== Extending QEMU ==

Starting with QEMU-2.2 there's a script [https://github.com/OSLL/qemu-xtensa/blob/master/target-xtensa/import_core.sh qemu/target-xtensa/import_core.sh] that adds new core to QEMU from the xtensa configuration overlay.

To manually extend QEMU with new xtensa core an appropriate XtensaConfig structure must be registered. This structure may be mostly automatically filled in from the overlay data, see mainline commits [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=53add759be1a39063cfc52ca2a6d0fcdd1091697 53add759be1a39063cfc52ca2a6d0fcdd1091697] (target-xtensa: add dc232b core) and [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=935f7a2b428f71a40e985f09955bccaaf09518fa 935f7a2b428f71a40e985f09955bccaaf09518fa] (target-xtensa: add fsf core) for example:
* target-xtensa/core-<core name>/core-isa.h is a copy of config/core.h from xtensa-config-overlay.tar.gz;
* target-xtensa/core-<core name>/gdb-config.c is made from gdb/xtensa-config.c from xtensa-config-overlay.tar.gz, only rmap entries for PC, A*, AR*, SR*, UR* are retained;
* XtensaConfig::gdb_regmap::num_regs is the total number of rmap entries in the gdb-config.c;
* XtensaConfig::gdb_regmap::num_core_regs is the number of contiguous non-privileged ((flags & 1) == 0) rmap entries in the gdb-config.c.

Xtensa on QEMU

2015-07-20T21:24:51Z

Jcmvbkbc: fix 'extending QEMU' paragraph

== Setting up and running Xtensa simulation on QEMU ==

[http://wiki.qemu.org QEMU] is a free processor/machine emulator and virtualizer.

Starting with version 1.0 QEMU offers [http://wiki.qemu.org/Features/Xtensa Xtensa architecture emulation].
DC232B, DC233C (LE) and fsf (BE) cores are supported on sim and LX60/LX200 platforms.

== Getting and building QEMU ==
QEMU is developed using git. The main tree is located at git://git.qemu.org/qemu.git. Release tarballs and guest systems images are available at http://wiki.qemu.org/Download.

This will make a copy of master QEMU repository, configure, build and install QEMU binaries for little-endian and big-endian targets:
$ '''git clone git://git.qemu.org/qemu.git'''
$ '''mkdir qemu-xtensa ; cd qemu-xtensa'''
$ '''../qemu/configure --prefix=`pwd`/root --target-list=xtensa-softmmu,xtensaeb-softmmu'''
$ '''make install'''

== QEMU options ==
{| border="1"
|-
! Option !! Meaning !! Example
|-
| -M <machine-name>
| Emulate specific platform type. Use '?' to list all supported platform types.
| -M lx60
|-
| -cpu <core-name>
| Emulate specific core type. Use '?' to list all supported core types.
| -cpu dc232b
|-
| -m <memory size>
| Set emulated RAM size. M and G suffixes for mega- and gigabytes.
| -m 128M
|-
| -semihosting
| Enable SIMCALL instruction for the protection ring 0.
| -semihosting
|-
| -kernel <elf-image/uImage>
| Load elf-image or uImage and start emulation from its entry point. Note that image is loaded to virtual addresses it is linked for. For XTFPGA this option also chooses 'boot from SRAM' mode, mapping SRAM to the System ROM region at physical address 0xFE000000.
| -kernel xtensa-2.6.29-smp/arch/xtensa/boot/Image.elf
|-
| -append <kernel arguments>
| (Not for sim) Wrap kernel arguments into the bootparameters block and pass pointer to it to the elf-image in a2.
| -append 'console=ttyS0 ip=dhcp root=/dev/nfs rw'
|-
| -initrd <initrd-image>
| (Not for sim) Load initial RAM disk image and pass its location to kernel in the bootparameters block.
| -initrd buildroot-build/images/rootfs.cpio
|-
| -dtb <Device-Tree-Blob>
| (Not for sim) Load DTB image and pass its location to kernel in the bootparameters block.
| -dtb xtensa-build/arch/xtensa/boot/dts/kc705.dtb
|-
| -pflash <flash-image>
| (Not for sim) Mount flash-image as a system FLASH. Note that image size must match platform FLASH size (4MB for LX60, 16MB for LX200, 32MB for ML605, 128MB for KC705).
| -pflash u-boot.dc232b_xtav200.flash
|-
| -net nic,<nic-parameters>
| (Not for sim) Set network interface parameters.
| -net nic,vlan=0
|-
| -net user,<SLIRP parameters>
| (Not for sim) Use the user mode network stack. SLIRP parameters may be used to tune built-in DHCP and TFTP servers.
| -net user,vlan=0,tftp=.,hostname=dc232b-lx200
|-
|}

== sim platform ==

The sim platform is modelled after Tensilica ISS. It may run ELF images built for ISS, see e.g. [[Buildroot_Build_Instructions#Build_and_Run_a_Linux_Kernel_on_the_Instruction_Set_Simulator_(ISS)|Build and Run a Linux Kernel on the Instruction Set Simulator (ISS)]].
There's only SIMCALL external interface available for the sim machine. To avoid console interference with QEMU monitor, monitor should be disabled with -monitor null or moved to network interface with e.g. -monitor tcp::1236,telnet,server,nowait.

$ '''qemu-system-xtensa -cpu dc232b -M sim -m 128M -semihosting -nographic -monitor null \'''
'''-kernel sim-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Since the QEMU doesn't manage terminal settings in semihosting mode, ^C and other terminal control codes interrupt QEMU instead of its guest. stty raw may be used to set up terminal not to terminate QEMU.

== XTFPGA (LX60/LX200/ML605/KC705/...) platforms ==

XTFPGA platform emulates system RAM, board-specific FPGA, UART, OpenCores 10/100 Mbit Ethernet MAC, National Semiconductors DP83848C 10/100 PHY and parallel FLASH.
The platform may boot from either SRAM (when -kernel option is specified, SRAM is mapped to the System ROM region and ELF image is loaded) or from FLASH.

NFS root export record for user network (there's no way to specify root-path option for the QEMU builtin DHCP server. Unless kernel parameter 'nfsroot' is given, you'll have to put the rootfs to /tftpboot/<hostname>):
/tftpboot/ 127.0.0.1(rw,no_root_squash,insecure)

Booting linux image from SRAM (FLASH not mapped):
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel lx60-2.6.29-smp/arch/xtensa/boot/Image.elf'''

Booting u-boot image from SRAM (empty FLASH image is created and mapped):
$ '''head -c 16M /dev/zero > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200'''
In this scenario binary u-boot image may be downloaded from TFTP and copied to the FLASH as described [[Setting_up_U-Boot|here]].

Alternatively FLASH image containing u-boot may be preformatted and QEMU may be booted from it:
$ '''cat u-boot-xtensa.git/u-boot.bin.dc232b_xtav200 /dev/zero | head -c 16M > lx200.flash'''
$ '''qemu-system-xtensa -cpu dc232b -M lx200 -m 96M -nographic \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 -pflash lx200.flash'''

(Note that KC705 expects bootloader to be at the offset 0x06000000 in the FLASH, unlike the older boards that boot at offset 0).

== Debugging with QEMU ==
=== GDB ===
There's builtin GDB server in the QEMU. -s command line switch activates it on the address 0.0.0.0:1234 (-gdb tcp:<ip>:<port> may be used instead of -s to change address and port to listen), -S switch stops QEMU at the first guest instruction waiting for 'continue' command from gdb.

$ '''qemu-system-xtensa -M lx200 -nographic -pflash lx200.flash \'''
'''-net nic,vlan=0 -net user,vlan=0,tftp=.,hostname=dc232b-lx200 \'''
'''-kernel u-boot-xtensa.git/u-boot.dc232b_xtav200 -s -S'''

By default xtensa gdb may only access unprivileged core registers. To allow access to all available registers ./gdb/xtensa-config.c needs to be modified, marking privileged registers as non-privileged. If [https://github.com/jcmvbkbc/xtensa-toolchain-build these scripts] are used to build the toolchain, the fixup-gdb.sh step does that.

GDB breakpoints, watchpoints, single-stepping and other debugging features work as usual.

$ '''xtensa-dc232b-elf-gdb'''
GNU gdb (GDB) 7.3.50.20110709-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=xtensa-dc232b-elf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) '''target remote 127.0.0.1:1234'''
Remote debugging using 127.0.0.1:1234
0xfe000000 in ?? ()
(gdb) '''symbol-file u-boot-xtensa.git/u-boot.dc232b_xtav200'''
Reading symbols from u-boot-xtensa.git/u-boot.dc232b_xtav200...done.
(gdb) '''x/10i $pc'''
=> 0xfe000000 <_ResetVector>: j 0xfe00000c <_ResetVector+12>
0xfe000003 <_ResetVector+3>: break 0, 0
0xfe000006 <_ResetVector+6>: excw
0xfe000009 <_ResetVector+9>: .byte 0xff
0xfe00000a <_ResetVector+10>: .byte 0xff
0xfe00000b <_ResetVector+11>: .byte 0xff
0xfe00000c <_ResetVector+12>: l32r a2, 0xfe000004 <_ResetVector+4>
0xfe00000f <_ResetVector+15>: jx a2
0xfe000012 <_ResetVector+18>: ill
0xfe000015 <_ResetVector+21>: subx2 a0, a0, a0
(gdb) '''si'''
69 1: l32r a2, 2b
(gdb)

QEMU monitor commands may be invoked via gdb monitor command:

(gdb) '''monitor info mtree'''
memory
00000000-fffffffe (prio 0): system
00000000-07ffffff (prio 0): lx60.dram
f0000000-fdffffff (prio 0): lx60.io
fd020000-fd02ffff (prio 0): lx60.fpga
fd030000-fd030053 (prio 0): open_eth.regs
fd030400-fd0307ff (prio 0): open_eth.desc
fd050020-fd05003f (prio 0): serial
fd800000-fd803fff (prio 0): open_eth.ram
f8000000-f83fffff (prio 0): lx60.io.flash
fe000000-fe01ffff (prio 0): lx60.sram

=== Logging ===

QEMU can log guest CPU state/opcode translation during execution. Invalid, reserved and not implemented opcodes, invalid conditions for certain opcodes, reserved/not implemented SR access, page table lookups, interrupt requests are all logged with this method. Logging may be enabled with -d<logging types> command line option, or with monitor command 'log <logging types>'. By default logs go to the file /tmp/qemu.log. Verbose logging slows down execution significantly, it may be useful to control logging from GDB, enabling it via monitor command for the specific code blocks.

{| border="1"
|-
! Logging type !! Meaning
|-
| none
| disable logging
|-
| out_asm
| show generated host assembly code for each compiled TB
|-
| in_asm
| show target assembly code for each compiled TB (not implemented for Xtensa)
|-
| op
| show micro ops for each compiled TB
|-
| op_opt
| show micro ops after liveness analysis
|-
| int
| show interrupts/exceptions in short format
|-
| exec
| show trace before each executed TB
|-
| cpu
| show CPU state before block translation
|-
|}

== Extending QEMU ==

Starting with QEMU-2.2 there's a script [https://github.com/OSLL/qemu-xtensa/blob/master/target-xtensa/import_core.sh qemu/target-xtensa/import_core.sh] that adds new core to QEMU from the xtensa configuration overlay.

To manually extend QEMU with new xtensa core an appropriate XtensaConfig structure must be registered. This structure may be mostly automatically filled in from the overlay data, see mainline commits [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=53add759be1a39063cfc52ca2a6d0fcdd1091697 53add759be1a39063cfc52ca2a6d0fcdd1091697] (target-xtensa: add dc232b core) and [http://jcmvbkbc.spb.ru/git/?p=dumb/qemu-xtensa.git;a=commitdiff;h=935f7a2b428f71a40e985f09955bccaaf09518fa 935f7a2b428f71a40e985f09955bccaaf09518fa] (target-xtensa: add fsf core) for example:
* target-xtensa/core-<core name>/core-isa.h is a copy of config/core.h from xtensa-config-overlay.tar.gz;
* target-xtensa/core-<core name>/gdb-config.c is made from gdb/xtensa-config.c from xtensa-config-overlay.tar.gz, only rmap entries for PC, A*, AR*, SR*, UR* are retained;
* XtensaConfig::gdb_regmap::num_regs is the total number of rmap entries in the gdb-config.c;
* XtensaConfig::gdb_regmap::num_core_regs is the number of contiguous non-privileged ((flags & 1) == 0) rmap entries in the gdb-config.c.

Linux Kernel

2015-06-25T04:13:02Z

Jcmvbkbc: update linux features list

= Getting linux kernel sources =
Now that linux mainline has up-to-date xtensa support kernel.org is the primary place to go for xtensa linux.
Depending on whether kernel tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest linux release archive: 'mainline' at the https://www.kernel.org/ points to it. May be buggy though.
* latest stable release archive: 'stable' at the https://www.kernel.org/ points to it. The best choice usually.
* mainline linux git tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/ . Use git clone with one of the URLs at that page to clone it.
* stable linux git tree: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/ . Use git clone with one of the URLs at that page to clone it.

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

= Adding specific processor configuration =
Note that the Xtensa port of Linux uses the terminology ''variant'' to describe a processor configuration. The steps involved adding a processor configuration are:

# Create a new directory for the processor ''variant'' and copy a set of configuration files that describes the processor to that directory.
# When configuring the kernel set .config variable CONFIG_XTENSA_VARIANT_CUSTOM_NAME to the name of used variant.

All processor configuration specific sources and header files are located in arch/xtensa/variants/''variant-name'' for source files, and arch/xtensa/variants/''variant-name''/include/variant for header files. The include directory typically contains at least the following three files that describe the processor configuration:

* tie.h - describing custom defined TIE registers
* tie-asm.h - macros to access additional TIE registers
* core.h - describing various processor configurations, such as cache sizes, register options, etc.

All processor configurations provide a set of these files as part of the ''overlay'' provided by Tensilica.

If we want to add another processor, we have to create the following directory and copy the three configurations file to that directory:

mkdir -p arch/xtensa/variants/MyCore/include/variant
cp tie.h tie-asm.h core.h arch/xtensa/variants/MyCore/include/variant

= Configuring and building the kernel =
To build the kernel a toolchain for the target variant is needed: gcc (it doesn't need to have libc) and binutils. Currently the kernel can not be built with xcc. The toolchain may be built with [[crosstool-NG]] or [[buildroot]].

Define CROSS_COMPILE=''cross-compiler-spec'' environment variable to specify cross-compiler to be used. ''cross-compiler-spec'' is a path to cross-compiler binaries plus cross-compiler name up to its last component (e.g. if your cross-compiler is /opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-gcc then specify '''CROSS_COMPILE=/opt/xtensa/tools/fsf/bin/xtensa-fsf-elf-''' to use it).

The kernel may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree kernel configuration/building.

== Configuring the kernel ==
Configuring the kernel creates build infrastructure and .config file in the build directory. .config file may be written from scratch, copied from the outside source or taken from a predefined configuration and edited. For a list of predefined configurations see contents of directory arch/xtensa/configs. It's a good idea to run configuration after copying external or predefined config, as newer kernel releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to use predefined .config run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' ''predefined_config_name'' '''
* to edit existing configuration run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' menuconfig'''.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* CONFIG_XTENSA_VARIANT_CUSTOM: must be set when a custom core variant is used
* CONFIG_XTENSA_VARIANT_CUSTOM_NAME: must specify custom core variant name
* CONFIG_XTENSA_VARIANT_MMU: if set to 'N' will build noMMU kernel even when core variant actually supports MMU. Should always be set for MMUv2 cores.
* CONFIG_HAVE_SMP: must always be set correctly, as presence of MX interrupt distributor affects external interrupts numbering.
* CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX: controls whether uImage will reinitialize memory mapping. Should be set for noMMU and MMUv3 cores, unless MMU is fully initialized by uBoot. Ignored for MMUv2 cores. Has no effect for Image.elf, as it has to initialize MMU either in reset vector or in vmlinux.
* XTENSA_PLATFORM_*: must specify target platform. xtensa currently does not support multiplatform kernels.
* CONFIG_CMDLINE: default kernel command line. Gets overridden by command line specified in the device tree (built-in or externally loaded) or passed from the bootloader.
* BUILTIN_DTB: default kernel device tree. Only used when device tree is not supplied by the bootloader.

* CONFIG_DEVTMPFS: enables kernel self-maintained /dev filesystem.
* CONFIG_SERIAL_OF_PLATFORM: enables UART driver that may be instantiated from the device tree.
* CONFIG_LD_NO_RELAX: disables link-time relaxation at kernel build. This is useful against linker relaxation bugs and currently speeds up the build significantly.

== Building the kernel ==

* to build kernel and modules run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' all'''
* to build only the kernel run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' make O=''build-dir-path'' zImage'''
* to install kernel modules into ''modules-dir'' run '''ARCH=xtensa CROSS_COMPILE=''cross-compiler-spec'' INSTALL_MOD_PATH=''modules-dir'' make O=''build-dir-path'' modules_install'''
* all commands supported by kernel build system may be listed with '''ARCH=xtensa make help'''

The resulting binaries are located relatively to the ''build-dir-path'':
* vmlinux: kernel ELF image, useful for debugging.
* arch/xtensa/boot/Image.elf: kernel with reset vector handler, may be used with xt-ocd or QEMU.
* arch/xtensa/boot/uImage: kernel image for uBoot, may as well be used with QEMU.

Built modules are located in their respective directories.

= Booting the kernel =
== ISS platform ==
Kernels built for ISS are only bootable on Xtensa ISS (xt-run) or on QEMU. Neither xt-run nor QEMU can pass parameters to the ISS kernel, so normally it should have everything built-in: kernel command line and possibly initramfs. The easiest way to generate initramfs cpio archive is by setting BR2_TARGET_ROOTFS_CPIO to y in the buildroot configuration.

The kernel can also be configured to use simdisk, which is a host file with a file system image for the guest: see kernel configuration parameters CONFIG_BLK_DEV_SIMDISK and CONFIG_SIMDISK0_FILENAME. If CONFIG_SIMDISK0_FILENAME is a relative path it is relative to the current directory of the running simulator. The easiest way to generate file system image is by setting BR2_TARGET_ROOTFS_EXT2 to y in the buildroot configuration.

Running linux in Xtensa ISS:
$ '''xt-run --turbo --memlimit=128 ''build-dir-path''/arch/xtensa/boot/Image.elf'''
Since xt-run doesn't manage terminal settings, ^C and other terminal control codes interrupt xt-run instead of its guest. '''stty raw''' may be used to set up terminal not to terminate xt-run.

Running linux in QEMU: see [[Xtensa_on_QEMU#sim_platform|QEMU reference]] page.

== XTFPGA platform ==
XTFPGA is a collective name for FPGA boards capable of running Xtensa bitstream: LX60, LX110, LX200, ML605 and KC705.

Kernels built for XTFPGA may be booted on FPGA board and on QEMU (see [[Xtensa_on_QEMU#XTFPGA_.28LX60.2FLX200.2FML605.2FKC705.2F....29_platforms|QEMU reference]] page on how to do it).

There are two supported ways of booting linux on FPGA board: with xt-ocd and with uBoot.

=== Booting with xt-ocd ===
To use xt-ocd Xtensa tools need to be installed, boot from flash disabled on the board (refer to the specific board prototyping guide for the location of corresponding DIP switch), JTAG probe connected to the board and topology.xml file set up. Please refer to the Tensilica On Chip Debugging Guide (onchip_debug_guide.pdf) for the list of supported probes and setup instructions.

Sample topology.xml for the KC705 daughterboard built-in USB JTAG probe and single core bitstream:
<pre>
<configuration>
<controller id='Controller0' module='ft2232' probe='Dual RS232-HS' speed='10MHz' examine='1'/>
<driver id='XtensaDriver0' module='xtensa' step-intr='mask,stepover,setps' />
<chain controller='Controller0'>
<tap id='TAP0' irwidth='5' />
</chain>
<system module='jtag'>
<component id='Component0' tap='TAP0' config='trax' />
</system>
<device id='Xtensa0' component='Component0' driver='XtensaDriver0' />
<application id='GDBStub' module='gdbstub' port='22000'>
<target device='Xtensa0' />
</application>
</configuration>
</pre>

Running xt-ocd:
$ '''xt-ocd -c topology.xml'''

Booting kernel on XTFPGA boards with xt-ocd + xt-gdb:
$ '''xt-gdb --xtensa-core=''xtensa-core-name'' ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target remote ''xt-ocd-address-and-port'' '''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''reset'''
(xt-gdb) '''load'''
(xt-gdb) '''c'''

=== Booting with uBoot ===
Please see [[Setting_up_U-Boot|Setting up U-Boot]] page for instructions on installing uBoot onto the board.

uBoot can load kernel uImage from FLASH or from the network via TFTP.
==== Loading kernel from FLASH ====
Kernel may be stored as raw data in a separate FLASH partition, or it may be stored in a FLASH file system and loaded as a file.
The following steps load kernel from UBIFS volume on a FLASH partition:
* make sure that mtdparts environment variable is set correctly. If it's not, use 'mtdparts default' command to reset MTD partitions configuration to default, or 'mtdparts add' to add partitions manually. Example of correct default settings for KC705:
U-Boot> '''mtdparts'''

device nor0 <f0000000.flash>, # parts = 4
#: name size offset mask_flags
0: data 0x06000000 0x00000000 0
1: boot loader area 0x00800000 0x06000000 0
2: kernel image 0x017e0000 0x06800000 0
3: boot environment 0x00020000 0x07fe0000 0

active partition: nor0,0 - (data) 0x06000000 @ 0x00000000

defaults:
mtdids : nor0=f0000000.flash
mtdparts: mtdparts=f0000000.flash:0x06000000(data),0x00800000(boot loader area),0x017e0000(kernel image),0x00020000(boot environment)

* attach MTD partition to UBI with 'ubi part ''mtd-partition-name''':
U-Boot> '''ubi part ''data'''''
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 130944 bytes
UBI: smallest flash I/O unit: 1
UBI: VID header offset: 64 (aligned 64)
UBI: data offset: 128
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=0"
UBI: MTD device size: 96 MiB
UBI: number of good PEBs: 768
UBI: number of bad PEBs: 0
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 1
UBI: available PEBs: 0
UBI: total number of reserved PEBs: 768
UBI: number of PEBs reserved for bad PEB handling: 0
UBI: max/mean erase counter: 8/4

* mount UBI volume that contains kernel uImage with 'ubifsmount ubi:''volume-name''':
U-Boot> '''ubifsmount ubi:''data'''''
UBIFS: mounted UBI device 0, volume 0, name "data"
UBIFS: mounted read-only
UBIFS: file system size: 98731776 bytes (96417 KiB, 94 MiB, 754 LEBs)
UBIFS: journal size: 4975872 bytes (4859 KiB, 4 MiB, 38 LEBs)
UBIFS: media format: w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root: 4879407 bytes (4765 KiB)

* load kernel image from the mounted file system with 'ubifsload ''address'' ''path-to-uImage''':
U-Boot> '''ubifsload 0x02000000 uImage'''
Loading file 'uImage' to addr 0x02000000 with size 2850403 (0x002b7e63)...
Done

==== Loading kernel from TFTP ====
In order for uBoot to be able to load uImage, device tree blob and initramfs they need to be copied somewhere under the tftproot and network must be configured in uBoot.

= Rootfs =
== Rootfs in initial RAMFS ==
== Rootfs over NFS ==
== Rootfs in FLASH (UBIFS) ==
The easiest way to generate UBIFS image is by setting BR2_TARGET_ROOTFS_UBIFS to y in the buildroot config. Buildroot will also need the following file system parameters, that can be queried with mtdinfo tool:
* BR2_TARGET_ROOTFS_UBIFS_LEBSIZE: logical erase block (LEB) size (130944 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE: minimum I/O unit size (1 for this device)
* BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT: maximum logical eraseblock count (this device has 768, 764 are usable, as reported by ubiattach).

# '''mtdinfo -a -u'''
<pre>
Count of MTD devices: 4
Present MTD devices: mtd0, mtd1, mtd2, mtd3
Sysfs interface supported: yes

mtd0
Name: data
Type: nor
Eraseblock size: 131072 bytes, 128.0 KiB
Amount of eraseblocks: 768 (100663296 bytes, 96.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size: 1 byte
Character device major/minor: 90:0
Bad blocks are allowed: false
Device is writable: true
Default UBI VID header offset: 64
Default UBI data offset: 128
Default UBI LEB size: 130944 bytes, 127.9 KiB
Maximum UBI volumes count: 128
</pre>

To format FLASH partition and put UBIFS image into it the following command sequence may be used:
* run ubiformat:
# '''ubiformat /dev/mtd0'''
ubiformat: mtd0 (nor), size 100663296 bytes (96.0 MiB), 768 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 1 bytes
libscan: scanning eraseblock 767 -- 100 % complete
ubiformat: 768 eraseblocks have valid erase counter, mean value is 2
ubiformat: formatting eraseblock 767 -- 100 % complete
* attach formatted device to UBI (this will create /dev/ubi0 device):
# '''ubiattach -p /dev/mtd0'''
UBI device number 0, total 768 LEBs (100564992 bytes, 95.9 MiB), available 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB)
* create UBI volumes on formatted device (this will create /dev/ubi0_0 device, AKA ubi0:rootfs):
# '''ubimkvol /dev/ubi0 -m -N rootfs'''
Set volume size to 100041216
Volume ID 0, size 764 LEBs (100041216 bytes, 95.4 MiB), LEB size 130944 bytes (127.9 KiB), dynamic, name "rootfs", alignment 1
* put UBI rootfs image so that xtensa linux could access it (in this example kernel is booted with NFS root and UBI image goes to /mnt/rootfs.ubifs)
* run ubiupdatevol to copy file system image to the specific UBI volume:
# '''ubiupdatevol /dev/ubi0_0 /mnt/rootfs.ubifs'''
* now the volume may be mounted:
# '''mount -t ubifs /dev/ubi0_0 /mnt'''
* to use that volume as rootfs add the following parameters to the kernel command line (note that 0 in the 'ubi.mtd=0' is the number of mtd partition to bind to ubi0, and 'rootfs' in the ubi0:rootfs below is the volume name):
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

Further reading:
* http://www.linux-mtd.infradead.org/doc/ubifs.html
* http://www.linux-mtd.infradead.org/faq/ubifs.html
* https://www.kernel.org/doc/Documentation/filesystems/ubifs.txt

= Features by release =

== Linux 4.1 ==
* ALSA playback driver for XTFPGA.

== Linux 3.19 ==
* noMMU.

== Linux 3.17 ==
* Highmem on cores with aliasing cache.
* New (simpler) method of configuring for specific core variant.

== Linux 3.16 ==
* Stability: xtensa linux may be rebuilt natively.

== Linux 3.15 ==
* Highmem.
* KC705 board.

== Linux 3.14 ==
* SMP.
* Futex.
* register spilling overhaul.
* OpenCores ethernet works when connected to gigabit network.
* c67x00 USB host is usable on XTFPGA.
* TAP network works in ISS.

== Linux 3.12 ==
* Kernel preemption.

== Linux 3.11 ==
* Multihit exceptions fixed.
* Static function tracer.

== Linux 3.10 ==
* MMUv3 cores.
* Lockdep/stack tracing.

== Linux 3.9 ==
* TLS (THREADPTR register).
* Oprofile.
* ISS simdisk.

== Linux 3.8 ==
* Device trees.
* uImage generation.
* IRQ subsystem overhaul (IRQ domains, medium-priority interrupts).
* XTFPGA boards.
* s32c1i-based atomic/bit operations and spinlocks.

== Linux 3.7 ==
* Overall architecture port overhaul, mainline is alive again, builds and runs.
* There were dragons prior to that release.

= Kernel loadable modules using FLIX with L32R =

The kernel module loader (for the Xtensa architecture) supports only the limited set of relocations needed for module loading. This includes, for example, relocation of literals referenced by L32R instructions. This loader is not given specific information about any custom TIE configured in the processor, and thus is currently not able to relocate L32R instructions encoded in a FLIX bundle.

To ensure kernel modules load properly, they need to avoid L32R instructions within FLIX instruction bundles.

Existing kernel code does not generate such instructions, so it is not susceptible.
However, it is possible to generate FLIX instructions using assembly code, or using the Tensilica XCC compiler. In the case of assembly code, simply avoid using L32R or MOVI instructions within a FLIX instruction bundle, unless a MOVI is known to fit without expanding ("relaxing") into an L32R instruction. In the case of XCC, pass the -mno-flix to the compiler (xt-xcc) to avoid generating FLIX bundles.

'''Note''': The use of Tensilica software tools (including XCC) for Linux development is described in the ''Tensilica OSKit Guide'', available from Tensilica.

== Debugging Kernel Loadable Modules ==

Debugging loadable modules isn't easy, a few pointers mentioned here might save you some time. There is a mechanism that Amit Kalie mentioned on his KGDB webpage on loading the module symbols into gdb while loading a module [http://kgdb.linsyssoft.com/initmodule.htm Debuging init_module].

The [http://linux.about.com/od/commands/l/blcmdl8_insmod.htm <tt>insmod</tt> command] has a <tt>''--map''</tt>
option that dumps the module component offsets needed by Amit's approach.
Unfortunately the initial Xtensa buildroot snapshots have a version of <tt>insmod</tt> that doesn't provide this option. Also, while debugging the kernel it's not convenient to run <tt>insmod</tt> on the console of the target.

A workaround to this problem is to hack the kernel <tt>module.c</tt> file and enable debugging.

#if 1
#define DEBUGP printk
#else
#define DEBUGP(fmt , a...)
#endif

This will enable <tt>module.c</tt> to print the module section addresses. You then just list them in the gdb <tt>add-symbol-file</tt> command using that command's section (<tt>''-s''</tt>) options.
Note that the <tt>.text</tt> section is specified without using the <tt>''-s''</tt> option: its address is given alone before any <tt>''-s''</tt> option.

final section addresses: [Kernel Console Log/Output]
0xc02a9098 .note.gnu.build-id
0xc02b1000 .init.literal
0xc02a7000 .text
0xc02b11d8 .init.text
0xc02a90bc .rodata
0xc02a92c4 .rodata.str1.4
0xc02ac27c .data
0xc02ac640 __param
0xc02ac668 .gnu.linkonce.this_module
0xc02ac76c .bss
0xc02aaf0c .symtab
0xc02ab8fc .strtab

add-symbol-file /export/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/ctlmeth.ko \
0xc02a7000 \
-s .data 0xc02ac27c \
-s .init.text 0xc02b11d8 \
-s .bss 0xc02ac76c \
-s .init.literal 0xc02b1000 \
-s .rodata 0xc02a90bc

You likely want to compile your modules unoptimized and unstripped to facilitate debugging and provide the path to the module source to gdb with the <tt>dir</tt> command.
For example:
dir /home/pdelaney/src/Transwitch/a2000_src/a2000_src/Linux_atlanta/ctlm-cpe/drivers/ctlmeth/

== Debugging on ISS ==

$ '''xt-gdb ''build-dir-path''/arch/xtensa/boot/Image.elf'''
(xt-gdb) '''target sim --turbo --memlimit=128'''
(xt-gdb) '''symbol-file ''build-dir-path''/vmlinux'''
(xt-gdb) '''run'''

Instructions for building and booting Linux

2015-03-12T18:58:23Z

Jcmvbkbc:

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[Setting_up_U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

= Booting GNU/Linux =
TBD

Instructions for building and booting Linux

2015-03-12T18:57:23Z

Jcmvbkbc: fix typos

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board or ISS. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting single image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[Setting_up_U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=$(pwd)/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

Instructions for building and booting Linux

2015-03-12T18:52:26Z

Jcmvbkbc: Rewrite the page, overview and building parts.

This page describes what parts are necessary in order to run GNU/Linux OS on Xtensa, their interaction and setup.

= GNU/Linux on Xtensa =

GNU is a collection of free software that provides POSIX environment.
Linux is an operating system kernel.
In order to boot GNU/Linux OS on Xtensa the following parts are needed:

* a board with Xtensa core supported by Linux, or Xtensa simulator
* bootable Linux kernel image
* GNU software packaged in a form suitable for booting on Linux, the so called root file system
* a bootloader or on-chip debugger capable of loading Linux kernel image into the board RAM

To build Linux image and root file system a toolchain correctly configured for used Xtensa core is needed. Toolchain configuration is performed by application of [[Toolchain_Overlay_File|Xtensa configuration overlay]]. Correct toolchain configuration is critical for building other pieces of software correctly.

== Xtensa boards supported by Linux ==

The following boards collectively referred as XTFPGA family are readily supported by Linux: Avnet LX60, LX110, LX200; Xilinx ML605, KC705. For board details/setup guide please see corresponding Board Guide or Board Prototyping Guide provided with Xtensa software release.

== Xtensa simulators ==

* Xtensa software provides Instruction Set Simulator (xt-run) capable of running Linux configured for ISS.
* QEMU is a free system emulator capable of emulating any XTFPGA board. It can be configured to emulate any Xtensa core, but it does not currently support TIE extensions to the core architecture. Please see [[Xtensa_on_QEMU|this]] page for more details.

== Linux kernel ==

Linux kernel needs to be configured for the specific Xtensa core. For that reason Linux is usually configured and built separately from the userspace software. Please see [[Linux_Kernel|this]] page for more details.

== Root file system ==

Currently we support building GNU software and packaging it into root file system with the [[Buildroot]].
There are several types of root file system that Linux can boot with:

=== CPIO archive ===
* CPIO archive may be built into the kernel (so called 'built-in initramfs'), the resulting signle image will contain both Linux and GNU software. It's easy to boot, but in order to change anything the kernel needs to be rebuilt.
* CPIO archive may also be loaded separately from the kernel, but used in the same way as the built-in initramfs. It's harder to boot (two files instead of one), but easier to change.

Both cases of CPIO use RAM for the root file system, so they're very fast. The downsides are that they take some time at startup to unpack, reserve some RAM (which may be an issue on low-memory boards like LX60) and are not persistent.

=== EXT2/3/4 file system image ===
EXT2/3/4 file system image may be used directly with ISS or QEMU through the Simulated block device Linux feature. Please see [[Linux_Kernel#ISS_platform|this]] chapter for more details.

=== TAR image ===
Tar image may be unpacked into a directory shared through NFS. Linux on XTFPGA boards and on QEMU can be configured to use NFS root file system.

=== UBIFS image ===
UBIFS image may be written directly to the daughterboard FLASH on XTFPGA boards. Please see [[Linux_Kernel#Rootfs_in_FLASH_.28UBIFS.29|this]] chapter for more details.

== Bootloader ==
U-Boot can be used as a bootloader on XTFPGA boards. Please see [[Setting_up_U-Boot|this]] page for more details about installing it into the daughterboard FLASH.

== On-chip debugger ==
On-chip debugger is good for debugging lowest level software: bootloader and Linux kernel itself. It also allows loading executable images into new board with empty FLASH.
It consists of JTAG probe connected to the XTFPGA daughterboard (ML605 and KC705 daughterboards also have built-in JTAG chip), OCD monitor (xt-ocd) running on developer machine where the other end of JTAG probe is connected, and xt-gdb, the modified version of GNU debugger from the Xtensa software release. Please see Tensilica Debug User's Guide for more information about OCD setup and configuration.

= Building GNU/Linux OS =
In order to follow these instruction a build machine with GNU/Linux and a set of development tools (native binutils and gcc, make, wget, tar, ...) is needed. In the following example all software will be downloaded and built in the current directory. Xtensa tools used in the examples are installed into $XT_ROOT, tools release is $XT_RELEASE and selected core is $XT_CORE.

== Prepare the overlay ==
Please see [[Toolchain_Overlay_File|this]] page for overlay repacking script and details of the overlay source location.
Create conversion script by pasting [[Toolchain_Overlay_File#Conversion_Script|this]] into the file overlay-script and making it executable:
$ '''cat > overlay-script'''
'''....'''
^D
$ '''chmod +x overlay-script'''

Run it and rename the resulting file:
$ '''./overlay-script ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/src/xtensa-config-overlay.tar.gz'''
$ '''mv xtensa-overlay.tar xtensa_${XT_CORE}.tar'''

== Get, configure and build the toolchain and root file system ==
Get the buildroot git tree:
$ '''git clone git://git.buildroot.net/buildroot'''

Check out some released version:
$ '''git checkout 2014.11'''

Create build directory and configure it:
$ '''mkdir buildroot-build'''
$ '''make -C buildroot O=$(pwd)/buildroot-build menuconfig'''

Please see [[Buildroot#Configuring_and_building_Buildroot|this]] chapter for more details about configuring the Buildroot.

Select 'Target Architecture' = Xtensa, 'Target Architecture Variant' = Custom Xtensa processor configuration, set 'Custom Xtensa processor configuration name' to the contents of ${XT_CORE}, set 'Overlay directory for custom configuration' = $(TOPDIR)/..

Select toolchain options, packages to be built and final root file system image type.

Build the toolchain and the root file system:
$ '''make -C buildroot O=`pwd`/buildroot-build'''

== Get, configure and build the Linux kernel ==
Get stable Linux release tree:
$ '''wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz'''
$ '''tar -xf linux-3.19.1.tar.xz'''

Create variant directory for the specific core and copy core configuration files there:
$ '''mkdir -p linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''
$ '''cp ${XT_ROOT}/XtDevTools/install/builds/${XT_RELEASE}-linux/${XT_CORE}/config/*.h linux-3.19.1/arch/xtensa/variants/${XT_CORE}/include/variant'''

Create build directory and configure it:
$ '''mkdir linux-build'''
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build menuconfig'''

Select 'Xtensa Processor Configuration' = Custom Xtensa processor configuration, set 'Xtensa Processor Custom Core Variant Name' to the contents of ${XT_CORE}. Select desired 'Xtensa System Type'.

Please see [[Linux_Kernel#Configuring_the_kernel|this]] chapter for more details about configuring Linux.

Build the kernel:
$ '''ARCH=xtensa CROSS_COMPILE=$(pwd)/buildroot-build/host/usr/bin/xtensa-buildroot-linux-uclibc- make -C linux-3.19.1 O=$(pwd)/linux-build all'''

Buildroot

2015-03-12T14:54:55Z

Jcmvbkbc: add 'features by release' for the Buildroot

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain and a root filesystem. Building linux kernel and bootloader for arbitrary Xtensa core with the Buildroot is currently not supported.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)

= Features by release =

== 2014.11 ==
* fix 'dangerous relocation: call8: call target out of range' linker bug
* fix 'Error: jump target out of range; no usable trampoline found' and 'Error: operand 1 of 'j' has out of range ' assembler bugs

== 2014.08 ==
* fix xtensa linker segfault on linking linux modules

== 2014.05 ==
* fixes for building with -mtext-section-literals
* assembler trampolines (able to assemble files with jumps longer than 128K)
* fixes for 'dangerous relocation: call8: misaligned call target' and 'dangerous relocation: overflow after relaxation' linker bugs

== 2014.02 ==
* fix overlay application to gcc

== 2012.11 ==
* xtensa in the Buildroot mainline

Buildroot

2015-03-12T13:20:32Z

Jcmvbkbc: Add the Buildroot instructions

This page describes the steps to build a toolchain and root file system for a specific processor configuration based on the latest release of Buildroot.

= Getting Buildroot and other prerequisites =
Buildroot is a set of Makefiles and patches that makes it easy to generate a complete embedded Linux system. Buildroot can generate a cross-compilation toolchain and a root filesystem. Building linux kernel and bootloader for arbitrary Xtensa core with the Buildroot is currently not supported.
== Getting Buildroot ==
Depending on whether buildroot tree with full history is needed for development or not either release tarball should be downloaded or git tree cloned.
* latest buildroot release archive: links in the Download section at http://buildroot.net/download.html
* mainline buildroot git tree: <code>git clone git://git.buildroot.net/buildroot</code>

Using tip of the cloned git tree for work is usually a bad idea, basing development branches on known releases (marked by tags) would give more predictable results.

== Xtensa core overlay file ==
The overlay file includes patches for toolchain tools, such as compiler, assembler, and debugger, that are required to support a specific Xtensa processor configuration. Tensilica provides such an overlay as part of the processor download, however, it needs to be reformatted to match the specific format required by buildroot. For a script to convert the overlay file, and additional information, please go to [[Toolchain Overlay File]]. The current version of buildroot requires that the overlay file name has the format xtensa_<CORE_NAME>.tar, where CORE_NAME can be any user selected name.

= Configuring and building Buildroot =

The Buildroot may be built in the same directory as its source (in-tree) or in a separate directory (out-of-tree). Building out-of-tree is usually cleaner and involves less actions when several configurations need to be built.
Adding parameter O=''build-dir-path'' to make controls out-of-tree Buildroot configuration/building.

For more information about Buildroot, please also read the documents provided by Buildroot under http://buildroot.net/docs.html.

== Configuring the Buildroot ==

Configuring the Buildroot creates build infrastructure and .config file in the build directory. .config file may be written from scratch or copied from the outside source. It's a good idea to run configuration after copying external or predefined config, as newer Buildroot releases may have added/removed or changed configuration parameters and the config file may need to be updated.

* to use external .config just copy it to the build directory
* to edit existing configuration or to create it from scratch run <code>make O=''build-dir-path'' menuconfig</code>.

To search for a specific configuration item in curses interface use '/'.
The following configuration parameters are important/interesting/worth checking:
* Target options submenu:
** BR2_xtensa: must be selected when building for Xtensa
** BR2_XTENSA_CUSTOM: must be set when a custom core variant is used
** BR2_XTENSA_CUSTOM_NAME: must specify custom core variant name. This name is used to derive configuration overlay file name.
** BR2_XTENSA_OVERLAY_DIR: path to the directory with overlay file. May be relative to the Buildroot directory (specified as $(TOPDIR)/...) or the build directory (specified as $(BASE_DIR)/...)
* Toolchain submenu: select toolchain component versions and options. Many packages depend on presence of specific options, like IPv6 support or WCHAR support. Usually a package that cannot be built with current settings will indicate missing options in the menu.
* Target packages submenu: select individual packages and libraries that will be built and installed to the rootfs.
* Filesystem images submenu:
** BR2_TARGET_ROOTFS_CPIO: select to build CPIO image suitable for use as linux initramfs
** BR2_TARGET_ROOTFS_EXT2: select to build ext filesystem suitable for use with simulated block device
** BR2_TARGET_ROOTFS_TAR: select to build tar image, that may be unpacked and used for NFS root

== Building buildroot ==

Run <code>make O=''build-dir-path''</code> to build the toolchain and root file system, or <code>make O=''build-dir-path'' toolchain</code> to build only the toolchain.

== Re-configuring and re-building Buildroot ==

The Buildroot does not support uninstalling packages from its build directories. It also does not track dependencies between the built packages. The easiest way to make sure that deselected package is removed from the rootfs, or that changed toolchain option is applied to built packages is cleaning up build directory, leaving only .config in it and re-building everything from scratch.

= Build Buildroot with Crosstool-NG =

Using the Crosstool-NG toolchain allows to build a version of a Buildroot image based on GlibC instead of uClibc. In the current version, Buildroot cannot build Crosstool-NG for Xtensa directly, but can be configured to use it as an ''external'' toolchain. The first step is to build and install Crosstool-NG, so Buildroot can use it to compile the root file system. More information to build Crosstool-NG for Xtensa can be found under [[Crosstool-NG]]. Assuming Crosstool-NG is installed in 'ABSOLUTE_PATH_TO_XTOOLS', which is typically <code>$HOME/x-tools</code>, you can make the following additional configurations changes in Buildroot:

Toolchain --->
Toolchain type (External toolchain)
Toolchain (Custom toolchain)
Toolchain origin (Pre-installed toolchain)
(ABSOLUTE_PATH_TO_XTOOLS) Toolchain path
($(ARCH)-unknown-linux-gnu) Toolchain prefix
External toolchain C library (glibc/eglibc)