cross-toolchains are typically used to build code that is supposed to run on a machine with a different architecture, e.g. when developing embedded code for an ARM-based box on a x86 dev machine. however, they’re also very useful when you want to make sure that all the devs in a group use exactly the same set of toolchain binaries and libraries regardless of which versions of gcc, headers and libraries those devs have installed locally on their dev machines.
a cross-toolchain consists of a bunch of tools and libraries, which must play well together if it’s ever to be useful. coming up with a specific set of specific versions of tools/libs that will peacefully co-exist is not for the faint at heart, especially if you have wide-ranging requirements and want extra goodies (which you will want). it is even more of a PITA if you want to build a static (and relocatable) cross-toolchain.
crosstool-NG can ease the pain somewhat, though it will not make it go away totally. it’s a rewrite of the original crosstool by Yann E. Morin. for an old overview of this tool given by Yann at Embedded Linux Conference Europe 2009 see: video, slides.
crosstool-NG is NOT in fedora’s yum repos (as of F15), so grab it from its home page, e.g.:
to build crosstool-NG itself, you’ll need at least:
sudo yum install bison flex gperf
then the standard:
./configure --prefix=$HOME/tmp/ct-ng && make -j4 && make install
crosstool-NG (from here on “CT”) comes with a largish set of “Preconfigured toolchains”, or “samples”. those, allegedly, were proven to work together to one degree or another, and can provide inspiration (read: copy-paste) for creating your own. to see the list of such samples:
to see the components (tools/libs) that make up such a sample and their versions:
e.g. (an actual sample output):
ct-ng show-x86_64-unknown-linux-gnu x86_64-unknown-linux-gnu [G ] OS : linux-184.108.40.206 Companion libs : gmp-4.3.2 mpfr-2.4.2 ppl-0.10.2 cloog-ppl-0.15.9 libelf-0.8.13 binutils : binutils-2.20.1a C compiler : gcc-4.4.3 (C,C++,Fortran,Java) C library : glibc-2.9 Tools : dmalloc-5.5.2 duma-2_5_15 gdb-6.8a ltrace-0.5.3 strace-4.5.19
to see similar output for a toolchain you config yourself using
after a number of false-starts, i arrived at the following mix of tools/libs versions that are close to my requirements and can be built into a static cross-toolchain on fedora-15, and seem to pass initial tests:
x86_64-acme-linux-gnu [l X] OS : linux-3.0.18 Companion libs : gmp-4.3.2 mpfr-3.1.0 ppl-0.10.2 cloog-ppl-0.15.10 mpc-0.9 libelf-0.8.13 binutils : binutils-2.21.53 C compiler : gcc-4.5.3 (C,C++) C library : glibc-2.13 Tools : dmalloc-5.5.2 duma-2_5_15 gdb-7.3a ltrace-0.5.3 strace-4.6
resulting disk usage is around 6GB, spread as follows:
~5.5GB $CT_ROOT/.build ~0.5GB $CT_XTOOLS/x86_64-acme-linux-gnu
building the cross-toolchain:
dir names used here:
$CT_INST- this is the
--prefixyou specified during
./configure, where your installed CT will land, assuming non-root install.
$CT_ROOT- this is wherever you run
ct-ngbinary from. CT will do everything under that dir, so make sure to run
ct-nginside a dedicated dir.
$CT_XTOOLS- root of the resultant cross-toolchains’ prefix dirs, this is where the final toolchains land.
flip through the dialogs of:
choosing the packages you want. for best results you’ll prolly want to enable
EXPERIMENTAL stuff early on, otherwise most of the recent package
versions will not be available and a lot of useful cross-toolchain
functionality will be missing.
first things first:
- “Paths and misc options” ->
- “Prefix directory” - this is where the final cross-toolchain will
$CT_XTOOLSin this document. it defaults under your
$HOME/x-tools, change as necessary.
- “Number of parallel jobs” - set according to your number of cores/CPUs, though help recommends to set it to TWICE the number of CPUs in my experience that’s way to high.
- “Maximum allowed load” - very handy, again, set according to the HW you build on, leaving yourself some breathing space.
- “Prefix directory” - this is where the final cross-toolchain will land, called
- “Toolchain option” ->
- “Toolchain ID string” - shows up on
gcc --versionoutput as:
acme-gcc (crosstool-NG 1.14.1 - <ID_string>) 4.5.3. the author recommends to put there date or build number.
- “Tuple’s vendor string” - the default is
unknown. “target tuple” is the combo:
powerpc-e300c3-linux-gnu. this is the
VENDORbit of the tuple.
- “Tuple’s alias” - this one’s handy, it generates short-named symlinks
to the same tools right alongside the tools themselves. e.g. if you
set it to
powerpc-e300c3-linux-gnutuple you’ll end up with
- “Toolchain ID string” - shows up on
the rest - per your target and requirements.
if you’re even slightly adventurous in your configuration, you’ll witness a pile of failures before you make the darned thing build your cross-toolchain to your liking, most of them trivial and obvious, some - well documented. make sure to read:
$CT_INST/share/doc/crosstool-ng/ct-ng.1.14.1/B\ -\ Known\ issues.txt
then see “known issues” at the bottom of this document.
first step in troubleshooting failed build is to open
jump to the bottom and try to guess what went wrong. if it’s some failed
dependency that one of the packages’
./configure script notices, you may have
to drill down to the specific config log to figure out what exactly went
wrong and made
./configure say that some thing or other is “missing”.
per-tool/lib logs land under:
NOTE: CT sporadically fails to download packages from
sourceforge, even though they appear to be there. just peek at the log that gets created as:
look up which package CT tried to download and failed, copy the address and download it manually into:
CT will notice the packages already there and use them when you re-run it.
dumaare especially prone to this at this time of the year…
if still in the experimentation stage, you’ll want to make sure you:
[EXTRA]traces to see at a glance where CT fails. you’ll want “Paths and misc options” -> “Maximum log level to see” set to
enable “Paths and misc options” -> “Debug crosstool-NG” and then under it: “Save intermediate steps” (“gzip saved states” is up to you). rest assured your build WILL fail due to dependencies for a while before you nail it. unless you have this option on, when you’ll restart the build again after fixing issues, CT will do everything FROM SCRATCH. with this option on, after each tool/lib successfully built, CT will print (with tool/lib name instead of
<step-name>, of course):
Saving state to restart at step '<step-name>'...
and you’ll be able to restart from that specific LNG state by doing:
YOU WANT THIS OPTION ON UNTIL YOU FIGURE OUT YOUR PERFECT SET OF TOOL/LIB VERSIONS THAT WORKS. no, really, you do!
back up the tarballs that CT downloads in the process to the:
dir - they DO get wiped out in case of
ct-ng distcleanand CT WILL take ages to re-download the entire >200MB again! stash them someplace read- only, then specify this location under Paths and misc options” -> “Local tarballs directory”.
read the help messages of various options.
you prolly will want to build a fully static cross-toolchain to make sure
it’ll run on any reasonably similar system (e.g. fedoras/RHELs/CentOSes) and
can be “deployed” by a simple
untar without having to install any libs on
target or specify any paths to any
.so libs (or just dumped on a central
server exporting NFS to be mounted from the dev machines - though you’ll pay
dearly in build times for that stunt!). to build even a moderately useful
static cross-toolchain you’ll probably need to install at least:
sudo yum install glibc-static zlib-static libstdc++-static ncurses-static
you can wait for build to fail case-by-case until it forces you to install those, or just take my word for it and install them beforehand.
NOTE: recent versions of GDB require
libexpat.a, but for some reason, the
expatpackage on fedora does NOT provide a static lib, and unlike the above
*-staticpackages there’s NO
expat-staticpackage on fedora (but there’s an open bug in RH’s bugzilla, if that’s any consolation). although CT downloads
expattarball, it ONLY builds a static version for the target, and even then, during
native gdbphase which comes AFTER the
cross-gdbphase. Yann refuses to build the
libexpat.afor HOST for ideological reasons so, just
makethe same version of
expatas CT grabs (just reuse the same tar) and park the resultant
sanity-checking your cross-toolchain:
once you’ve managed to build your cross-toolchain start-to-finish without
failures, it’s HIGHLY RECOMMENDED that you do some sanity checks on it. tick
off: “Companion libraries” -> “Check the companion libraries builds” and
“Test suite” -> “GCC test suite” in
menuconfig, then re-start the build
from scratch. no need for
distclean, but do a
ct-ng build - restarting from
the middle using
<stage>+ will NOT pick up the changes in
the former option will run the tests during the build, so once the build is successfully over - you’re good. it also takes oh-shit-long (on the order of 90 minutes on a 4 x 3.1GHz machine with gobs of RAM) which is why you DON’T want to enable it by default…
the latter option just builds the GCC test suite, leaving it under:
this one is a bit of a beast: it’s really designed to TEST that a
cross-compiler works by running compiled test snippets on an actual target.
expect on the host, and passwordless ssh connection to
a target. IF your host is able to run the target binaries, like a 64-bit ->
32-bit cross-compiler for the same arch, you can just specify
a target. luckily, i THINK CT pre-configures the test suite by putting the
right strings into its
Makefile, so all you really need to do is:
sudo yum install dejagnu
then kick off the tests (note
DG_TOOLNAME is just
make DG_TOOLNAME=gcc DG_TARGET_HOSTNAME=<tgt_addy> DG_TARGET_USERNAME=<user>
make DG_TOOLNAME=gcc DG_TARGET_HOSTNAME=127.0.0.1 DG_TARGET_USERNAME=luser
luser is the username
dejagnu will use to connect, q.v. passwordless
above). this will first compile a bunch of compile-only GCC regression tests on
your host, then enter a loop where on each iteration it’ll compile a single
compile+run regression test,
scp it onto
<tgt_addy> (possibly prompting you
with yes/no if it’s the 1st ssh connection to that host, so don’t leave
unattended if you want results!), then it’ll ssh to
<tgt_addy> to run this
test, etc. finally, repeat for
make DG_TOOLNAME=g++ DG_TARGET_HOSTNAME=127.0.0.1 DG_TARGET_USERNAME=luser
the whole process took some 3.5 hours for gcc + 40min for g++.
there’s a pretty good chance that in the tests summary you WILL see SOME “unexpected failures” (likely the same several tests failing multiple times with different compilation flags, inflating the reported failures numbers). google up the circumstances on a per case basis: usually it boils down to bugs already reported upstream in RH/GNU/sourceware bugzillas, some of them already fixed upstream and possibly released in newer tools versions. MAKE SURE THAT THIS IS THE CASE and that it’s NOT your particular cross-toolchain that is broken, or you’ll find yourself debugging ghosts during actual development using the resultant cross-toolchain!
for the sake of comparison, on my cross-toolchains i got results in the
vicinity of >20 “unexpected failures” for
gcc, one or more “unexpected
g++, the latter often involving the linker.
- older glibc refuse to build with
makev3.82: “mixed implicit and normal rules”. just select “Companion tools” -> “Build some companion tools” and select
makewhich will build make v3.81. CT will use it happily from there on.
pplturned out to be a spiteful little thing, bearing a grudge against GMP (and CT users, apparently):
[INFO ] Installing PPL [ERROR] configure: error: Cannot find GMP version 4.1.3 or higher.
PPL v0.11* configure script will ignore pretty much ANY version of GMP you might select in CT config. just fall back to v0.10*:
ppl-0.10.2seem to work, but will probably necessitate downgrade of GCC from v4.6.x to v4.5.x.
ld,goldfor linkers will FAIL to build static toolchain:
[EXTRA] Building binutils [ERROR] g++: error: unrecognized option '-all-static'
libtool, so - yes, you’re screwed. stick to
ld-only for linker for static cross-toolchains.
cross-gdbfails to build, whining about
[EXTRA] Building cross-gdb [ERROR] configure: error: expat is missing or unusable
even though you have
expatRPMs installed - see above, fedora don’t bundle the static
libexpat.aanymore, so you’ll need to build it yourself.
[ALL ] /home/luser/tmp/x-tools/x86_64-acme-linux-gnu/lib/gcc/\ x86_64-acme-linux-gnu/4.5.3/../../../../x86_64-acme-linux-gnu/bin/ld:\ /home/luser/tmp/x-tools/x86_64-acme-linux-gnu/lib/gcc/\ x86_64-acme-linux-gnu/4.5.3/crtbeginT.o: relocation R_X86_64_32\ against `__DTOR_END__' can not be used when making a shared object;\ recompile with -fPIC [ALL ] /home/luser/tmp/x-tools/x86_64-acme-linux-gnu/lib/gcc/\ x86_64-acme-linux-gnu/4.5.3/crtbeginT.o: could not read symbols: Bad value [ALL ] collect2: ld returned 1 exit status [ERROR] make: *** [libinproctrace.so] Error 1 [ALL ] make: *** Waiting for unfinished jobs.... [ALL ] make: Leaving directory `/home/luser/tmp/crosstool-acme/.build/\ x86_64-acme-linux-gnu/build/build-gdb-gdbserver'
no known solution, but it’s been noted before. in the meanwhile - just drop