Supporting Clang in the CI20 bare-metal project

This post is part of the CI20 bare-metal project.

Today I added support for the LLVM toolchain (The C/C++ frontend, Clang, and the linker, lld), in the CI20 bare-metal project. It is now possible to build the CI20 bare-metal project using only LLVM tools.

That doesn’t mean GCC support is gone, though! You can now choose between the two toolchains by setting the TOOLCHAIN CMake variable to either ‘gcc’ or ‘clang' (for example, by running ccmake, or by passing it to the cmake command line). If you’re using GCC, you can set TOOLCHAIN_PREFIX; if you’re using Clang, you can set CLANG_PATH (to the bin directory containing clang and friends). This reflects the fact that gcc tends to be prefixed for different targets, whereas Clang just uses the same binary for all targets.

Adding the support wasn’t as hard as I had thought — massive respect to the llvm team for building something so compatible. Here are some notes. I’m going to try to make them interesting, but I’ve got to be honest: it’s about adding toolchain support to an operating system; there are limits as to how interesting things could possibly get. 

• It is significantly easier to build the LLVM toolchain than it is to build GCC — LLVM uses cmake, includes support for all architectures by default, and automatically includes tool support (clang and lld) if it’s in the right place. This is much nicer than the Makefiles, option-tastic configure scripts, and (if you wish to stay sane) third-party cross-compiler build tools of GCC.
• It is much harder to get information out of llvm tools. Want to know what target triples clang supports? Check the source! On the other hand, the defaults actually do what I want — e.g. the right flavour of MIPS is in there by default (when I figured out how to ask for it) and the linker doesn’t need any particular configuration to support everything. 
• CMake’s support for configuration is somewhat  rudimentary but works. Internally, user configuration is done by changing variables in the CMake cache — when CMakeLists.txt is evaluated, variables are substituted from the cache if they are present. This weird nomenclature threw me for a bit but it makes sense once you know the internal details.
• LLVM’s linker, lld, is a bit less forgiving than GNU ld. I ran into a somewhat opaque error in which lld was complaining that the section flags for the .text section were 0x4 but I wanted to add a file with flags of 0x6; turns out that this is because I had set my start.S file as executable (0x4, ‘x') but not as allocating space (0x2, ‘a’) and lld was using that object (the first one it saw) to determine section flags. GNU ld, on the other hand, did the sensible thing and added the ‘a’ flag by itself.

Building an LLVM toolchain for the CI20 bare-metal project

If you want to use clang/lld instead of gcc/ld, and you don’t have a suitable clang installed, you’ll need to compile it yourself. I followed the Clang getting started instructions (also see the LLVM getting started instructions) and made these changes:

• Downloaded lld: cd llvm/tools ; svn co http://llvm.org/svn/llvm-project/lld/trunk lld
• Set CMAKE_BUILD_TYPE to Release as I’m not interested in actually hacking on llvm
• Set CMAKE_INSTALL_PREFIX so as not to interfere with my system toolchain

After that it was a simple make -j4 and make install.

Building a GCC toolchain for the CI20 bare-metal project, 2017 edition (on a Mac)

This isn’t very fair as a comparison, because I’m using a case-insensitive file system. But if you are, too, and want to use GCC, here’s how:

• Install Crosstool-ng. Previously I’ve used Homebrew for this, but the Homebrew crosstool recipe is currently broken, so I cloned it from Github and installed it manually.
• Create a case-sensitive disk image (for building) and mount it somewhere.
• Create a case-sensitive disk image (for installing) and mount it somewhere. (Tip: hdiutil attach <dmg> -mountpoint <mountpoint> if you don’t want to run your toolchain from /Volumes)
• Create a build directory inside your build image and cd into it.
• ct-ng mips-unknown-elf
• make menuconfig
• Set prefix directory (Paths and Options) to your case-sensitive disk image.
• Set endianness (Target Options) to “Little endian”.
• Save and exit.
• ct-ng build
• Unmount (and later delete) the build disk image when done.

As a Mac-specific bonus, gcc, ld, ar, and ranlib (at least) appear to work fine even if they’re running from a case-insensitive filesystem. So if you like, you can unmount your installation image, mount it in /Volumes, and copy the installed files to the right spot (i.e. to the same directory structure as the previously-mounted image). You then don’t need to have any disk images mounted to run gcc.