Portable GNU-Linux binaries
The builds we distribute for GNU/Linux, including those included in exported "Linux Tarballs" and "Debian Linux Packages" are hopefully portable to other distributions. This page documents how we achieve that, and methods considered.
To compile a portable build, run scons portable=1.
I looked at how several Linux games I had were packaged/distributed.
- Almost all of them included separate binaries for x86 and for x64.
- Almost all included .sos for SDL/SDL2 and other SDL libraries if they used them, as well as libraries like GLEW
- A few of the games included libstdc++.so.X or libgcc_s or similiar
- A few included libogg, etc.
- Nobody seemed to have statically linked anything such as libc or other common libraries
- Steam includes heaps of libraries from Ubuntu
12.04that override many of the system ones, such as SDL
It's not necessary to statically link the binaries.
We can't require a recent version of glibc. For creating portable binaries I have seen people recommend building on an old distro such as CentOS 5. But using a too-ancient distro will result in requiring libraries like libstdc++5 (gcc 3.2-3.3.3 from 2002-2004) that some distros like Ubuntu don't bother to include by default (while Slackware does, but not libstdc++4).
An ELF binary includes fine-grained symbol version requirement tags for glibc and some other libraries, and these requirements don't get added unless you actually use a function or global which is annotated with such a requirement. So it is OK to compile against a recent glibc as long as you check the result and fix any problems. Unfortunately, this needs to also be done for any dynamic libraries that we distribute.
We link to glibc_compat.c and pass some magic flags to the linker to avoid dependencies on certain glibc functions - see below.
The required version of libstdc++.so depends on the version of g++ used (see the table here, item 4) and the features of that library which are actually used. These can be checked by looking at GLIBCXX in the objdump -p output; see below. New versions of libstdc++.so.X.Y can be used with any program compiled against libstdc++.so.X.Z with Z <= Y.
The last compatibility-breaking release was with GCC 3.4.0 (20040419).
- compile with an old version of GCC
- ensure that we aren't using any functions in libstdc++.so that are new (by checking GLIBCXX), and provide shims if we are
- statically link to libstdc++.so
- CXXLINKFLAGS += ['-static-libstdc++']
- include libstdc++.so in the packages and link to it with LD_LIBRARY_PATH
On 32-bit x86, statically linking libstdc++.so.6.0.21 adds about 1MB to the binary size (with debug=0), and 400KB after zipping. libstdc++.so itself zips to 500KB. Since unlump, relump, ohrrpgce-game and ohrrpgce-custom all depend on it, it could save a bit to ship libstdc++.so.
We went with option #2. Shims are provided in lib/stdc++compat.cpp, which is adapted from Mozilla. Also we compile C++ code with -D_GLIBCXX_USE_CXX11_ABI=0 which makes GCC avoid some newer parts of libstdc++.
This dependency can be avoided by adding CXXLINKFLAGS += ['-static-libgcc'] (when linking with gcc instead of fbc). It only adds 15KB to the binary!
Programs compiled by fbc will depend on libtinfo.so or libncurses.so, which is very unportable*.
We avoid this problem by not linking to these libraries (by using gcc instead of fbc to link objects; or you can pass -nodeflibs to fbc) and instead linking to lib/termcap_stub.c.
* libtinfo.so is a very common incompatibility between distros. ncurses can be compiled either as just libncurses, or as separate libtinfo and libncurses libraries, where libtinfo just has terminfo functions. Various distros compile it differently; Debian compiles as two libraries. If you get an error the workaround is to symlink libtinfo.so to libncurses.so. FB itself requires libtinfo if it exists, and otherwise falls back to libncurses. (This is logic in src/compiler/fbc.bas - well-intentioned but annoying).
Very commonly, a shell script is used to set LD_LIBRARY_PATH, a relative path for the linker for included libraries, although it is possible to set a link-time flag when compiling to look in the same directory as the executable (or some other relative directory) for shared libraries with "gcc -Wl,-rpath,'$ORIGIN'". Using a shell script however makes it easier for people to understand and override this behaviour.
I think we should include just SDL and SDL_mixer libraries (each about 450kB), but not libogg, etc.
However, this is not enough to hear MIDI music. Android builds also need to include MIDI instrument patches, and I think we found some relatively small ones for that? It might be possible to include patches, but prefer any that are already installed on the system, but I'm guessing this would require patching SDL_mixer.
What's described in this section is checked automatically (by check_lib_requirements()) when compiling with scons portable=1. It's printed at the bottom of the build output and looks like
>> ohrrpgce-custom requires glibc 2.14 (released 2011-06-01) and libs for gcc 3.4.0 (released before 2008-03-05)
To check a binary's dependencies, use objdump -p. E.g., for a x86 build:
> objdump -p ohrrpgce-game ohrrpgce-game: file format elf32-i386 Program Header: PHDR off 0x00000034 vaddr 0x08048034 paddr 0x08048034 align 2**2 filesz 0x00000100 memsz 0x00000100 flags r-x INTERP off 0x00000134 vaddr 0x08048134 paddr 0x08048134 align 2**0 filesz 0x00000013 memsz 0x00000013 flags r-- LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12 filesz 0x0020460a memsz 0x0020460a flags r-x LOAD off 0x0020460c vaddr 0x0824d60c paddr 0x0824d60c align 2**12 filesz 0x00008ccc memsz 0x000321e4 flags rw- DYNAMIC off 0x00204698 vaddr 0x0824d698 paddr 0x0824d698 align 2**2 filesz 0x00000160 memsz 0x00000160 flags rw- NOTE off 0x00000148 vaddr 0x08048148 paddr 0x08048148 align 2**2 filesz 0x00000044 memsz 0x00000044 flags r-- EH_FRAME off 0x002021ec vaddr 0x0824a1ec paddr 0x0824a1ec align 2**2 filesz 0x000004dc memsz 0x000004dc flags r-- STACK off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2 filesz 0x00000000 memsz 0x00000000 flags rwx Dynamic Section: NEEDED libncurses.so.5 NEEDED libtinfo.so.5 NEEDED libpthread.so.0 NEEDED libSDL-1.2.so.0 NEEDED libSDL_mixer-1.2.so.0 NEEDED libX11.so.6 NEEDED libXext.so.6 NEEDED libXpm.so.4 NEEDED libXrandr.so.2 NEEDED libXrender.so.1 NEEDED libstdc++.so.6 NEEDED libm.so.6 NEEDED libgcc_s.so.1 NEEDED libc.so.6 NEEDED libdl.so.2 INIT 0x0804c304 FINI 0x0823133c INIT_ARRAY 0x0824d60c INIT_ARRAYSZ 0x0000006c FINI_ARRAY 0x0824d678 FINI_ARRAYSZ 0x0000001c HASH 0x0804818c GNU_HASH 0x08048b1c STRTAB 0x0804a188 SYMTAB 0x08048bd8 STRSZ 0x000012a3 SYMENT 0x00000010 DEBUG 0x00000000 PLTGOT 0x0824d800 PLTRELSZ 0x00000a48 PLTREL 0x00000011 JMPREL 0x0804b8bc REL 0x0804b874 RELSZ 0x00000048 RELENT 0x00000008 VERNEED 0x0804b6e4 VERNEEDNUM 0x00000006 VERSYM 0x0804b42c Version References: required from libgcc_s.so.1: 0x0b792650 0x00 20 GCC_3.0 0x0d696910 0x00 15 GLIBC_2.0 required from libdl.so.2: 0x0d696910 0x00 18 GLIBC_2.0 0x0d696911 0x00 12 GLIBC_2.1 required from libm.so.6: 0x0d696911 0x00 19 GLIBC_2.1 0x0d696910 0x00 10 GLIBC_2.0 required from libstdc++.so.6: 0x056bafd3 0x00 14 CXXABI_1.3 0x0297f865 0x00 08 GLIBCXX_3.4.15 0x08922974 0x00 06 GLIBCXX_3.4 required from libc.so.6: 0x09691f73 0x00 17 GLIBC_2.1.3 0x0d696912 0x00 16 GLIBC_2.2 0x0d696913 0x00 11 GLIBC_2.3 0x0d696910 0x00 07 GLIBC_2.0 0x09691a73 0x00 05 GLIBC_2.2.3 0x0d696911 0x00 03 GLIBC_2.1 required from libpthread.so.0: 0x0d696912 0x00 13 GLIBC_2.2 0x0d696911 0x00 09 GLIBC_2.1 0x0d696910 0x00 04 GLIBC_2.0 0x09691972 0x00 02 GLIBC_2.3.2
See  for making sense of this. It seems this binary requires glibc 2.3.2+, which was released 2003-02-28. libstdc++.so.6 was first introduced in GCC 3.4.0 GLIBCXX_3.4.15 is GCC 4.6.0+, which was released 20110325. For example Ubuntu 11.10 defaults to installing gcc 4.6 (I don't know what ) but some conservative distros released in 2012 may not support this binary. I guess at present (2016) requiring a 2012 release of a linux distro is acceptable.
Comparing the above to a binary built on slackware64-current with a very recent glibc, the version requirements are identical except that libtinfo.so.5 isn't required and there are no GLIBC_* tags at all for libdl.
Finding and replacing a function requiring a recent library
Once you know that something compiled into a binary has an unwanted requirement, e.g. GLIBC_2.28, run objdump -T ohrrpgce-game | grep GLIBC_2.28 to find the cause.
Requirements for a recent glibc can be fixed by either removing all calls to the function, or wrapping it with a wrapper that calls a specific older version. See fcntl for an example - search for fcntl in SConscript and lib/glibc_compat.c.
Actually, instead of a wrapper, we could put an asm (".symver ...") line in a header and include it in any file calling that function. However this is not possible in general, because it might be called from libfb, or from C code generated by fbc.
Inspect the system library to find out what versioned symbols are available, to figure out the correct asm (".symver ...") line to add to lib/glibc_compat.c. For example:
> objdump -t /lib/lib.so.6 | grep fcntl ... 00101b70 g F .text 00000215 fcntl@@GLIBC_2.28 00101d90 g F .text 00000023 fcntl@GLIBC_2.0 ...
This shows a new fcntl was added in glibc 2.28 and the older one is available with asm (".symver fcntl, fcntl@GLIBC_2.0");
32-bit and 64-bit builds will probably need different '.symver' lines.
For more information, see here.