How can I compile and use my own glibc C standard library from source? (2023)

Last tested on Ubuntu 20.04 with glibc development version 2.33.9000 (seeglibc/version.h) on June 27, 2021.

How to download and compile glibc and run its benchmarks

You can get the glibc source code manually here: https://www.gnu.org/software/libc/sources.html:

git-Klon https://sourceware.org/git/glibc.git cd glibc git checkout master

Third-party mirror on GitHub: https://github.com/bminor/glibc/tree/master/benchtests

See also:

1. https://kazoo.ga/una-herramienta-sencilla-para-probar-el-rendimiento-de-malloc/

If you want to create glibc and its bench tests manually, do the following:

# IMPORTANT: Start AT THE SAME DIRECTORY LEVEL as the `glibc` source # directory, NOT inside the `glibc` source directory! In other words, if # you are in the correct directory, running `ls` will show the glibc source directory # (that you just cloned) in the directory you are in. mkdir -p glibc-build mkdir -p glibc -install cd glibc-build ../glibc/configure --prefix="$(realpath "../glibc-install)")" time make -j8 # compile with 8 threads ( works); On a fast laptop this takes around 3 minutes. time make install # (optional: install in the `glibc-install` directory you created) # Also create the bench tests (all inside the `glibc/benchtests` directory); # See the 'glibc/benchtests/Makefile' makefile for more build commands. time make bench-build -j8 # You now have this executable that you can use, for example, for malloc speed tests!: # ../glibc-build/benchtests/bench-malloc-thread # To compile **y run* * all glibc bank tests, flag: timestamp bank

References:

1. https://www.gnu.org/software/libc/manual/html_node/Configuración-y-compilación.html
2. https://kazoo.ga/una-herramienta-sencilla-para-probar-el-rendimiento-de-malloc/
3. https://github.com/f18m/malloc-benchmarks/blob/master/Makefile#L122-L129 - I learned a lot by studying this makefile target:

   $(glibc_install_dir)/lib/libc.so.6: @echo "Build GNU libc... go get a cup of coffee... this will take time!" mkdir -p $(glibc_build_dir) cd $(glibc_build_dir) && \ ../glibc/configure --prefix=$(glibc_install_dir) && \ make $(parallel_flags) && \ make install [ -x $(glibc_build_dir)/benchtests/bench -malloc-thread ] && echo "The GNU libc benchmarking utility is ready!" || echo "Could not find GNU benchmarking utility libc! Unable to collect benchmark results"

4. How can I compile and use my own standard glibc C library from source?

_{Keywords: how to build and run glibc and its bench tests, including malloc bench tests, from source; Compile glibc from source on Linux Ubuntu}

oMakefilewill exist in yoursbuild-glibcdirectory if theto set upThe script completes successfully.

If meanwhile everything seems to have gone wellto set upand not yetMakefile, then you probably missed a quirk:

when making oneto set upfor glibc you are normally expected to provide an alternative--Prefix, since the installation is in the default location (/usr/local) can paralyze the system. If you do not provide one, you must activate it--disable-sanity-checks.

If this is not the case, look for aconfig.logfile and read its contents.

Configuration 1: glibc without dedicated GCC

This setup might work and is fast as it doesn't rebuild the entire GCC toolchain, just glibc.

The only problem I have with this setup is that I haven't found a good way to use runtime objects likecrt1.o,to draw, zcrtn.oprovided by our glibc, and I'm using the host's one for now. This is mentioned at: https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location These objects do the initial configuration that glibc depends on, so I wouldn't be surprised if things are pretty bad, wonderful and incredibly subtle forms. See possible solutions below.

Compile glibc and install it locally:

git clone git://sourceware.org/git/glibc.git cd glibc git checkout glibc-2.32 mkdir build cd build export glibc_install="$(pwd)/install" ../configure --prefix "$glibc_install" make - j `nproc` hacer instalar -j `nproc`

Configuration 1: Check the build

test_glibc.c

#define _GNU_SOURCE #include <assert.h> #include <gnu/libc-version.h> #include <stdatomic.h> #include <stdio.h> #include <threads.h> atomic_int acnt; cont int; int f(void* thr_data) { for(int n = 0; n < 1000; ++n) { ++cnt; ++accent; } return 0; } int main(int argc, char **argv) { /* Check the basic library version. */ printf("gnu_get_libc_version() = %s\n", gnu_get_libc_version()); /* Exercise thrd_create from -pthread, * which is not present in glibc 2.27 on Ubuntu 18.04. * https://stackoverflow.com/questions/56810/how-to-start-threads-in-plain-c/52453291#52453291 */ thrd_t thr[10]; for ( int n = 0 ; n < 10 ; ++ n ) thrd_create ( & thr [ n ] , f , NULL ) ; for ( int n = 0 ; n < 10 ; ++n ) thrd_join ( thr [ n ] , NULL ) ; printf("The number of atoms is %u\n", acnt); printf("The non-atomic counter is %u\n", cnt); } }

Compile and run withprueba_glibc.sh:

#!/usr/bin/env bash set -eux gcc \ -L "${glibc_install}/lib" \ -I "${glibc_install}/include" \ -Wl,--rpath="${glibc_install}/lib " \ -Wl,--dynamic-linker="${glibc_install}/lib/ld-linux-x86-64.so.2" \ -std=c11 \ -o test_glibc.out \ -v \ test_glibc.c \ -philo\; ldd ./test_glibc.out ./test_glibc.out

Command adapted from https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location

The program generates what is expected:

gnu_get_libc_version() = 2.32 Atomic counter is 10000 Non-atomic counter is 8674

lddThe output confirms that thelddand the newly created libraries are used as expected:

+ ldd test_glibc.out linux-vdso.so.1 (0x00007ffe4bfd3000) libpthread.so.0 => /home/ciro/glibc/build/install/lib/libpthread.so.0 (0x00007fc12ed92000) libc.so.6 => /home/ciro/glibc/build/install/lib/libc.so.6 (0x00007fc12e9dc000) /home/ciro/glibc/build/install/lib/ld-linux-x86-64.so.2 => /lib64/ ld-linux-x86-64.so.2 (0x00007fc12f1b3000)

oCCGThe debug output of the build shows that my host's runtime objects were used, which is bad as mentioned, but I don't know how to fix this, for example, it contains:

COLLECT_GCC_OPTIONS=/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o

Configuration 1: Change glibc

Now we change glibc with:

diferencia --git a/nptl/thrd_create.c b/nptl/thrd_create.c índice 113ba0d93e..b00f088abb 100644 --- a/nptl/thrd_create.c +++ b/nptl/thrd_create.c @@ -16,11 + 16,14 @@ Licença junto com eine GNU-C-Bibliothek; caso contrario, consultar <http://www.gnu.org/licenses/>. */ +#include <stdio.h> + #include "thrd_priv.h" int thrd_create (thrd_t *thr, thrd_start_t func, void *arg) { + puts("gehackt"); _Static_assert (tamaño de (thr) == tamaño de (pthread_t), "tamaño de (thr) != tamaño de (pthread_t)");

Then rebuild and reinstall glibc and recompile and run our program again:

cd glibc/build make -j `nproc` make -j `nproc` install ./test_glibc.sh

and seehackedprinted a few times as expected.

This further confirms that we are in fact using the glibc we compiled and not the one on the host.

Tested on Ubuntu 20.10.

Configuration 1: Try to use the correct onecrt*objects

https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location suggests adding it--sysrootFor himCCGcommand but:

• doesn't actually change the objects to ours according to the protocols
• and causes the build to fail with/usr/bin/ld: could not find libgcc_s.so.1probably because thesystem rootis used for this GCC-provided object that we don't have in this sysroot because we only build glibc

At https://stackoverflow.com/a/66634184/895245 ZeZNiQ provides a solution that is probably correct in passing:

-nostartfiles

followed by all objects. You just need to extract the correct objects from the full command with it-nostartfilesand send them manually.

For example, on my AMD64 machine, the objects used were different than the 32-bit instruction, so this is a bit tricky.

Bibliography:

• How to change GCC's default search directory to crti.o?
• https://gcc.gnu.org/legacy-ml/gcc-help/2015-02/msg00016.html
• https://gcc.gnu.org/legacy-ml/gcc-help/2001-11/msg00029.html

Configuration 2: Crosstool NG primitive configuration

This is an alternative to configuration 1 and the most correct configuration I've gotten so far - as far as I can see, everything is correct, including C runtime objects likecrt1.o,to draw, zcrtn.o.

In this configuration, we will build a fully dedicated GCC toolchain using the desired glibc.

The only downside to this method is that it takes longer to compile. But I wouldn't risk a production setup with less.

crosstool-NG is a set of scripts that will download and compile everything for us from source, including gcc, glibc, and binutils.

Yes, the GCC build system is so bad that we need a separate project for it.

This configuration is simply not perfect as crosstool-NG supports creating executables without-WlFlags, which seems strange since we build GCC ourselves. But everything seems to be working, so this is just an inconvenience.

Get crosstool-NG, configure it and build it:

git-Klon https://github.com/crosstool-ng/crosstool-ng cd crosstool-ng git checkout a6580b8e8b55345a5a342b5bd96e42c83e640ac5 exportar CT_PREFIX="$(pwd)/.build/install" exportar PATH="/usr/lib/ccache: $ {RUTA}" ./bootstrap ./configure --enable-local make -j `nproc` ./ct-ng x86_64-unknown-linux-gnu ./ct-ng menuconfig env -u LD_LIBRARY_PATH hora ./ct-ng construir CT_JOBS=`nproc`

Construction takes around thirty minutes to two hours.

The only mandatory configuration option I can see is adapting to the host kernel version to use the correct kernel headers. Find your host's kernel version with:

uname -a

what it shows me:

4.15.0-34-generic

also reinmenu settingsYes:

• work system
  • linux version

then I choose:

4.14.71

This is the first version equal or previous. It must be older as the kernel is backward compatible.

Configuration 2: optional configurations

o.configwith which we generate./ct-ng x86_64-unknown-linux-gnudeadline:

CT_GLIBC_V_2_27=y

To change that, inmenu settingsDisapproval gesture:

• C library
• glibc version

save the.config, and continue with the build.

Or if you want to use your own glibc source, for example to use glibc from the latest git, do the following:

• Various paths and options.
  • Try the features marked as EXPERIMENTAL: set to true
• C library
  • glibc-Those
    • custom location: Say yes
    • custom location
      • custom place of origin: points to a directory containing your glibc source

where glibc was cloned as:

git clone git://sourceware.org/git/glibc.git cd glibc git checkout glibc-2.28

Configuration 2: test

After creating the toolchain you want, test it with:

#!/usr/bin/env bash set -eux install_dir="${CT_PREFIX}/x86_64-unknown-linux-gnu" PATH="${PATH}:${install_dir}/bin" \ x86_64-unknown-linux- gnu-gcc \ -Wl,--dynamic-linker="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib/ld-linux-x86-64.so.2" \ -Wl,--rpath ="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib" \ -v \ -o test_glibc.out \ test_glibc.c \ -pthread \ ; ldd test_glibc.out ./test_glibc.out

Everything seems to work as in configuration 1, except now the correct runtime objects are used:

COLLECT_GCC_OPTIONS=/home/ciro/crosstool-ng/.build/install/x86_64-unknown-linux-gnu/bin/../x86_64-unknown-linux-gnu/sysroot/usr/lib/../lib64/crt1. o

Configuration 2: An attempt to recompile glibc efficiently failed

This doesn't seem to be possible with crosstool-NG, as explained below.

If you are just remodeling;

env -u LD_LIBRARY_PATH hora ./ct-ng build CT_JOBS=`nproc`

So changes to the custom glibc source location will be honored, but it will build everything from scratch, making it unusable for iterative development.

If we do:

./ct-ng Listen text

gives a good overview of the build steps:

Verfügbare Build-Schritte in dieser Reihenfolge: - Companion_Tools_for_Build - Companion_Libs_for_Build - Binutils_for_Build - Companion_Tools_for_Host - Companion_Libs_for_Host - Binutils_for_Host - cc_Core_Pass_1 - Kernel_Headers - Libc_Start_Files - cc_Core_Pass_2 - Libc - cc_for_Build - cc_for_Host - libc_post_cc - Companion_Libs_for_Build " als Aktion, um genau diesen Schritt auszuführen. Verwenden Sie "+<paso>" as acción para ir a este paso Utilice "<paso>+" as acción para empezar desde este paso.

So we see that mainly glibc steps are intertwined with multiple gcc steps.libc_start_filesIt comes beforecc_core_pass_2, which is probably the most expensive step along withcc_core_pass_1.

To build only one step, you must first activate "Save Intermediate Steps"..configFirst build option:

• Various paths and options.
  • cross debugging tool
    • Save intermediate steps

and then you can try:

env -u LD_LIBRARY_PATH hora ./ct-ng libc+ -j`nproc`

but unfortunately the+required as mentioned in: https://github.com/crosstool-ng/crosstool-ng/issues/1033#issuecomment-424877536

Note, however, that a reboot as an intermediate step will restore the installation directory to the state it was in during this step. That means you have a newly built libc, but not a final compiler built with that libc (and therefore not a build library like libstdc++ either).

and basically it still makes the rebuild too slow to be viable for development and I don't see how to get past this without fixing Crosstool-NG.

also of thelibrarythe step didn't seem to copy the source backcustom place of origin, which makes this method even more useless.

Bono: stdlibc++

An added bonus if you're also interested in the C++ standard library: how can I edit and rebuild the source of the GCC libstdc++ C++ standard library?

Addendum to Ciro's answer/solution above https://stackoverflow.com/a/52454710/4726668:

@CiroSantilli Editing your answer returns "The proposed edit queue is full." The ldd script that you call on theprueba_glibc.shThe script points to the host's dynamic linker:/home/ciro/glibc/build/install/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fc12f1b3000). To fix this, inprueba_glibc.sh, changelddPro${glibc_install}/bin/ldd. To do this you need to add the buildcrt*.or also to the script:

-nostartfiles \ ${glibc_install}/lib/crti.o \ ${glibc_install}/lib/crtn.o \ ${glibc_install}/lib/crt1.o \

My work continues on my GNU/Linux i386/i686 machine (32 bit x86 Arch)prueba_glibc.sh:

#!/usr/bin/env bash set -eux gcc \ -L "${glibc_install}/lib" \ -I "${glibc_install}/include" \ -Wl,--rpath="${glibc_install}/lib " \ -Wl,--dynamic-linker="${glibc_install}/lib/ld-linux.so.2" \ -std=c11 \ -nostartfiles \ ${glibc_install}/lib/crti.o \ ${glibc_install }/lib/crtn.o \ ${glibc_install}/lib/crt1.o \ -o test_glibc.out \ -v \ test_glibc.c \ -pthread \ ; ${glibc_install}/bin/ldd ./test_glibc.out ./test_glibc.out