Accelerated compilation

It is possible to accelerate the compilation stage by introducing one or more of the following strategies:

• using a filesystem in memory
• caching files generated by the preprocessor
• distributing compilation accros computers

Memory filesystem

If you have enough RAM, it is possible to reduce the updating time by choosing a filesystem in memory rather than on disk for the various compilation stages. The amount of memory to allocate depends of course on the available resources and what will need to be updated (system, ports).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Create memory file system
mdmfs -M -S -o async -s 4g md1 /mnt    # Create and mount the filesystem (4GB)

# Set one or several of the following environment variable so that
# the memory file system is used in the different stage of the compilation
export MAKEOBJDIRPREFIX=/mnt           # Set location for compiling the system
export WRKDIRPREFIX=/mnt               # Set location for compiling the ports
export CCACHE_DIR=/mnt/.ccache         # If using ccache for ports compilation
export DISTCC_DIR=/mnt/.distcc         # If using distcc for ports compilation
mkdir -p ${CCACHE_DIR} ${DISTCC_DIR}   # Ensure ccache/distcc directories exist

# make                                 # Compile

# Release ressource
umount /mnt                            # Unmount the filesystem
mdconfig -d -u 1                       # Release the allocated memory

Use of ccache and distcc

• Man pages: ccache (1), distcc (1)

Since FreeBSD 11.0, packages and distribution can be build using ccache by simply adding WITH_CCACHE_BUILD=yes respectively in /etc/make.conf and /etc/src.conf.

The compilation and installation is performed through:

For ccache:

1	# No selectable options

For distcc

1 2	[X] RFC2553 Enable IPv6 support [X] COMPILER_LINKS Create symlinks to distcc

For computers which participate in the distributed compilation, the distccd daemon must be enabled, this step is not necessary for the computer starting the compilation as it will be referred to with the special keyword localhost. Configuration is realised in the /etc/rc.conf file:

1 2	distccd_enable="YES" distccd_flags="--listen 192.168.1.5 --allow 192.168.1.0/24 --user distcc --daemon -P /var/run/distccd.pid"

Here are the possible invocations for the ccache and distcc programs and their combination when using the cc compiler:

In the case of the path modification, the path above must appear first in the PATH environment variable so that the corresponding wrappers are called. This PATH modification allows to avoid making modifications to scripts. However, for that, the compilers must be called without a specified path, and a joint use of ccache and distcc is not possible.

It is then possible to start the compilation by enabling the various accelerations using the variables USE_CCACHE and USE_DISTCC; either by placing them in the /etc/make.conf file or by setting them as parameter in make:

1
2

export DISTCC_HOSTS="--randomize localhost 192.168.1.10/2 192.168.2.25/4,lzo"
make -DUSE_DISTCC -DUSE_CCACHE

The DISTCC_HOSTS must be an environment variable. However, it is possible to use the $DISTCC_DIR/hosts file instead, but beware of the $DISTCC_DIR value, especially if it has been redefined.

The main parameters and options defining the computer contact list used to distribute the compilation are described below:

Base distribution

• Man pages: src.conf (5)

1
2
3
4
5
6

# Retrieve repository (without keeping git history)
git clone -b releng/12.2 --depth 1 \
    https://github.com/freebsd/freebsd-src /usr/src/

# Set fast-forward pull as recommanded when tracking FreeBSD repository
git -C /usr/src config pull.ff only

The /etc/src.conf file can control what will be included or excluded from the base distribution. This will be used to remove tools or hardware support for obsolete or not commonly used items, enable extra features such as NAND and DTrace support.

Use of ccache can be enabled to decrease build time, useful only if the base system is compiled regularly:

1
2

# Enable ccache build
WITH_CCACHE_BUILD=yes

Add support for debugging with DTrace:

1
2

# DTrace debugging
WITH_CTF=yes

Build extra components, like nand memory support:

1
2

# Extra commands
WITH_NAND=yes           # NAND support

Use retpoline to protect against CVE-2017-5715: Speculative execution vulnerabilities

1
2
3

# Spectre/Meltdown protection
WITH_KERNEL_RETPOLINE=yes
WITH_RETPOLINE=yes

Remove i386 support, as it’s not needed anymore on modern system:

1
2
3

# Don't build i386 library
#  (VirtualBox >= 6 dropped this requirement)
WITHOUT_LIB32=yes

Remove support for obsolete hardware or unused components:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# Remove the unused firewall: ipfilter, ipfw
#  => The selected firewall is: pf
WITHOUT_IPFILTER=yes
WITHOUT_IPFW=yes

# Remove the protocols: atm
WITHOUT_ATM=yes

# Remove the window drivers compatibility
WITHOUT_NDIS=yes

# Remove support for obsolete hardware (floppy disk)
WITHOUT_FLOPPY=yes
WITHOUT_APM=yes

# Remove obsolete applications
WITHOUT_CTM=yes
WITHOUT_LPR=yes       # Cups
WITHOUT_NIS=yes       # LDAP
WITHOUT_CVS=yes       # Git / Subversion
WITHOUT_RCS=yes       # Git / Subversion
WITHOUT_CCD=yes       # ZFS, geom_{mirror,stripe,raid3}
WITHOUT_TALK=yes
WITHOUT_TIMED=yes     # ntpd
WITHOUT_RCMDS=yes     # scp, rsync

# Not used that much
WITHOUT_LOCATE=yes
WITHOUT_RBOOTD=yes
WITHOUT_HYPERV=yes

Don’t include test infrastructure:

1
2

# No tests
WITHOUT_TESTS=yes

If building a server we don’t need support for wireless communication:

1
2
3

# Not really useful for a server
WITHOUT_WIRELESS=yes
WITHOUT_BLUETOOTH=yes

Now that the build options have been configured in /etc/rc.conf, we will describe the standard build process, but don’t forget to take a look at /usr/src/UPDATING for detailed instructions:

1
2
3
4
5
6
7
8
9
10

cd /usr/src                           # Change to source directory
make buildworld                       # Build toolchain and base distribution
make kernel KERNCONF=YOUR_KERNEL      # Build and install kernel

#<reboot in single user>
mergemaster -p                        # Apply pre-build configuration changes
make installworld                     # Install base distribution
make delete-old                       # Delete obsolete programs
mergemaster -i -F                     # Apply configuration changes
#<reboot>

1

cd /etc/mail && make && make install

To ensure that applications are using the last API, one could wish to rebuild all the packages

Ports and packages

• Man pages: make.conf (5)

First, you need to retrieve the necessary sources according to your release of FreeBSD. This is done with the git command:

1
2
3
4
5
6

# Retrieve repository (without keeping git history)
git clone -b main --depth 1 \
    https://github.com/freebsd/freebsd-ports /usr/ports/

# Set fast-forward pull as recommanded when tracking FreeBSD repository
git -C /usr/ports config pull.ff only

1
2

git -C /usr/ports diff --name-only "HEAD@{0}" "HEAD@{1}" |
    cut -d / -f 1-2 | uniq

To upgrade the packages, you will need to update the ports, build the index, fetch all the applications sources (this step is not necessary but will avoid problems if a source is missing in the middle of an upgrade), read the upgrade notice, create a snapshot (to have the possibility to rollback if something goes wrong) and finally upgrade:

1
2
3
4
5
6
7

git -C /usr/ports/ pull           # Update port sources
portsdb -uU                       # Build port index
portupgrade -a -F                 # Ensure that all the sources are available
pkg updating | less               # Read notice about potential problems
zfs snapshot -r system/pkg@tag    # Create a recursive zfs snapshot
                                  #  (tag can be: `date +%Y%m%d`)
portupgrade -a                    # Performe upgrade

If for some reason, the upgrade broke things, you can rollback to the previous stable state. Nevertheless, be careful because some applications once runned may have updated their data in an incompatible format with previous versions (ex: mysql_upgrade):

1

zfs rollback -r system/pkg@tag    # Rollback in case of problems

Clean up

After several upgrades, it is possible that little bugs in ports definition lead to residual files, without forgetting that shared libraries are not deleted but moved to /usr/local/lib/compat/pkg/ by portupgrade.

Be wise in removing files, you need to be sure that they are not needed, don’t blindly remove files returned by:

1 2 3 4 5

# Unused libraries libchk # installed from: /usr/ports/sysutils/libchk # Orphaned files find /usr/local/ -type f \ -exec sh -c ' pkg which -q "$0" > /dev/null || echo "$0" ' {} \;