Use ccache, distcc, dmucs to accelerate builds

ccache

ccache is a compiler cache. It speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++.

Mac:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# install
$ brew install ccache
# where
$ brew ls ccache
$ which ccache
$ brew info ccache
# env
$ vi ~/.bash_profile
export PATH=/usr/local/opt/ccache/libexec:$PATH
# verify
$ source ~/.bash_profile
$ which clang
/usr/local/opt/ccache/libexec/clang

Ubuntu:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# install
$ sudo apt-get install ccache
# where
$ whereis ccache
$ which ccache
# env
$ vi ~/.bashrc
export PATH=/usr/lib/ccache:$PATH
# verify
$ source ~/.bashrc
$ which g++ gcc
/usr/lib/ccache/g++
/usr/lib/ccache/gcc

distcc on Ubuntu

distcc is a program to distribute compilation of C or C++ code across several machines on a network. distcc should always generate the same results as a local compile, is simple to install and use, and is often two or more times faster than a local compile.

Install:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# install
$ sudo apt-get install distcc
# optional
$ sudo apt-get install distccmon-gnome
$ distcc --help
# show ip
$ ifconfig | grep "inet " | grep -v 127.0.0.1
192.168.199.247
# config
$ sudo vi /etc/default/distcc
STARTDISTCC="true"
ALLOWEDNETS="192.168.0.0/16"
LISTENER=""
ZEROCONF="false"
# start server
$ sudo /etc/init.d/distcc start
# reboot & verify
$ ps aux | grep distccd

Usage:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# download
$ curl -R -O http://www.lua.org/ftp/lua-5.3.4.tar.gz
$ tar zxf lua-5.3.4.tar.gz
$ cd lua-5.3.4
# build
$ time make linux
# clean
$ make clean; ccache -C
# display distcc jobs, plz in another terminal
$ distccmon-text 2
# or, with gui
$ distccmon-gnome &
# build with distcc
$ export DISTCC_HOSTS="192.168.199.247 192.168.199.104"
$ time make -j12 CC="distcc gcc -std=gnu99" linux

Cost:

normal ccache distcc distcc + ccache
cost 4.069s 0.134s 1.301s 1.296s

P.S. distcc: 2 quad-core pc, -j12

normal make make -j4
cost 4.069s 1.403s

Use with ccache:

NOTE: This use of ccache is incompatible with use of distcc’s “pump” mode.

distcc on Mac

Install:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# install
$ brew install distcc
$ brew info distcc
$ distcc --help
$ distccd --help
# show ip
$ ifconfig | grep "inet " | grep -v 127.0.0.1
192.168.199.182
# config
$ vi /usr/local/Cellar/distcc/3.2rc1/homebrew.mxcl.distcc.plist
...
<key>ProgramArguments</key>
<array>
<string>/usr/local/opt/distcc/bin/distccd</string>
<string>--daemon</string>
<string>--no-detach</string>
<string>--allow=192.168.0.0/16</string>
</array>
...
# start server
$ brew services start distcc
# reboot & verify
$ ps aux | grep distccd

Usage:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# download
$ curl -R -O http://www.lua.org/ftp/lua-5.3.4.tar.gz
$ tar zxf lua-5.3.4.tar.gz
$ cd lua-5.3.4
# build
$ time make macosx
# clean
$ make clean; ccache -C
# display distcc jobs, plz in another terminal
$ distccmon-text 2
# build with distcc
$ export DISTCC_HOSTS="192.168.199.182 192.168.199.105"
$ vi src/Makefile
macosx:
$(MAKE) $(ALL) SYSCFLAGS="-DLUA_USE_MACOSX" SYSLIBS="-lreadline" CC="distcc cc"
$ time make -j12 macosx

Note: It is recommended that using distcc splits different platforms.

dmucs on Ubuntu

DMUCS is a system that allows a group of users to share a compilation farm. Each compilation request from each user will be sent to the fastest available machine, every time.

Install:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# install
$ sudo apt-get install dmucs
# bug fix for restart on boot
# dmucs: race condition in start scripts prevents loadavg starting
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798830
$ sudo vi /etc/init.d/dmucs
start)
if [ "$SERVER" = "yes" ]; then
...
if start_server ; then
...
if [ "$USE_SERVER" ]; then
...
if start_loadavg ; then
...

P.S. Unfortunately, this fix doesn’t work on my Ubuntu, although start_loadavg result is ok ==.

You could log the output of “/etc/init.d/dmucs” script, add this at the top:

1
2
exec 1>/tmp/init.log 2>&1
set -x

Server machine (e.g. 192.168.199.210):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# config
$ sudo vi /etc/default/dmucs
SERVER=yes
USE_SERVER=
# hosts-info
$ sudo vi /etc/dmucs.conf
# Format: machine number-of-cpus power-index
192.168.199.104 4 8
192.168.199.247 4 10
# start
$ sudo /etc/init.d/dmucs start
# verify restart on boot
$ sudo reboot
$ ps aux | grep dmucs
# display which hosts/cpus are available
$ monitor

Each host (e.g. 192.168.199.104 192.168.199.247):

1
2
3
4
5
6
7
8
9
10
11
12
# config
$ sudo vi /etc/default/dmucs
SERVER=no
USE_SERVER=192.168.199.210
# start
$ sudo /etc/init.d/dmucs start
# verify restart on boot
$ sudo reboot
$ ps aux | grep loadavg

Usage:

1
2
3
4
$ cd lua-5.3.4
# build
$ time make -j12 CC="gethost -s 192.168.199.210 distcc gcc -std=gnu99" linux

Issue: loadavg get wrong host on Ubuntu 14.04, then dmucs not recognize the hosts

Run loadavg -s 192.168.199.210 -D, see “Writing –>127.0.1.1 …<– to the server”.

Easily avoid this issue as follows:

1
2
3
4
5
6
7
8
9
# comment host "127.0.1.1"
$ sudo vi /etc/hosts
127.0.0.1 localhost
#127.0.1.1 ubuntu
...
# restart loadavg
$ sudo /etc/init.d/dmucs restart

However, it will affect dnsmasq at least.

Learn more, plz google “127.0.1.1”, “ubuntu dnsmasq 127.0.1.1”.

您可以投食小伊布 ,奖励一下哦~