Using and maintaining Gentoo Linux
Written by Kevin Korb
as a presentation for GOLUG
Presented on 2005-07-07
This document is available at http://www.sanitarium.net/golug/using_gentoo_linux.html
- What is Gentoo Linux?
Gentoo Linux is a distribution of the Linux operating system like RedHat, Debian, and Slackware are. Gentoo uses a package management system call portage. Portage was inspired by ports on *BSD however IMHO it is a more evolved and easier system to use. What makes portage different/better than other package management systems is that it builds its packages from source code instead of simply downloading pre-compiled binaries like rpm or apt-get do. Building from source allows for more flexible customizations and sometimes greater performance at the cost of longer install times.
- Why would someone choose Gentoo over some other distro?
- More customizable. Portage gives you total control over the features and dependencies of every package. This doesn't sound like much of a statement but as I demonstrate portage later it will become more clear.
- Core OS customizability. There are many core components of the OS that you can simply rip out and replace with something else. Cron is a good example as there are 3 completely different cron systems within portage: fcron, dcron, and vixie-cron. Email server software is the same way. You can choose between qmail, sendmail, postfix, exim, nullmailer, probably others that I don't know about. What makes this unique is that you are not "deviating from the norm" as there is no "norm" for these options. The choice is totally up to the user.
- Perpetual upgrades. Gentoo handles distro level upgrades very differently than other distros. There isn't even a version number for Gentoo itself. When you see that the current Gentoo is "Gentoo 2005.0" that is just the version of the install CD. Your system can be constantly upgraded through the portage system. There will never be a need to reinstall or to get a new version of Gentoo to get new versions of other packages.
- Non-x86 arch support. Since Gentoo is compiled during installation it is available for more archs than any other Linux distro. I have personally run Gentoo on x86, amd64, sparc32, and my XBox game console.
- Version control. You can control which version of each package you are running without causing dependency problems (usually). This (usually) allows you to choose between running an old version with security patches added or a newer version that doesn't need them.
- Security. Since you can control what option features and library dependencies are used by each package you can remove features and libraries that are not needed. Every line of code is a potential security hole or bug so why have code you don't need. All distros recommend that you remove packages you don't plan to use but they don't give you a way to remove a library package without breaking all the packages that depend on it.
- Security updates. Gentoo tends to be very fast to release security updates and it is even very easy to make your own updates if you are too impatient.
- Download speed. Source code tarballs are usually smaller than compiled binaries so usually you have less to download for Gentoo than with other distros.
- Runtime performance. This is a controversial benefit of Gentoo. Since portage is compiling all binaries from source code it is possible to compile the binaries specificly for your CPU. Depending on what you are doing this can make a huge performance difference or one that is so tiny you will never notice it. I put this benefit last as it very rarely exists at all and the people who claim that it is the most wonderful thing in the world usually don't know what they are doing and end up breaking their systems while trying to over-optimize it to squeeze out another .000002% performance. The only thing I have EVER noticed an improvement on is running lame on an AMD Athlon XP using the mfpmath=sse,387 switch which gives almost a 2x performance boost.
- Why would someone choose some other distro over Gentoo?
- Misconceptions about compiling. Whenever I say that Gentoo builds its packages from source people immediately think that means "configure --enable-this --disable-that (try 5 times to get all the dependencies in place) ; make ; make install". The reality is that portage exists to remove the need for compiling things by hand. When you install something on a Gentoo system portage does the compiling NOT you. You will not have to type configure or make. The entire purpose of portage is to create a package management system that is so flexible there will be no need for the use to compile things by hand EVER.
- Compile times. It takes a LOT longer to compile from source than it does to simply install a pre-compiled binary package. A full Gentoo install from source on a modern system can take as much as 12 hours (of compiling not typing). There are generic binaries available to you that will significantly speed up installs however that removes much of the customization capabilities of portage.
- 3rd party support. Gentoo is considered to be a "fringe distro" by most companies that support Linux. While the Gentoo support community is wonderful it is difficult if not impossible to get support from companies that make Linux software. However, the Gentoo community is very good at dealing with this. VMWare is a good example. It is released as an RPM package which portage downloads, rips apart, and installs it in the normal Gentoo way.
- Manual configuration work. While many GUIs and automation tools do exist Gentoo does have more manual maintenance needs than some other distros. This tends to scare Linux newbies away from Gentoo but it is also what Linux experts tend to want. There are some packages that require manual work after an install or upgrade. There is always a message during the install that explains exactly what needs to be done.
- Non-standard setup. There are a few aspects of Gentoo that are very non-standard. This makes them different from all other distros. This tends to scare Linux experts away because they think it is strange or wrong. A perfect example of this would be Gentoo's startup scripts and use of named instead of numbered runlevels.
- Sudden system changes. Because of Gentoo's perpetual upgrades there are occasionally updates that look very simple but end up changing something important. A good example of this would be the recent change to the baselayout package which completely changed the format of the network configuration file. This scares some people away as it gives the impression that Gentoo is not stable. This is where a Gentoo user must be a bit vigilant. When updates come out you sometimes need to question what an update will do to you before you actually install it. After a while you will get the feel for Gentoo and you will know which packages can affect your system (there aren't many) and which ones don't. There is a changelog for each package so it is usually easy to determine what is changing. There is also a config file management system to help you merge in new config files without losing old ones.
- Configuration files in /etc
- /etc/rc.conf: This is the config file for system level settings that don't have their own config file. Here is a list of the options that go in here as well as my personal choices for them.
- UNICODE: Set this if you require unicode support on your console. You must also select a console font and a keymap that support it. I always leave this on "no".
- EDITOR: This is what you want the default editor on the system to be if a user doesn't have $EDITOR set and they run something like crontab -e. The default is nano but I always change it to vim.
- DISPLAYMANAGER: This is which graphical login program you wish to use if X is set to start at bootup. I use xdm because I do not install Gnome or KDE.
- XSESSION: This tells the system what default window manager config file to use if a user logs in and has no ~/.xsession file. I do not bother with this since I have my own config file in my home directory.
- My rc.conf is available at http://www.sanitarium.net/unix_stuff/config_files/rc.conf.txt if anyone wants to see the entire file.
- /etc/make.globals: This file contains global settings for portage. You should not make changes to this file directly but instead override the settings you don't like in /etc/make.conf. This file is a good reference for some of the settings that are not normally in /etc/make.conf but can be put there.
- Bootup scripts
- Network configuration
- The network configuration file is /etc/conf.d/net. Here is a simple example from one of my boxes:
config_eth0="192.168.100.13/24"
routes_eth0="default via 192.168.100.1"
- Here is a more complex example that includes multiple NICs, bonding, and an alias:
config_eth0="192.168.100.202/24"
config_eth1="192.168.100.202/24"
config_bond0=(
"192.168.100.202/24"
"192.168.100.201/24"
)
slaves_bond0="eth0 eth1"
depend_br0() {
need net.eth0 net.eth1
}
routes_bond0="default via 192.168.100.1"
- Here is another more complex example that includes another alias and a static route:
config_eth0=(
"69.5.6.163/24"
"192.168.42.163/24"
)
routes_eth0=(
"default via 69.5.6.1"
"-net 69.5.0.0/19 dev eth0"
)
- Main portage configuration
- The main portage config file is /etc/make.conf. Most of the files that will be talked about later are used to override this file for specific packages. Here are the available settings that can go in /etc/make.conf...
- USE: This is what Gentoo refers to as the USE flags. These are what optional features are used when compiling packages within portage. For example, if gtk is in the USE variable then programs would be configured with --with-gtk. Here are my USE flag settings:
On my server: USE="-* 3dnow 3dnowex X Xaw3d a52 aac aalib acl apache1 berkdb bitmap-fonts bzlib cdb cddb cdinstall crypt cups curl djbfft encode fbcon fbdev font-server foomaticdb fortran fpx gd gdbm gif graphviz innodb jbig jpeg libgd lzo lzw lzw-tiff mailbox maildir mbox mha mmx mng mp3 mpeg mysql mysqli ncurses nodrm nptl offensive oggvorbis opengl pam perl png ppds procmail python qmail rage128 readline roundrobin rtc samba skey slang sox spamassassin ssl threads tiff truetype truetype-fonts type1-fonts userlocales withsamplescripts wmf xpm xv zlib"
On my desktop: USE="3dnow 3dnowext 7zip Xaw3d a52 aac acl audiofile bzip2 cdb cdda cddb cdparanoia dga divx4linux djbfft dts dvb dvd dvdr dvdread escreen fbcon fbdev ffmpeg font-server fpx freetype gimp gimpprint gkrellm graphviz jbig lzo lzw lzw-tiff matroska matrox mbox mjpeg mmx mmxext mng moznocompose moznoirc moznomail moznoxft mozsvg mpeg4 mplayer net network nocd nodrm noplugin nptl offensive ppds real rtc skey sndfile sox src sse theora threads timidity transcode type1 uptimed userlocales v4l v4l2 vcd vcdimager vidix wmf xanim xpm xscreensaver xvid xvmc yv12 -arts -cscope -esd -gnome -gpm -ipv6 -kde -mysql -nls -xml -xml2"
- Note that the programs ufed and profuse are both editors for the USE variable and they explain what each possible parameter does.
- CHOST: This is just the compiler string that is used. Normally it will be "i686-pc-linux-gnu" however you may have to change it if you are on a different arch.
- CFLAGS: These are the parameters to pass to gcc during compiles. This is the most controversial part of Gentoo. This is where you can over-optimize and probably break the box or you can make it rock solid. If you put something in here other than the recommended settings you had better know what it does. A good example is -ffast-math which works fine on some packages but breaks others completely. Here are my current CFLAGS:
On my server: CFLAGS="-mcpu=athlon-tbird -march=athlon-tbird -O2 -pipe -fstack-protector -fomit-frame-pointer -mmmx -m3dnow"
On my desktop: CFLAGS="-mcpu=athlon-xp -march=athlon-xp -O2 -pipe -fomit-frame-pointer -mmmx -m3dnow -msse -mfpmath=sse,387"
- CXXFLAGS: This is like CFLAGS except it is for c++. I do not know of any reason to set it to anything other than $CFLAGS.
- ACCEPT_KEYWORDS: You almost certainly want to leave this line commented out. Its purpose is to unmask all of the unstable packages so you can run the absolute latest stuff whether it works or not which it probably won't.
- PORTAGE_TMPDIR: This is where portage will actually do its compiling and package building. I usually leave it set to /var/tmp which is actually a symlink to /usr/tmp which has more disk space.
- PORTDIR: This is where portage is. I see no reason to change it from /usr/portage however if you change any of the other lines that reference ${PORTDIR} you will need to uncomment this line so the variable is set.
- DISTDIR: This is where portage downloads source tarballs before compiling packages.
- PKGDIR: This is where portage puts binary packages (if you tell it to make them) after it compiles them. I always set it to ${PORTDIR}/packages/[hostname] that way if 2 boxes have the same settings I can just symlink their directories together and use the same binaries on both instead of compiling the same thing twice.
- PORT_LOGDIR: This is an option to log the entire output of every compile. It will eat your disk faster than you think so I suggest you leave it commented out unless you have a specific need for it.
- PORTDIR_OVERLAY: This is a directory where you can put your own ebuild files. Any file placed in here will override a file of the same path/name within /usr/portage. If you choose something like /usr/portage/local like I do then you will have to exclude that from the rsync later in this file or you will lose your ebuilds.
- There are a bunch of options in the fetching and syncing sections that I am not going to cover because they are obvious, well documented, and I have never found a reason to change any of them.
- MAKEOPTS: These are parameters passed to the make command during compiles. Normally it is set to "-jn" where n is the number of CPUs you have +1. If you are short on RAM then you should probably set it to -j1 but YMMV.
- PORTAGE_NICENESS: You can have portage always run "nice" when it is compiling things.
- AUTOCLEAN: You can change this to no if you don't want portage to remove the old version of a package after an upgrade but of course usually the upgrade overwrites most if not all of the old package anyways.
- FEATURES: This is a list of portage features that you want to have enabled. There are a list of them in the config file and a description of what each does. I normally use "sandbox usersandbox notitles userpriv". I also use buildpkg on systems that share binary packages with other systems.
- RSYNC_EXCLUDEFROM: This is a text file containing a list of things within $PORTDIR that you don't want rsync to touch. This is where you list your overlay dir so you don't lose your custom ebuild files.
- Here are my make.conf files: http://www.sanitarium.net/unix_stuff/config_files/make.conf.asylum.txt http://www.sanitarium.net/unix_stuff/config_files/make.conf.dementia.txt
- Using portage
- Updating portage: There are actually 3 different ways to do this. The most basic is to just run "emerge sync". That will use rsync to download the latest version of the portage tree. If you are unable to use rsync because of a firewall you can use "emerge webrsync" instead which uses port 80 instead. There is also a nice package called esearch which you can install. It includes a program called esync. If you run "esync -v" you will get the same results as "emerge search" but esync will also give you a list of completely new ebuild files from all packages instead of just installed packages. Esync will then build an index of the available packages so you can use esearch later for faster indexed searches.
- Finding out what is new in portage: After you update the portage tree you probably want to know what upgrades are available for the packages already installed on your system. The best way to do this is to run "emerge --newuse -uvDp world". That will list all new versions even if they are dependencies and it will list packages that have had USE flag changes. I have written a that does all both of these steps at once. You can download it from: http://www.sanitarium.net/unix_stuff/wn.txt Be sure to read the comment at the top as it has some stuff hard coded for my network.
- Finding out what is new in a package: If you want to see the changelog for a package that has an update available you can run "emerge --nodeps -uvpl [package]"
- Installing packages: Installing a package is usually as simple as running "emerge [package]". If you want to install a specific version of a package you can run "emerge =[package]-[version]". I always suggest using the -vp parameter on emrege first so you can see what it is planning to do before allowing it to actually do it.
- Upgrading packages: Package upgrades work exactly like installs except that you add a -u to the emerge command line. Gentoo is actually pretty smart and will do an upgrade if you leave out the -u on a package that is already installed but it will not reinstall a package if there is no upgrade and -u is used.
- Uninstalling packages: To uninstall a package just run "emerge unmerge [package]" Note that this does NOT check dependencies. After it is done you should run an "emerge -uvDp world" again to see if something is requiring that package. You also might want to use revdep-rebuild (documented below) to check for things linked against it that didn't bother to register the dependency although this is somewhat rare.
- Searching for packages: To search for a package run "emerge search "keyword". This searches for packages that have the keyword in the name. If you want to search within the package descriptions instead use "emerge -S keyword". This search process is very slow but you can speed it up by using either of the indexed searching programs esearch or eix but both must be re-indexed after a sync.
- Listing available versions of a package: After you install gentoolkit there is a nice program called etcat. If you run "etcat versions [package]" it will show you all available versions of the package, which is installed, and which are masked.
- Finding unneeded packages: The emerge command has an option to have it tell you the packages that it doesn't think you need. These are usually packages that were installed as dependencies for something that is no longer installed. You can get the list with "emerge depclean -vp" (note that you can ignore the ugly warning messages as long as you are using the -p).
- Finding broken packages: Gentoo comes with a program that will scan the system for binaries that have broken dependencies. It will also re-merge the associated packages by default however I usually disable that with a -p so I can see what it plans to do first. To check for broken binaries run "revdep-rebuild -- -vp". That will give you a list of the packages that contain broken binaries. Usually re-merging those packages is all it takes however if you just removed a library package then you probably want to re-merge it instead. You can also use "revdep-rebuild --soname [shared library file] -- -vp" to find out what packages are actually using a library.
- Rebuilding ALL packages: If for some reason you feel the need to recompile EVERYTHING on the system it can be done with 1 command. This is normally only used when you are making a significant change to the CFLAGS setting. To recompile everything on the system run "emerge -euvDbk world". I strongly suggest using the -b and -k parameters for building and using binary packages because if you don't and there is a problem half way through you get to start over from the beginning.
- Finding out which package a file belongs to: You can do this with "equery b [file]". If you install epm you can also do it the RedHat way with "epm -qf [file]".
- Build a binary package from an already installed package: Sometimes you will want to make a backup of the binaries that you are running before doing an upgrade just in case you need to downgrade back to the old version in a hurry. You can do this by running "quickpkg [package]" Then if you need to downgrade back to that version just run "emerge -uK =[package]-[version]".
- Merge in new config files: There are 2 choices on this one. The "standard" tool is called etc-update. It takes you through new config files 1 at a time and asks what you want to do with them. I prefer to use the dispatch-conf tool which does the same thing but keeps backup copies in /etc/config-archive.
- How to share a common portage tree between multiple boxes: I have 7 Gentoo boxes at home and even more at work. I don't want to have to do "emerge sync" on all of them and I don't want to have to fetch the source packages on each of them. My solution is to NFS mount /usr/portage on all of my boxes from my file server. If you look at my wn script above you will see that it checks the hostname and only runs the sync portion if it is running on my file server named asylum. Since the source tarballs are downloaded to /usr/portage/distfiles all of my systems can share them too. If 2 systems use the same settings I can build binary packages on one (emerge -b) and use the same binaries on the others (emerge -K) to save compile time. I prefer eix over esearch for searching because it uses a simple index file that I can symlink into my NFS mount and share on all my systems. That can be done with "mv /var/cache/eix /usr/portage/local/" then on each box do "ln -s /usr/portage/local/eix /var/cache/"
- How to verify the files provided by an installed package: You can have portage verify the files that were installed with a package to see if they have been modified since the package was installed. You can do it with the command "equery check [package]" or you can use "epm -y [package]" which IMHO has a nicer output format.
- How to determine what USE flags were used on an installed package: If you need to know what USE flags were in effect when you installed a package you can find out by running "equery uses [package]".
- Package specific portage settings
You can configure USE flags for individual packages in the file /etc/portage/package.use. This file overrides the system level settings in /etc/make.conf. Here is an example of mine with some comments:
app-admin/sudo -pam skey # I like skey instead of passwords
app-crypt/gnupg -X # I don't need a gpg GUI
app-editors/vim -gpm -perl -python # These are just bloat features of vim IMHO
app-office/openoffice -java # I don't like java much either
mail-client/pine -ssl # I don't use pine's IMAP so I don't need its SSL
mail-mta/qmail -ssl # I don't use qmail's popper so I don't need its SSL
media-gfx/imagemagick jbig lcms wmf xml2 # I want imagemagick to support everything
media-sound/lame -gtk # I don't need lame's GUI
media-video/mplayer -truetype -gtk -gtk2 # I don't need mplayer's GUI
media-video/vcdimager xml2 # vcdimager is kind of useless without XML
net-analyzer/nmap -gtk # I don't need nmap's GUI
net-irc/bitchx -gtk # I don't like bitchx's GUI
net-irc/unrealircd -ssl # I don't need SSL on my ircd
net-misc/openssh -pam skey # I like skey instead of passwords
net-misc/vnc -java # I don't do java
net-p2p/mhxd exec # I needed exec support for some neat stuff in mhxd
sys-apps/lshw gtk # lshw-gtk looks cool
sys-boot/grub static # Might someday need grub to be static
x11-libs/qt -mysql # wtf would qt need to be linked against libmysql?
x11-libs/wxGTK gtk2 # has to have one of the GUI options
x11-misc/xlockmore -gtk -motif # Don't want GUIs just screensavers
[un]masking packages within portage
- If there is a version of a package that you don't want installed you can mask it in /etc/portage/package.mask. Once you do this the masked version will not show up as an available upgrade choice during an emerge. A common usage of this would be to use apache-1.3.x instead of apache-2.0.x. Here is my package.mask file with some comments:
=dev-db/mysql-4.0.23-r1 # This was a bad release of mysql though I don't remember why
=dev-perl/gtk-perl-0.7009-r1 # This was a bad release of gtk-perl though I don't remember why
=media-libs/libmatroska-0.7.5 # Not compatible with mkvtoolsnix package
=media-libs/libmatroska-0.7.6 # Not compatible with mkvtoolsnix package
>x11-wm/fvwm-2.0 # I use fvwm-1 not 2
>net-www/apache-2.0 # I use apache-1.3.x not apache-2
net-misc/dhcpcd # I don't use dhcp and I don't want it in the "system" alias
sys-apps/modutils # This is depreciated but something kept trying to install it
- There are 2 different ways to unmask a package depending on how it is masked. If it shows up in etcat versions with an M then it can be unmasked in /etc/portage/package.unmask which has the same syntax as /etc/portage/package.mask. If etcat versions also shows a ~ then it is masked by keyword. The file for that is /etc/portage/package.keywords. Here is an example of mine with some comments:
dev-db/mtop ~x86 # I sometimes use mtop on my mysql server and it seems to work.
>media-video/mpgtx-1.0 ~x86 # I sometimes use this to manipulate mpeg files and it seems to work.
app-office/plan ~x86 # I use plan as my calendaring software and it seems to work.
Suggested packages to merge
- app-admin/realpath: determines the full path to a file. Useful in many scripts
- app-editors/e3: a very tiny editor written in assembly that is handy in emergencies
- app-editors/hexedit: always nice to have a hex editor on hand
- app-editors/vim: every unix box should have a vi
- app-misc/screen: if you don't know what screen is you should definately try it and see
- app-portage/eix: an indexed portage searcher
- app-portage/epm: command line compatible with RedHat's rpm
- app-portage/esearch: like eix but has a nice display of what new ebuilds were added after a sync
- app-portage/g-cpan: auto-creates and installs ebuilds for Perl CPAN modules
- app-portage/genlop: Gentoo portage log parser. Gives you history of previous installs
- app-portage/gentoolkit: Various useful utilities such as etcat and equery
- app-portage/splat: like genlop but also has the neat -vscp world params which give a summary of the entire history of your box
- app-portage/ufed: USE flag editor (ncurses based) Profuse is another choice.
- app-shells/tcsh: always good to have a c-shell around
- net-analyzer/mtr: a traceroute/ping in one
- net-analyzer/tcpdump: the most simple sniffer (but still very powerful)
- net-analyzer/traceroute: traceroute
- net-dialup/minicom: serial terminal program
- net-dns/bind-tools: This gives you dig and nslookup without named
- net-fs/nfs-utils: If you plan to do NFS
- net-ftp/ftp: plain old ftp
- net-misc/telnet-bsd: plain old telnet
- net-misc/whois: a nice whois that auto-detects which whois server to ask based on the query
- sci-calculators/rpc: An ncurses based calculator using RPN notation.
- sci-calculators/units: a cool console units converter
- sci-mathematics/calc: a simple console calculator (for people that don't like RPN)
- sys-apps/busybox: a recovery shell
- sys-apps/eject: ejects (and unmounts) removable media
- sys-apps/hdparm: tests and sets parameters on hard drives
- sys-apps/lshw: a nice hardware detection program
- sys-apps/pciutils: gives lspci
- sys-apps/smartmontools: checks SMART data on hard drives
- sys-fs/raidtools: manipulates software RAID arrays (if needed)
- sys-fs/squashfs-tools: tools for the very cool compressed squashfs filesystem
- sys-process/htop: like top but nicer
- sys-process/lsof: list open file (descriptors)
- www-client/lynx: console web browser
Creating your own .ebuild files for portage
This is actually easier than it sounds. If you can write a bash script to unpack, configure, compile, and install a program within a chroot jail then you can do this. Portage even provides a bunch of commands to make things easier. Usually when I have done this I have been making slight modifications to existing ebuilds which is really easy to do. After you do that a couple of times (even I have only had to do it a couple of times) you should be able to write your own from scratch or using an existing file as a template. I have published several ebuild files that I have created or modified here: http://www.sanitarium.net/unix_stuff/gentoo-ebuilds/. Every question I have ever had was answered in either http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1 or http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=3.
Helpful web sites about Gentoo Linux