Personal tools
You are here: Home Documentation Tutorials Introduction to zc.buildout Source, Binary and RPM experiments

Source, Binary and RPM experiments

Distributing your buildout as source or binary, experiments with the RPM format
Jim Fulton's tutorial for using buildout, originally given at DZUG 2007
Page 16 of 17.

Source vs Binary

  • Binary distributions are Python version and often platform specific

  • Platform-dependent distribution can reflect build-time setting not reflected in egg specification.

    • Unicode size
    • Library names and locations
  • Source distributions are more flexible

  • Binary eggs can go rotten when system libraries are upgraded


    Recently, I had to manually remove eggs from my shared eggs directory. I had installed an operating system upgrade that caused the names of open-ssl library files to change. Eggs build against the old libraries no-longer functioned.

RPM experiments

Initial work creating RPMs for deployment in our hosting environment:

  • Separation of software and configuration
  • Buildout used to create rpm containing software
  • Later, the installed buildout is used to set up specific processes
    • Run as root in offline mode
    • Uses network configuration server

Our philosophy is to separate software and configuration. We install software using RPMs. Later, we configure the use of the software using a centralized configuration database.

I'll briefly present the RPM building process below. This is interesting, in part, because it illustrates some interesting issues.

ZRS spec file (1/3)

%define python zpython
%define svn_url svn+ssh://
requires: zpython
Name: zrs15
Version: 1.5.1
Release: 1
Summary: Zope Replication Service

Copyright: ZVSL
Vendor: Zope Corporation
Packager: Zope Corporation <>
Buildroot: /tmp/buildroot
Prefix: /opt
Group: Applications/Database
AutoReqProv: no

Most of the options above are pretty run of the mill.

We specify the Python that we're going to use as a dependency. We build our Python RPMs so we can control what's in them. System packagers tend to be too creative for us.

Normally, RPM installs files in their run-time locations at build time. This is undesirable in a number of ways. I used the rpm build-root mechanism to allow files to be build in a temporary tree.

Because the build location is different than the final install location, paths written by the buildout, such as egg paths in scripts are wrong. There are a couple of ways to deal with this:

  • I could try to adjust the paths at build time,
  • I could try to adjust the paths at install time.

Adjusting the paths at build time means that the install locations can;'t be controlled at install time. It would also add complexity to all recipes that deal with paths. Adjusting the paths at install time simply requires rerunning some of the recipes to generate the paths.

To reinforce the decision to allow paths to be specified at install time, we've made the RPM relocatable using the prefix option.

ZRS spec file (2/3)


mkdir $RPM_BUILD_ROOT/opt
mkdir $RPM_BUILD_ROOT/etc
mkdir $RPM_BUILD_ROOT/etc/init.d
touch $RPM_BUILD_ROOT/etc/init.d/%{name}
svn export %{svn_url} $RPM_BUILD_ROOT/opt/%{name}
cd $RPM_BUILD_ROOT/opt/%{name}
%{python} -Uc rpm.cfg
bin/buildout -Uc rpm.cfg buildout:installed= \

I'm not an RPM expert and RPM experts would probably cringe to see my spec file. RPM specifies a number of build steps that I've collapsed into one.

  • The first few lines set up build root.
  • We export the buildout into the build root.
  • We run the buildout
    • The -U option is used mainly to avoid using a shared eggs directory
    • The -c option is used to specify an RPM-specific buildout file that installs just software, including recipe eggs that will be needed after installation for configuration.
    • We suppress creation of an .installed.cfg file
    • We specify a recipe for a special bootstrap part. The bootstrap part is a script that will adjust the paths in the buildout script after installation of the rpm.

ZRS spec file (3/3)

%{python} bin/bootstrap -Uc rpmpost.cfg
bin/buildout -Uc rpmpost.cfg \
   buildout:offline=true buildout:find-links= buildout:installed= \
   mercury:name=%{name} mercury:recipe=buildoutmercury
chmod -R -w .

chmod -R +w .
find . -name \*.pyc | xargs rm -f

%attr(-, root, root) /opt/%{name}
%attr(744, root, root) /etc/init.d/%{name}

We specify a post-installation script that:

  • Re-bootstraps the buildout using the special bootstrap script installed in the RPM.
  • Reruns the buildout:
    • Using a post-installation configuration that specified the parts whose paths need to be adjusted.
    • In offline mode because we don't want any network access or new software installed that isn't in the RPM.
    • Removing any find links. This is largely due to a specific detail of our configurations.
    • Suppressing the creation of .installed.cfg
    • Specifying information for installing a special script that reads our centralized configuration database to configure the application after the RPM is installed.

We have a pre-uninstall script that cleans up .pyc files.

We specify the files to be installed. This is just the buildout directory and a configuration script.