Configuring a Grok site under mod_wsgi

Getting the Grok application ready for mod_wsgi

WSGI applications use 'entry points' to let the WSGI server how to run the program. The entry point is usually a simple Python script that provides a function for calling the application and passing in the appropriate initialization file to the server. Some servers, like the paster server, need just the path to the .ini file, which is what we normally use to start up a Grok application. That doesn't mean there's no entry point script for paster. The entry point in fact is defined in the setup.py file that is created by grokproject when a new project is initialized. Take a look at the last few lines of the file:

[paste.app_factory]
main = grokcore.startup:application_factory
debug = grokcore.startup:debug_application_factory

The heading paste.app_factory tells the server where to find the factory functions for each section of the .ini file. In Grok, a general application factory function is defined in the grokcore.startup package, which is what paster uses to start applications.

However, mod_wsgi requires a path to the factory, which would be cumbersome to include in our configuration, because that would mean that it would need to point to a file inside the grokcore.startup egg, and since eggs include the version number, a simple update could crash our site if the old egg is removed. It would be better to have our own factory defined inside the application package.

Given that the factory code can be almost identical from project to project, it would be good to have it included automatically when we create the project, to avoid having to recreate the same script every time. Fortunately for us, Grok's use of buildout turns out to be very helpful in this case, since there is a buildout recipe available that creates the WSGI application factory for us.

The recipe is called collective.recipe.modwsgi. To use it, simply add a part to the buildout named, for example, wsgi_app. The recipe requires two parameters, the first being the eggs that have to be made available to the Python process that will run the app under WSGI and the second the path to the configuration file that will be used for the site. This last parameter value is the usual parts/etc/deploy.ini path we have been using for running the application under paster. That's it. Edit the buildout.cfg file parts list to look like this:

parts =
    eggbasket
    app
    i18n
    test
    mkdirs
    zpasswd
    zope_conf
    site_zcml
    zdaemon_conf
    wsgi_app
    deploy_ini
    debug_ini

Next, add the following section anywhere on the file:

[wsgi_app]
recipe = collective.recipe.modwsgi
eggs = ${app:eggs}
config-file = ${buildout:directory}/parts/etc/deploy.ini

There. Note that the eggs parameter simply points back to the main egg section defined at the start of the buildout, to avoid repetition.

When the buildout is run again, we'll find a parts/wsgi_app directory (or whichever name we used for the buildout part. Inside that directory, there will be a wsgi file that can be used as is by mod_wsgi to run the application.

Configuring an Apache site to use mod_wsgi

The last step is to add a site to the Apache server that uses mod_wsgi to serve our application. This is standard mod_wsgi configuration, we'll just add the path to the application factory that we created in the previous section.

To set up the virtual host, create a file in the /etc/apache2/sites-available directory and call it, for example, grok. Put the following in it, assuming your Grok application is at /home/cguardia/grok/hello:

WSGIPythonHome /usr
WSGIDaemonProcess grok user=cguardia group=cguardia threads=4 maximum-requests=10000

<VirtualHost *:80>
  ServerName wsgi.example.com
  WSGIScriptAlias /hello /home/cguardia/grok/hello/parts/wsgi_app/wsgi
  WSGIProcessGroup grok
  WSGIPassAuthorization On
  WSGIReloadMechanism Process
  SetEnv HTTP_X_VHM_HOST http://wsgi.example.com/hello
</VirtualHost>

This will run mod_wsgi in 'daemon' mode, which means it will launch a number of processes to run the configured WSGI application instead of using the apache process. If you are using virtualenv, the site-packages directory of the virtual Python used to run it needs to be passed in the WSGIPythonHome variable. To tell mod_wsgi which WSGI application to run, we use the WSGIScriptAlias directive and pass it the path to application factory that we created earlier.

Note that we assign a user and group to run the process. It is required that this user has access to the application directory.

WSGIDaemonProcess grok threads=4 maximum-requests=10000 python-eggs=/tmp/python-eggs

Running the application

Once the configuration is ready, we need to enable the site in Apache, since we just created it. This is only necessary the first time we run it:

$ sudo a2ensite grok

Then, we can start serving our application from Apache, simply by reloading the configuration for the server:

$ sudo /etc/init.d/apache2 reload

When you visit the site in a browser (http://wsgi.example.com/hello), you should see the Grok Admin UI. You should be able to log in using the admin login name and password (found in 'etc/users.zcml').

Adding a ZEO server

By default, mod_wsgi will use a single process to run the application. Since this configuration is intended for production use, it may be desirable to have a higher number of processes available to serve the application. The ZODB that Grok uses comes with a server named ZEO (Zope Enterprise Objects) that allows us to add as many processes to our configuration as our system permits, providing unlimited horizontal scalability. Typically, the recommended number of processes is one for each core in the system's processors. Let's set up a ZEO server and configure the Grok process to connect to it.

Once again, the easiest way to get ZEO running is to use an existing buildout recipe. This time we'll use one named zc:zodbrecipes. Add a zeo_server part to your buildout.cfg file, like this:

parts =
    eggbasket
    app
    i18n
    test
    mkdirs
    zpasswd
    zope_conf
    site_zcml
    zdaemon_conf
    zeo_server
    wsgi_app
    deploy_ini
    debug_ini

Next, add a zeo_server section like the following:

[zeo_server]
recipe = zc.zodbrecipes:server
zeo.conf =
    <zeo>
      address 8100
    </zeo>
    <blobstorage 1>
         blob-dir ${buildout:directory}/var/blobstorage
      <filestorage 1>
        path ${buildout:directory}/var/filestorage/Data.fs
      </filestorage>
    </blobstorage>
    <eventlog>
      level info
      <logfile>
        path ${buildout:directory}/parts/log/zeo.log
      </logfile>
    </eventlog>

This will add the ZEO server and configure it to listen on port 8100. The rest of the configuration is pretty much boilerplate, so just copy it to new projects when you need ZEO there.

Next, we need that the buildout add scripts for starting and stopping ZEO. This is easily accomplished by adding the ZODB3 egg to our app section:

[app]
recipe = zc.recipe.egg
eggs = gwsgi
   z3c.evalexception>=2.0
   Paste
   PasteScript
   PasteDeploy
   ZODB3

Configuring the ZEO client

Currently, the Grok application that we are using is working with the regular Zope server. To use ZEO, we need to change the configuration to connect to the server at port 8100. Fortunately, the required changes already come inside the regular zope.conf file that is created inside the grok project, so we only need to uncomment those lines. Uncomment the following lines inside the zope.conf.in file in the etc directory of your Grok project:

# Uncomment this if you want to connect to a ZEO server instead:
  <zeoclient>
    server localhost:8100
    storage 1
    # ZEO client cache, in bytes
    cache-size 20MB
    # Uncomment to have a persistent disk cache
    #client zeo1
  </zeoclient>

The important line in there is the one with the ZEO server address, in this case the same host as the Grok application and the port we defined in our ZEO configuration in the previous section.

Launching the ZEO server

After running the buildout again, we'll be ready to start the ZEO server in the background. To do that, we only have to run the server script that was automatically created for us. The name of the script is the same as the name of the part in the buildout where we configured the server:

$ bin/zeo_server start

Our Grok application is running. To stop it:

$ bin/zeo_server stop

Augmenting the number of processes

Recall that we mentioned earlier that mod_wsgi runs the application in a single process by default. To really take advantage of ZEO, we want to have more processes available. We need to make a small addition to our mod_wsgi Apache virtual host configuration for that. Change the WSGIDaemonProcess line near the top to look like this:

WSGIDaemonProcess grok user=cguardia group=cguardia processes=2 threads=4 maximum-requests=10000

In this example, we'll have two processes running, with four threads each. Using ZEO and mod_wsgi, we now have an escalable site.