Personal tools
You are here: Home Documentation How-Tos Writing tests, discovering and running them with z3c.testsetup for dummies

Writing tests, discovering and running them with z3c.testsetup for dummies

Warning: This item is marked as outdated.

This How-to applies to: 0.12, 0.11, 0.10
This How-to is intended for: Developer

This How-to shows you how to get started with writing tests for your Grok project and then use z3c.testsetup to discover the tests.
Author: Peter Bengtsson

Purpose

Show you how to use z3c.testsetup to run tests on your Grok application.

Prerequisities

A grok project where you've got started on that doesn't yet have any tests. That and an Internet connection. To make this How-To easy, you can install a blank new project called Sample.:

$ cd /tmp
$ grokproject Sample
$ cd Sample

Step by step

The first thing to do is to add z3c.testsetup to your project buildout. That's best done by editing the project's setup.py file so that it includes 'z3c.testsetup':

install_requires=['setuptools',
                  'grok',
                  'z3c.testsetup'
                  ],

Then when you've done that re-run buildout so that the new dependency is installed:

$ ./bin/buildout

Test that the project's testrunner works when you run ./bin/test. You should see something like this:

$ ./bin/test
Running tests at level 1
Total: 0 tests, 0 failures, 0 errors in 0.000 seconds.

Now you're ready to start adding your tests. There are several different kinds of tests and we'll scratch the surface of each one. The tests are either doctest or python and the difference is that you either write them in pure python or as a string that you make the doc string of your classes or a separate file. z3c.testsetup is about discovering tests, not writing them or running them.

Let's write a very simple unit test for the app.py that has been created.:

$ cd src/sample
$ mkdir app_tests; cd app_tests
$ touch __init__.py
$ emacs test_app.py

Notice the importance of creating a __init__.py file in that new directory. Here's some example code that is really silly but at least proves that the test is discovered works:

"""
Do a Python test on the app.
:Test-Layer: python
"""
import unittest
from sample.app import Sample

class SimpleSampleTest(unittest.TestCase):
    """ Test the Sample application. """

    def test1(self):
        """ test that something works """
        grokapp = Sample()
        self.assertEqual(list(grokapp.keys()), [])

The next thing is to help z3c.testsetup find this test. That's achieved by creating a file called tests.py in the 'src/sample' directory with the following content:

import z3c.testsetup
test_suite = z3c.testsetup.register_all_tests('sample')

That's it! How cool is that?! I love the Just Works'ism of just writing "register_all_tests('sample')" and it does it.

Let's now crack on with a "doc test". The best way to get started on a doc test is to run grok in debug mode and when you're done, copy and paste what you've written in the interactive prompt. Here's a really simple doctest copy and paste after having run ./bin/zopectl debug. I call this file doctest.txt and I place it in the app_tests directory too:

Do a simple doctest test on the app.
************************************
:Test-Layer: unit

When you create an instance there are no objects in it::

   >>> from sample.app import Sample
   >>> grokapp = Sample()
   >>> list(grokapp.keys())
   []

Make sure it is found by the testrunner:

$ ./bin/test
Running tests at level 1
Running unit tests:
  Running:
..
  Ran 2 tests with 0 failures and 0 errors in 0.007 seconds.

These tests are just making sure that you're code is working but we haven't yet tried to run any tests in a full blown zope3 environment. That's called functional testing (aka. integration testing). The first test we'll make is a functional test in python. To be able to do this we need to use a so called "functional layer". Fortunately grokproject sets one up for you automatically which you can use in your functional test case. The magic you need is the instance object FunctionalLayer which you'll find in the file testing.py. Again pay attention to the marker in the doc string is set to python and also bare in mind that the test is extremely simple. Save this as functional.py in the app_tests/ directory.:

"""
Do a functional test on the app.

:Test-Layer: python
"""
from sample.app import Sample
from sample.testing import FunctionalLayer
from zope.app.testing.functional import FunctionalTestCase
class SampleFunctionalTest(FunctionalTestCase):
    layer = FunctionalLayer
class SimpleSampleFunctionalTest(SampleFunctionalTest):
    """ This the app in ZODB. """
    def test_simple(self):
        """ test creating a Sample instance into Zope """
        root = self.getRootFolder()
        root['instance'] = Sample()
        self.assertEqual(root.get('instance').__class__, Sample)

Apologies for the stupidity of the test but at least it's picked up the next we run bin/test:

$ ./bin/test
Running tests at level 1
Running unit tests:
  Running:
..
  Ran 2 tests with 0 failures and 0 errors in 0.021 seconds.
Running sample.testing.FunctionalLayer tests:
  Set up sample.testing.FunctionalLayer in 1.756 seconds.
  Running:
.
  Ran 1 tests with 0 failures and 0 errors in 0.022 seconds.
Tearing down left over layers:
  Tear down sample.testing.FunctionalLayer ... not supported
Total: 3 tests, 0 failures, 0 errors in 1.956 seconds.

As you can see, running the functional test is a lot slower. The first two tests took 0.007 seconds and now all three tests took 1.96 seconds. I guess the conclusion is to try to write as much of your tests non-functional and then put them together as a final touch as functional tests. This was not possible in Zope2 since all tests involving Zope classes had to be run as functional tests. The final type of test is a functional doc test which is really sexy in its simplicity. Create a file in app_tests/ called functional.txt and let it have the following content:

Do a functional doctest test on the app.
****************************************

:Test-Layer: functional

Test creating a Sample instance into Grok::

   >>> from sample.app import Sample
   >>> root = getRootFolder()
   >>> root['instance'] = Sample()
   >>> root.get('instance').__class__.__name__
   'Sample'

We now have one python unit test, a doc test, a python functional test and a function doc test. Let's check that they all run:

$ ./bin/test
Running tests at level 1
Running unit tests:
  Running:
..
  Ran 2 tests with 0 failures and 0 errors in 0.005 seconds.
Running sample.FunctionalLayer tests:
  Set up sample.FunctionalLayer in 1.755 seconds.
  Running:
.
  Ran 1 tests with 0 failures and 0 errors in 0.005 seconds.
Running sample.testing.FunctionalLayer tests:
  Tear down sample.FunctionalLayer ... not supported
  Running in a subprocess.
  Set up sample.testing.FunctionalLayer in 1.771 seconds.
  Running:
.
  Ran 1 tests with 0 failures and 0 errors in 0.004 seconds.
  Tear down sample.testing.FunctionalLayer ... not supported
Total: 4 tests, 0 failures, 0 errors in 4.896 seconds.

Further information

This How-to is meant to be an introduction to get you started on writing tests, how to discover them and how to run them. You now need to make up your mind of what kind of style of tests you prefer and what you need to test. To help making this How-to as short as possible, I've skipped...

Hopefully this How-to can improve over time to make things even simpler and dummy-proof but looking back you'll have to admit that we managed to get a lot done with very little configuration work.

You can download the whole Sample Grok project that I've used for writing this How-to.

see also:

Writing tests, discovering and running them with grok.testing
Get started writing tests for your Grok project and use grok.testing to automatically discover the tests.