Improved zc.buildout recipes with ZopeSkel

22 Jan

Today I worked with Tarek Ziadé on ZopeSkel. Tarek concentrated on refactoring the ZopeSkel layout to put each template in its own module and wrote doctests for all available templates. Go Tarek! The test runner actually runs tests in two layers: first testing the output of the generated items and then, if the items contain tests themselves running them also.

I concentrated on improving the template for creating new zc.buildout recipes. Many useful recipes suffer from lacking documentation and an unappealing front page on PyPI. I refactored the template to include a common set of documentation files, such as CHANGES.txt, README.txt, CONTRIBUTORS.txt etc. and added code that puts all those documents nicely together to produce a serious looking ReST document that looks good on PyPI. So now its up to the recipe author to just fill in those files accordingly.

To help recipe authors and especially people new to zc.buildout I also added comments in both the documentation files and the code to help on implementing the recipe and especially on how to document it so that other people are able to use the recipe in their own buildouts. To me, one of the most importart parts of a recipe’s documentation is the list of available options and their semantics. Looking at the PyPI pages for zc.buildout and zc.recipe.egg you can easily get information about the component. I’ve also tried to do the same with my own recipes (hexagonit.recipe.cmmi, The template provides a stub for documenting the options in the README.txt file that authors can fill in.

I also created a minimal doctest for the buildout. While being only a skeleton the test actually runs a buildout using the recipe so you can run the test case for the recipe right after ZopeSkel is finished generating it. This should help recipe authors to get started with testing the recipe while they implement it.

In addition I updated the trove classifiers to appropriate values for a buildout recipe and added support for getting the trove classifier for the license to be added automatically in the file. So now when paster asks for a license for the recipe and you answer, for example, ZPL you get ‘License :: OSI Approved :: Zope Public License’ in your automatically. This code is actually in zopeskel.base and you can easily re-use it in the other ZopeSkel templates. Just take a look at how the recipe template uses it.

If you haven’t used ZopeSkel before, give it a try!

$ easy_install ZopeSkel
$ paster create --list-templates
$ paster create -t recipe collective.recipe.foobar

If you want to try the recent changes, you need to get ZopeSkel from the collective.

There’s been lots of interest in ZopeSkel here at the Snowsprint so expect to have cool new templates there soon!

Update: 25.01.2007

ZopeSkel 1.5 was released which contains the latest changes.


Posted by on January 22, 2008 in plone, software engineering, zope


8 responses to “Improved zc.buildout recipes with ZopeSkel

  1. Hanno Schlichting

    January 23, 2008 at 0:00

    Awesome work, keep it up 🙂

  2. David Glick

    March 17, 2008 at 2:47

    I took advantage of this this past week while working on collective.recipe.omelette. Thanks — it saved me a bunch of time.

    I’ve run into one small issue with the sample doctest — because it runs the buildout using a system() call, there’s no way I can insert a pdb breakpoint in my recipe. The doctest runner simply sees the pdb prompt and calls it a failing test. Do you have any ideas about how to get around this?

    Could you also clarify how the tests are intended to be run? My only testing experience is with zope products, using bin/instance test, so I’m a bit out of my depth. Is python test the right way?

  3. Kai Lautaportti

    March 17, 2008 at 21:58


    I usually develop my recipes inside a buildout and create a test runner script that will run the test suite. In this case I’d have something like this in my buildout.cfg:

    develop = src/collective.recipe.omelette
    parts = test

    recipe = zc.recipe.testrunner
    eggs = collective.recipe.omelette
    defaults = [‘–auto-color’, ‘-v’]

    Assuming that the recipe code is checked out in ./src/collectice.recipe.omelette running the above buildout will generate a ./bin/test script that runs the test suite. You can of course have multiple recipes in the buildout, just add them in the eggs = line in the [test] section and all their tests will be run.

    As for the pdb problem, that’s a tricky one. Under the hood zc.buildout uses os.popen3() to execute the buildout which, as you found out, makes using pdb the normal way useless. Unfortunately I don’t have a good solution for this at the moment except that if you need to use pdb, you can create a small buildout configuration that uses the recipe and run it directly using buildout (instead of doing it in the doctest). If you come up with a solution please let me know!

    Btw, nice work on collective.recipe.omelette, it’s quite handy!

  4. David Glick

    March 17, 2008 at 23:06

    Thanks for the additional information. I did try running the auto-generated buildout and then bin/test; however I ran into trouble here because my recipe depends on zc.recipe.egg, and I wasn’t sure about the proper way to make that egg available to the test running in the buildout environment.

  5. Kai Lautaportti

    March 18, 2008 at 8:42

    You can make zc.recipe.egg available for the tests by editing the setUp() function in collective/recipe/omelette/tests/ to the following:

    def setUp(test):
        # Install the recipe in develop mode
        zc.buildout.testing.install_develop('zc.recipe.egg', test)
        zc.buildout.testing.install_develop('collective.recipe.omelette', test)

    That should fix it.

  6. David Glick

    March 29, 2008 at 6:52

    I just got around to trying this, but it still doesn’t work. I still get the following when I try to run bin/test:


    Test-module import failures:

    Module: build.lib.collective.recipe.omelette.tests.test_docs

    ImportError: No module named build.lib.collective.recipe.omelette.tests.test_docs

    Module: collective.recipe.omelette.tests.test_docs

    Traceback (most recent call last):
    File “/Users/davidg/ONENW/collective.recipe.omelette/collective/recipe/omelette/”, line 24, in ?
    import zc.recipe.egg
    ImportError: No module named zc.recipe.egg

    Total: 0 tests, 0 failures, 0 errors in 0.000 seconds.

    Test-modules with import problems:


    I notice that zc.recipe.egg isn’t showing up in sys.path in bin/test, but I don’t know what to do to get it there. I have it listed in install_requires and tests_require in my

  7. Kai Lautaportti

    March 29, 2008 at 17:30

    Are you sure you tried adding the

    zc.buildout.testing.install_develop(‘zc.recipe.egg’, test)

    line to your file like I suggested above? I just checked out collective.recipe.omelette trunk from the SVN and applied the said change and I’m able to run the tests fine without import errors. There are currently some test failures, but those should be easy to fix.

    I can commit the change in collective if you’d like?

  8. David Glick

    March 29, 2008 at 19:25

    It works for me with a fresh checkout; I think something about my buildout was messed up. Thanks for your help.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: