5dollarwhitebox.org - theboxownsyou

Summary

I came across an issue today whilst trying to get a TurboGears application setup, and that is... how do you setup a TurboGears applications as part of a bigger application. Meaning, the overall project has multiple parts: The web frontend (TurboGears), the command line client, etc. I found that using paster to get a TurboGears application created in the way isn't the most straight forward thing in the world.

That said, with a bit of fenagling, it is possible to get a TG application created as part of a shared namespace (See SetupTools Namespace Packages).

Please note that this article is not intended to give solid info on setting up and using TurboGears, but rather to simply address the issue of namespace packages and TurboGears. Please review the Reference at the bottom of this article for more information.

Please also note that I am not a TurboGears or Python expert. If there is a better way to do this, please drop me a line or leave a comment.

Getting Started - How Things Work Out of The Box

For those that might not be entirely familiar with TurboGears, I want to show how things work first before throwing in namespacing. The following steps create, setup, and start a TurboGears project:

Create the project:

[tmp] $ paster quickstart Helloworld
 
Enter package name [helloworld]: 
Do you need authentication and authorization in this project? [yes] 
Selected and implied templates:
  tg.devtools#turbogears2  TurboGears 2.0 Standard Quickstart Template
 
Variables:
  auth:        sqlalchemy
  egg:         Helloworld
  geo:         None
  package:     helloworld
  project:     Helloworld
  sqlalchemy:  True
  sqlobject:   False
  tgversion:   2.0
Creating template turbogears2
Creating directory ./Helloworld
<output snipped>

As you can see, the 'paster' utility easily created us a brand new TurboGears 2 project. To get this going, we just need to setup the app, and then start it up. Using paster once again, we issue the 'setup-app' command to do any initial setup like creating databases, and what-not.

[Helloworld] $ paster setup-app development.ini 
 
Running setup_config() from helloworld.websetup
<output snipped>

I snipped out all the output, but essentially anything in ./helloworld/websetup.py gets run to setup databases and such. Remember that line there that says "Running setup_config() from helloworld.websetup" as that will be important later in the article. Finally, we run our development web server and test out our application:

[Helloworld] $ paster serve --reload development.ini
 
Starting subprocess with file monitor
Starting server in PID 18816.
serving on http://127.0.0.1:8080
...

And obviously, you can hit your web app by directing your web browser to 'http://127.0.0.1:8080'.

So What The Problem Is

Now that we've seen how easy it is to setup a basic TG2 application, lets now look at another example that is a little bit more complicated.

Lets say that our 'Helloworld' application is actually just a small part of a larger application we will call 'SampleApplications'. And as such, we want to have a shared namespace so that I can do things like:

>>> from sample_apps.web.helloworld import HelloWorld
>>> from sample_apps.web.goodbyeworld import GoodbyeWorld
>>> ....

You can see what we mean now by a shared namespace. All applications are installed as part of 'sample_apps' and are called via that library path 'sample_apps.namespace.subapp_name'.

Lets see what happens when we try to create a TurboGears project as a 'sub application' of another larger application.

Create The Parent Application

[tmp] $ paster create sample_apps
 
[tmp] $ cd sample_apps

      entry_points="""
      # -*- Entry points: -*-
      """,
      namespace_packages=['sample_apps', 'sample_apps.web'],
      )

Then we just create the namespace dir and add a '__init__.py' file in it:

[sample_apps] $ mkdir sample_apps/web
 
[sample_apps] $ touch sample_apps/web/__init__.py

Note that every namespace package must have the following in their '__init__.py' files:

__import__('pkg_resources').declare_namespace(__name__)

So for this example we need to add the above line to the files:

./sample_apps/__init__.py
./sample_apps/web/__init__.py

And now that we have our namespace setup, we need to install our application. Since we are only developing, I'll just use the 'develop' install method.

[sample_apps] $ python setup.py develop

The TurboGears SubProject Within Namespace

Now that we have our global project 'sample_apps' setup, and the 'sample_apps.web' namespace, we want to create a TurboGears application under that namespace.

Note that we create the project outside of the sample_apps directory.

[tmp] $ paster quickstart sample_apps.web.helloworld
 
[tmp] $ cd sample_apps.web.helloworld

We've created the project as 'sample_apps.helloworld' as a convenience, however we need to clean this up a little bit to make it look the way we want (as a sub project/namespace package of sample_apps).

[sample_apps.web.helloworld] $ mkdir -p sample_apps/web
 
[sample_apps.web.helloworld] $ mv sample_apps.web.helloworld sample_apps/web/helloworld

Again, you need to add the following to all the __init__.py file in the namespace packages:

__import__('pkg_resources').declare_namespace(__name__)

For this example, that would be the following files:

sample_apps/__init__.py
sample_apps/web/__init__.py
sample_apps/web/helloworld/__init__.py

There are a few references to 'sample_apps.web.helloworld' in setup.py that need to be changed to 'sample_apps/web/helloworld'. Note that you only want to change the references to the directory, not the library path.

In setup.py change the following:

package_data={'sample_apps.web.helloworld' => package_data={'sample_apps/web/helloworld':
message_extractors={'sample_apps.web.helloworld': => message_extractors={'sample_apps/web/helloworld':

Remove the .egg-info directory, and regenerate it:

[sample_apps.web.helloworld] $ rm -rf sample_apps.web.helloworld.egg-info/
[sample_apps.web.helloworld] $ python setup.py egg_info

Now, the final piece is a bit of a hack, however 'paster setup-app' will not work unless you modify the file 'top_level.txt' in your .egg-info directory. This will need to be changed anytime you need to run 'paster setup-app' after the .egg-info has be regenerated (python setup.py [install, develop, etc]). I simple change it from 'sample_apps' to 'sample_apps.web.helloworld':

[sample_apps.web.helloworld] $ echo "sample_apps.web.helloworld" > sample_apps.web.helloworld.egg-info/top_level.txt

You should now be able to run the following successfully:

[sample_apps.web.helloworld] $ paster setup-app development.ini
[sample_apps.web.helloworld] $ paster serve --reload development.ini

Lets just see how that shared namespace works now in a python shell:

>>> from sample_apps.web.helloworld import controllers

And there you have it, a TurboGears projects installed as part of a shared namespace.

Reference

* SetupTools
* Turbo Gears
* Paste