Packaging in python from a former Perl dev point of view

It's easy

I use github, readthedocs post commit hooks, and it all works fine: I am delighted to be honest.

As far as I am concerned packaging in python is freaking easy, a fun and rewarding experience. So here is my point: this is not a rant since I love packaging in python.

However, to be honest there are quite a few things that trouble me:
  • I don't really know what I am doing since I mostly follow cookbooks;
  • I am lacking some of the CPAN features;
  • and I do have (the maybe wrong feeling) that the packaging culture amongst python community is not as strong as in Perl (this also applies to my production).

from CPAN import wisdom

Most of Perl's quality module are not coming from PEP they are coming from cultural habits. I dare cut and paste some of tutorial for perl

  • Write the documentation for a module first, before writing any code. Discuss the module with other people first, before writing any code. Plan the module first, before writing any code.
  • It's easy to come up with a solution to a problem. It takes planning to come up with a good solution. Remember: the documentation, not the code, defines what a module does.
  • Every module should have a purpose. There's a proliferation of modules with names like "", "", and "" that have no obvious purpose, and it's difficult to know what each does. This leads to confusion and duplication of code.
Well, since we are plagued on pypi with the infamous «nested list printer» and ports of PHP file_get_contents, I guess this wisdom is not yet totally in python.

I think python tutorial focus too much on the technical part (how to build a package) and not enough on the QA part (how to make a useful and maintainable package).

There is also something in CPAN I love, and I don't follow because I lost myself in geeking with sphinx, it is the straightforward documentation in one page following this plan:
  • Name 
  • Version
  • Synopsis (short code snippet that works)
  • Description (in full english with no code)
  • Methods (with code snippets if useful)
  • Notes (extra informations needed)
  • See also (similar packages)
  • Limitations
  • Bugs
  • Author
  • Licence
It is a kind of very informative plan.

These are what the culture provides. Plans are not imposed to the packager, it just converged as being efficient.

from CPAN import tools

Now, CPAN has also great tools we miss in python: for instance prior to installation there are the automated tests and eventually automated reports.

I do always test a package before pushing it, but, I'd rather force tests that prevent installation if they fail on the user side. I tried ditribute test suites feature, but I fumbled.

You know what, I miss this feature, and the deployment matrix.

I miss to see if maintainer is active by being able to watch its ticketing queue. The ticketing system is in CPAN.

I also miss the direct link to the source. Or the dependencies chart.

I also like how they handle the «Missing In Action» of packagers and how they can decide to hand over a package's maintenance to another maintainer.

This have very few chances to happen in a close future. However, I can see how we can all improve our package.

from packager import good_will

Good news is we don't need code to solve most of these problems.


In the README I provide with my packages I (try to) include:
  • a link to the sources
  • a link to the full documentation (on readthedocs and
  • a link to the ticketing of github
  • a synopsis
  • requirements (in case my dependencies get weirdly not computed)
  • a changelog
I will try from now on to be terser in my documentation and follow the previously mentioned Perl plan.

I noticed repoze.lru changed its former nice README for a useless one. I am sad.


Always test before pushing. It may seem obvious, but I have noticed some maintainers don't. Build a sdist, make a clean virtualenv with nothing to install your package before uploading it too. It is nice.



Always tag your source code in your repository with the adequate pypi version.

Don't ask GvR what you can achieve by yourself

First I understand nothing of the mess between distribute, distribute2, setuptools, so I gave up dreaming of hacking my way to a solution through brute coding. And I guess you don't make a donkey that is not thirsty drink water.

My workaround is as a packager I can by showing the example and hope people will follow me improve python packaging with my own lever: practice and culture.

In the ecosystem I am not only a producer I am also a consumer of packages. So I think that as consumers of packages YOU can also improve the packaging ecosystem by checking that a package follows most of these rules before installing it:

  • is the README on pypi including 
    • a link to the source, 
    • the ticketing system, 
    • a synopsis, 
    • a changelog, 
    • a link to the the full documentation (1 point per present info);
  • is the full documentation following the Perl canonical Plan (in regard to the complexity of the package don't be too picky (5 points if the doc is relevant))
  • does the source code contains a test suite? (5 points)
  • can I reach the maintainer IRL (2 points)
  • are there outstanding issues in the ticketing (2 points if all issues are opened for less than 1 month)
  • is there an auto reporting tool in the package (that triggers the test suite and submit it to a ticketing system) (5 points (I won't have them)). 
  • versioning the PEP way (3 points)
In my case I won't use a package having less than 20 points on my own scale. If we all do that we have a chance that packaging improves. I guess Perl has made mistakes so I really don't advocate following their steps blindly, I advocate that we do also slowly build our own strong culture of Quality Assurance the python way.

Oh, and since I am proud of finalizing this «Book» (perl dev) snippet, here is a polyglot in Perl and python to do i++ and ${A}++

q = 0 or """ #=;$A=41;sub A { ~-$A+2}; A() && q' """
A=lambda A: -~A #';
print A(41) # python + perl = <3

No comments: