279 lines
11 KiB
Plaintext
279 lines
11 KiB
Plaintext
==============================================
|
|
Why, who, what and how do you do *the py lib*?
|
|
==============================================
|
|
|
|
.. contents::
|
|
.. sectnum::
|
|
|
|
|
|
Why did we start the py lib?
|
|
============================
|
|
|
|
.. _frustrations:
|
|
|
|
Frustrations and the fun of developing new ways
|
|
-----------------------------------------------
|
|
|
|
Among the main motivations for writing the py lib is
|
|
frustration at existing python modules and packages,
|
|
among them:
|
|
|
|
- there is no standard way of testing python applications,
|
|
scripts or modules. everybody does its own hack around
|
|
the unittest.py if testing happens at all.
|
|
|
|
- due to python's default *export all names* policy for modules
|
|
and packages it is hard to refactor code across
|
|
releases because you might break usages of your code
|
|
|
|
- the python "batteries included" standard modules are tied to
|
|
the python release process. You can't easily benefit from
|
|
new python module in older python versions which you might
|
|
have to use for whatever reason.
|
|
|
|
- support for richer interactive interaction with command line
|
|
utilities or e.g. pygame-based shells is missing.
|
|
|
|
- modules/packages are often implemented in javaesque -style
|
|
|
|
- distributed applications are implemented using
|
|
a variant of Remote Method Invocation (RMI) which carries
|
|
a lot of problems and is often difficult to deploy
|
|
with respect to different platforms and versions.
|
|
|
|
- there is no _automated_ way of installing, maintaining
|
|
and upgrading applications
|
|
|
|
The py lib tries to address these problems. It is thus in
|
|
risk of trying to do too many things at once. Of course, we
|
|
can't solve them all at once but you will find that the above
|
|
points currently drive the development of the py lib.
|
|
|
|
|
|
Experimenting with new days, appreciating the existing ones
|
|
-----------------------------------------------------------
|
|
|
|
First note that we very much appreciate the great work of all
|
|
the python developers and the python-dev team in particular.
|
|
Criticism of old ways and experimenting with new ways doesn't
|
|
imply that the old ways are bad. But we all strive for the
|
|
holy development grail at least some of the time, don't we?
|
|
|
|
And because it happens far too rarely, let us especially emphasize
|
|
our appreciation for the `great work that Fred L. Drake, Jr.
|
|
is doing`_ with maintaining the core Python documentation.
|
|
|
|
What is the py libs current focus?
|
|
==================================
|
|
|
|
testing testing testing
|
|
-----------------------
|
|
|
|
Currently, the main focus of the py lib is to get a decent
|
|
`test environment`_, indeed to produce the best one out there.
|
|
Writing, distributing and deploying tests should become
|
|
a snap ... and fun!
|
|
|
|
Automated tests fit very well to the dynamism of Python.
|
|
Automated tests ease development and allow fast refactoring
|
|
cycles. Automated tests are a means of communication
|
|
as well.
|
|
|
|
We try to allow test scripts with minimal boilerplate code or
|
|
no boilerplate at all. With the py lib you can simply use
|
|
``assert`` statements in order to - well - assert something
|
|
about objects in your code. No ``assertEqual(s)`` and all the
|
|
other kinds of funny names which only cover part of what you
|
|
want to assert about an expression, anyway.
|
|
|
|
allowing maximum refactoring in the future ...
|
|
----------------------------------------------
|
|
|
|
explicit name export control
|
|
............................
|
|
|
|
In order, to allow a fast development pace across versions of
|
|
the py lib there is **explicit name export control**. You
|
|
will only see names which make sense to use from the outside
|
|
and which the py lib developers want to guarantee across multiple
|
|
revisions. However, you don't need to treat the ``py`` lib as
|
|
anything special. You can simply use the usual ``import``
|
|
statement and will not notice much of a difference - except that
|
|
the namespaces you'll see from the ``py`` lib will be extreamly
|
|
clean and have no clutter.
|
|
|
|
Indeed, much thought is given to reduce the exported *name
|
|
complexity*. This is an area where the python "batteries" and
|
|
many python packages unfortunately lack a lot. They expose so
|
|
many names that it becomes very hard to change APIs across
|
|
releases. People have been adding whole interface systems
|
|
because of that but arguably this adds another layer of names
|
|
instead of reducing the original problem.
|
|
|
|
Anyway, exporting to many names kills the fun of refactoring
|
|
and improving things. We want to avoid that as much as
|
|
possible.
|
|
|
|
Upcoming Release policy & API guarantees
|
|
........................................
|
|
|
|
We'll talk about major, minor and micro numbers as the three
|
|
numbers in "1.2.3" respectively. These are the (draft!)
|
|
release policies:
|
|
|
|
- Micro-releases are bug fix releases and may not introduce
|
|
new names to the public API. They may add tests and thus
|
|
further define the behaviour of the py lib. They may
|
|
completly change the implementation but the public API
|
|
tests will continue to run.
|
|
|
|
- No **tested feature** of the exported `py API`_ is to vanish
|
|
across minor releases until it is marked deprecated.
|
|
|
|
For example, pure API tests of a future version 1.0 are to
|
|
continue to fully run on 1.1 and so on. If an API gets
|
|
deprecated with a minor release it goes with the next minor
|
|
release. Thus if you don't use deprecated APIs you should
|
|
be able to use the next two minor releases. However, if
|
|
you relied on some untested implementation behaviour,
|
|
you may still get screwed. Solution: add API tests to the
|
|
py lib :-) It's really the tests that make the difference.
|
|
|
|
- Pure API tests are not allowed to access any implementation
|
|
level details. For example, accessing names starting with
|
|
a single leading '_' is generally seen as an implementation
|
|
level detail.
|
|
|
|
- we intend to involve expert developers who give new APIs an
|
|
independent review before they go into a minor release
|
|
and even more so before they go into a major release.
|
|
|
|
- major releases *should*, but are not required to, pass
|
|
all API tests of the previous latest major released
|
|
version. A full list of changes is to be included in
|
|
the release notes, including the tests that got abandoned.
|
|
|
|
the need to find the right *paths* ...
|
|
--------------------------------------
|
|
|
|
Another focus are well tested so called *path* implementations
|
|
that allow you to seemlessly work with different backends,
|
|
currently a local filesystem, subversion working copies and
|
|
subversion remote URLs. Moreover, there is an experimental
|
|
``extpy`` path to address a Python object on the (possibly
|
|
remote) filesystem. The `jorendorff path.py`_ implementation
|
|
goes somewhat in the same direction but doesn't try to
|
|
systematically access local and remote file systems as well as
|
|
other hierarchic namespaces. The latter is the focus of the
|
|
``py.path`` API.
|
|
|
|
If you are ready to grasp more then you may try reading
|
|
about future_ coding goals of the py lib, which reflect the
|
|
current developers thoughts and discussions.
|
|
|
|
.. _`jorendorff path.py`: http://www.jorendorff.com/articles/python/path/
|
|
|
|
How does py development work?
|
|
=============================
|
|
|
|
Communication and coding style
|
|
------------------------------
|
|
|
|
We are discussing things on our `py-dev mailing list`_
|
|
and collaborate via the codespeak subversion repository.
|
|
|
|
We follow a `coding style`_ which strongly builds on `PEP 8`_,
|
|
the basic python coding style document.
|
|
|
|
It's easy to get commit rights especially if you are an
|
|
experienced python developer and share some of the
|
|
frustrations described above.
|
|
|
|
Moreover, the world will be granted svn commit rights to all
|
|
py test files so that you can easily add bug-tests or tests
|
|
for behaviour to make sure the tested behaviour persists
|
|
across releases. If you are itching to actually fix or
|
|
refactor any implementation code you can likely get commit
|
|
rights to do it. However, it is then (and anyway) a good idea to
|
|
follow the `py-dev mailing list`_ and grasp some of the things
|
|
that are going on in the `future`_ book.
|
|
|
|
Licensing, please
|
|
-----------------
|
|
|
|
Oh right, and you should also agree to release your contributions
|
|
under an ``MIT license`` and consequently any other OSI-approved
|
|
license. This is FOSS [#]_ and we want to have the py lib interopable
|
|
with whatever license style you prefer. Copyright generally
|
|
stays with the contributors. We are following the
|
|
linux-kernel dev-model with this one.
|
|
|
|
Hum, maybe we can also compute the individual copyrights from
|
|
the subversion blame command. Would be a fun way to handle it.
|
|
Basically, nobody should ever have a problem to use the py lib
|
|
under any OSI approved license and also for commercial
|
|
purposes.
|
|
|
|
Are there connections with PyPy_?
|
|
---------------------------------
|
|
|
|
Some of the motivation for writing the py lib stems from needs
|
|
during PyPy_ development, most importantly testing and
|
|
file system access issues. PyPy puts a lot of pressure
|
|
on a testing environment and thus is a great **reality test**
|
|
kind of thing.
|
|
|
|
More importantly, the development perspective taken from the
|
|
PyPy developers has some influence. For example, the
|
|
`envisioned import/export system`_ clearly has some thought
|
|
references to the way PyPy tries to separate different levels
|
|
of objects and execution in order to reach a higher level
|
|
view of what is going on.
|
|
|
|
Who is "we"? Some history ...
|
|
=============================
|
|
|
|
Some initial code was written from *Jens-Uwe Mager* and *Holger
|
|
Krekel*, after which Holger continued on a previous
|
|
incarnation of the py.test tool (known first as 'utest', then
|
|
as 'std.utest', now, finally and at last 'py.test').
|
|
|
|
Helpful discussions took place with *Martijn Faassen*, *Stephan
|
|
Schwarzer* and then *Armin Rigo* who contributed important parts.
|
|
He and Holger came up with a couple of iterations of the
|
|
testing-code that reduced the API to almost nothing: just the
|
|
plain assert statement and a ``py.test.raises`` method to
|
|
check for occuring exceptions.
|
|
|
|
Now recently, after Holgers `talk at EP2004`_ more people
|
|
were interested and there were discussions with *Jim Fulton*,
|
|
*Marius Gedminas*, *Laura Creighton* and more recently, *Ian Bicking*.
|
|
|
|
However, there is no real core development team as such, yet.
|
|
Also we are somewhat lacking in the win32 area. Every now and
|
|
then the py lib is tested on windows but it's currently not a
|
|
practical concern of one of the current developers or
|
|
contributors.
|
|
|
|
However, one of the future directions of the `py.test tool and
|
|
library`_ is to allow running tests across multiple python
|
|
versions and computers. Then we can run tests without having
|
|
to walk up or boot up a windows machine :-)
|
|
|
|
.. _`talk at EP2004`: http://codespeak.net/svn/user/hpk/talks/std-talk.txt
|
|
.. _`coding style`: coding-style.html
|
|
.. _`PEP 8`: http://www.python.org/peps/pep-0008.html
|
|
.. _`py-dev mailing list`: http://codespeak.net/mailman/listinfo/py-dev
|
|
.. _`test environment`: test.html
|
|
.. _`PyPy`: http://codespeak.net/pypy
|
|
.. _`envisioned import/export system`: future/future.html#importexport
|
|
.. _future: future/future.html
|
|
.. _`py.test tool and library`: test.html
|
|
.. _`py API`: api.html
|
|
.. _`great work that Fred L. Drake, Jr. is doing`: http://www.python.org/doc/2.3.4/lib/lib.html
|
|
|
|
--
|
|
|
|
.. [#] FOSS is an evolving acronym for Free and Open Source Software
|
|
|