怎么样打包自己的Python模块，并且分享到PyPI，相信下面的文章会给你一个满意的答案！
A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI.
Abstract
Few things give me caremads like Python modules I want to use that aren’t on PyPI (pronounced “pie pee eye”, or “cheese shop”, not “pie pie”!). On the other hand – aspydanny points out – the current situation on packaging is rather confusing.
Therefore I want to help everyone who has some great code but feels lost with getting it on PyPI. I will be using my latest opus “pem” as a realistic yet simple example of how to get a pure-Python 2 and 3 module packaged up, tested and uploaded to PyPI. Including the new and shiny binary wheel format that’s faster and allows for binary extensions (read the wheel story if you want to know more)!
I’ll keep it super simple to get everyone started. At the end, I’ll link more complete documentation to show you the way ahead. Sorry, no Windows.
Tools Used
This is not a history lesson, therefore we will use:
- pip 1.4+ (if you have no pip yet, bootstrapping is easy),
- setuptools 0.9+,
- and wheel 0.21+.
1 | $ pip install -U "pip>=1.4" "setuptools>=0.9" "wheel>=0.21" |
$ pip install -U "pip>=1.4" "setuptools>=0.9" "wheel>=0.21"
Please make sure all installs succeed, ancient installations may need some extra manual labor.
A Minimal Glimpse Into The Past
Forget that there ever was distribute (cordially merged into setuptools), easy_install (part of setuptools, supplanted by pip), or distutils2 aka packaging (was supposed to be the official thing from Python 3.3 on, didn’t get done in time due to lack of helping hands, got ripped out by a heart-broken Éric and abandoned now).
Be just vaguely aware that there are distutils and distlibsomewhere underneath but ideally it shouldn't matter to you at all for now.
setup.py
Nowadays, every project that you want to package needs asetup.py file. Let’s have a look at what pem’s could look like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | from setuptools import setup setup( name='pem', version='0.1.0', description='Parse and split PEM files painlessly.', long_description=(open('README.rst').read() + '\n\n' + open('HISTORY.rst').read() + '\n\n' + open('AUTHORS.rst').read()), url='http://github.com/hynek/pem/', license='MIT', author='Hynek Schlawack', author_email='hs@ox.cx', py_modules=['pem'], include_package_data=True, classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'Natural Language :: English', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Topic :: Software Development :: Libraries :: Python Modules', ], ) |
from setuptools import setup
setup(
name='pem',
version='0.1.0',
description='Parse and split PEM files painlessly.',
long_description=(open('README.rst').read() + '\n\n' +
open('HISTORY.rst').read() + '\n\n' +
open('AUTHORS.rst').read()),
url='http://github.com/hynek/pem/',
license='MIT',
author='Hynek Schlawack',
author_email='hs@ox.cx',
py_modules=['pem'],
include_package_data=True,
classifiers=[
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'Natural Language :: English',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Topic :: Software Development :: Libraries :: Python Modules',
],
)In the simplest case, you just import setup from setuptoolsand run it with some keyword arguments.
Before I go into these arguments, I want to point out one of the most neglected and yet most important one: license.Always set a license! Otherwise nobody can use your module, which would be a pity, right?
I take a few shortcuts here which you may want to copy too:
- I love DRY, therefore my long description is just a concatenation of my README, change log and author credits.
- I use bumpversion for manipulating my version strings.
An essential field is py_modules which you'll use to denote the relevant code for packaging, if your package is actually just a single module like in pem’s case. If you have a full-blown package (i.e. a directory), you’ll go for:
1 2 3 4 5 6 7 | from setuptools import setup, find_packages setup( ... packages=find_packages(exclude=['tests*']), ... ) |
from setuptools import setup, find_packages
setup(
...
packages=find_packages(exclude=['tests*']),
...
)The exclude makes sure that a top-level tests package doesn’t get installed (it’s still part of the source distribution) since that would wreak havoc.
You can also specify them by hand like I used to fordoc2dash’s setup.py. However, unless you have good reasons, just use find_packages() and be done.
The classifiers field’s usefulness is openly disputed, nevertheless pick them from here. PyPI will refuse to accept packages with unknown classifiers, hence I like to use "Private :: Do Not Upload" for private packages to protect myself from my own stupidity.
One icky thing are dependencies. Unless you really know what you’re doing, don’t pin them (specifying minimal version your package requires to work is fine of course) or your users won’t be able to install security updates of your dependencies:
1 2 3 4 5 | setup( ... install_requires=['Django', 'django-annoying'], ... ) |
setup(
...
install_requires=['Django', 'django-annoying'],
...
)Rule of thumb: requirements.txt should contain only ==,setup.py the rest (>=, !=, <=, …).
Non-Code Files
Every Python project has a “Fix MANIFEST.in” commit. Look it up, it’s true.
But it’s not really that hard: you add all files and directories that are not packaged due to py_modules or packages in your setup.py.
For “pem”, it’s just
include *.rst LICENSE
For more commands, have a look at the MANIFEST.in docs – especially recursive-include is used widely. There’s also a handy tool to validate your MANIFEST.in: check-manifest.
Important: If you want the files and directories fromMANIFEST.in to also be installed (e.g. if it’s runtime-relevant data), you will have to set include_package_data=True in yoursetup() call as in the example above!
Configuration
For our minimal Python-only project, we’ll only need two lines in setup.cfg:
1 2 | [wheel] universal = 1 |
[wheel] universal = 1
These will make wheel build a universal wheel file (e.g.pem-0.1.0-py2.py3-none-any.whl) and you won’t have to circle through virtual environments of all supported Python versions to build them separately.
Documentation
As I’ve hopefully established, every open source project needs a license. No excuses.
Additionally, even the simplest package needs a README that tells potential users what they’re looking at. Make itreStructuredText (reST) so PyPI can properly render it on your project page. As a courtesy to your users, also keep a change log so they know what to expect from your releases. And finally it’s good style to credit your contributors in a third file. I like to call them README.rst,HISTORY.rst, and AUTHORS.rst respectively, but there’s no hard rule.
My long project description and thus PyPI text is the concatenation of those three (and I’ve shamelessly stolen it from Kenneth Reitz).
If you host on GitHub, you may want to add aCONTRIBUTING.md that gets displayed when someone wants to open a pull request. Unfortunately it has to be a Markdown file because GitHub doesn’t support reST for this. Have a look at pem’s if you need inspiration.
Let’s Build Finally!
Building a source distribution (which is what is usually used for pure-Python project) is just a matter of
1 | $ python setup.py sdist |
$ python setup.py sdist
Because it’s 2013 and wheels are awesome, we’ll build one too! Fortunately, it’s just as easy (please note that you’ll need to install wheel as per above for this to work!):
1 | $ python setup.py bdist_wheel |
$ python setup.py bdist_wheel
Of course, the wheel package is optional, but your users will thank you for much faster installation times.
Now you should have a new directory called dist containing a source distribution file and a wheel file. For pem, it currently looks like this:
dist ├── pem-0.1.0-py2.py3-none-any.whl └── pem-0.1.0.tar.gz
You can test whether both install properly before we move on to uploading:
$ rm -rf 27-sdist # ensure clean state if ran repeatedly $ virtualenv 27-sdist ... $ 27-sdist/bin/pip install --no-index dist/pem-0.1.0.tar.gz Ignoring indexes: https://pypi.python.org/simple/ Unpacking ./dist/pem-0.1.0.tar.gz Running setup.py egg_info for package from file:///Users/hynek/Projects/pem/dist/pem-0.1.0.tar.gz Installing collected packages: pem Running setup.py install for pem Successfully installed pem Cleaning up... $ 27-sdist/bin/python Python 2.7.4 (default, Apr 21 2013, 09:35:41) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pem >>> pem.__version__ '0.1.0'
and
$ rm -rf 27-wheel # ensure clean state if ran repeatedly $ virtualenv 27-wheel ... $ 27-wheel/bin/pip install --use-wheel --no-index --find-links dist pem Ignoring indexes: https://pypi.python.org/simple/ Downloading/unpacking pem Installing collected packages: pem Successfully installed pem Cleaning up... $ 27-wheel/bin/python Python 2.7.4 (default, Apr 21 2013, 09:35:41) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pem >>> pem.__version__ '0.1.0'
Please note the lack of “Running setup.py install for pem” in the second test. Yes, you can finally install packages without executing arbitrary code!
So you’re confident that your package is perfect? Let’s use the Test PyPI server to find out!
The PyPI Staging Server
Again, I’ll be using “pem” as the example project name to avoid <your project name> everywhere.
First, sign up on the test server, you will receive a user name and a password. Please note that this is independent from the live servers. Thus you’ll have to re-register both yourself and your packages. It also gets cleaned from time to time so don’t be surprised if it suddenly doesn’t know about you or your projects anymore. Just re-register.
Next, create a ~/.pypirc consisting of:
1 2 3 4 5 6 7 8 | [distutils] index-servers= test [test] repository = https://testpypi.python.org/pypi username = <your user name goes here> password = <your password goes here> |
[distutils]
index-servers=
test
[test]
repository = https://testpypi.python.org/pypi
username = <your user name goes here>
password = <your password goes here>Then use
1 | $ python setup.py register -r test |
$ python setup.py register -r test
to register your project with the PyPI test server.
Finally, upload your distributions:
1 2 | $ python setup.py sdist upload -r test $ python setup.py bdist_wheel upload -r test |
$ python setup.py sdist upload -r test $ python setup.py bdist_wheel upload -r test
Please note that calling upload without building something in the same command line will give you a:
error: No dist file created in earlier command
Now test your packages again:
1 | $ pip install -i https://testpypi.python.org/pypi pem |
$ pip install -i https://testpypi.python.org/pypi pem
Everything dandy? Then lets tackle the last step: putting it on the real PyPI!
The Final Step
First, register at PyPI, then complete your ~/.pypirc:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | [distutils] index-servers= pypi test [test] repository = https://testpypi.python.org/pypi username = <your test user name goes here> password = <your test password goes here> [pypi] repository = http://pypi.python.org/pypi username = <your production user name goes here> password = <your production password goes here> |
[distutils]
index-servers=
pypi
test
[test]
repository = https://testpypi.python.org/pypi
username = <your test user name goes here>
password = <your test password goes here>
[pypi]
repository = http://pypi.python.org/pypi
username = <your production user name goes here>
password = <your production password goes here>One last deep breath and let’s rock:
1 2 | $ python setup.py sdist upload -r pypi $ python setup.py bdist_wheel upload -r pypi |
$ python setup.py sdist upload -r pypi $ python setup.py bdist_wheel upload -r pypi
And thus, your package is only a pip install away for everyone! Congratulations, do more of that!
Next Steps
The information herein will probably get you pretty far but if you get stuck, the current canonical truths for Python packaging are:
- Python Packaging User Guide
- setuptools
- wheel
- distutils
- and specifically the PyPI chapter in the latter one.
Please don’t let your software rot on GitHub or launchpad! Share!
Thanks
This article has been kindly proof-read by Lynn Root,Donald Stufft, Alex Gaynor, Thomas Heinrichsdobler, andJannis Leidel. All mistakes are still mine though.
I’d be happy if you have any feedback to make this article even more straight-forward for people new to Python packaging!
