现在的位置: 首页 > 程序设计> 正文
小 中 大
分享Python模块到PyPI
2013年08月03日 ⁄ 程序设计 ⁄ 暂无评论 ⁄ 被围观 3,963+

怎么样打包自己的Python模块，并且分享到PyPI，相信下面的文章会给你一个满意的答案！

A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI.

Abstract

Few things give me caremads like Python modules I want to use that aren’t on PyPI (pronounced “pie pee eye”, or “cheese shop”, not “pie pie”!). On the other hand – aspydanny points out – the current situation on packaging is rather confusing.

Therefore I want to help everyone who has some great code but feels lost with getting it on PyPI. I will be using my latest opus “pem” as a realistic yet simple example of how to get a pure-Python 2 and 3 module packaged up, tested and uploaded to PyPI. Including the new and shiny binary wheel format that’s faster and allows for binary extensions (read the wheel story if you want to know more)!

I’ll keep it super simple to get everyone started. At the end, I’ll link more complete documentation to show you the way ahead. Sorry, no Windows.

Tools Used

This is not a history lesson, therefore we will use:

1

$ pip install -U "pip>=1.4" "setuptools>=0.9" "wheel>=0.21"
$ pip install -U "pip>=1.4" "setuptools>=0.9" "wheel>=0.21"

Please make sure all installs succeed, ancient installations may need some extra manual labor.

A Minimal Glimpse Into The Past

Forget that there ever was distribute (cordially merged into setuptools), easy_install (part of setuptools, supplanted by pip), or distutils2 aka packaging (was supposed to be the official thing from Python 3.3 on, didn’t get done in time due to lack of helping hands, got ripped out by a heart-broken Éric and abandoned now).

Be just vaguely aware that there are distutils and distlibsomewhere underneath but ideally it shouldn't matter to you at all for now.

setup.py

Nowadays, every project that you want to package needs asetup.py file. Let’s have a look at what pem’s could look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

from setuptools import setup
 
setup(
    name='pem',
    version='0.1.0',
    description='Parse and split PEM files painlessly.',
    long_description=(open('README.rst').read() + '\n\n' +
                      open('HISTORY.rst').read() + '\n\n' +
                      open('AUTHORS.rst').read()),
    url='http://github.com/hynek/pem/',
    license='MIT',
    author='Hynek Schlawack',
    author_email='hs@ox.cx',
    py_modules=['pem'],
    include_package_data=True,
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Intended Audience :: Developers',
        'Natural Language :: English',
        'License :: OSI Approved :: MIT License',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.6',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
        'Topic :: Software Development :: Libraries :: Python Modules',
    ],
)
from setuptools import setup

setup(
    name='pem',
    version='0.1.0',
    description='Parse and split PEM files painlessly.',
    long_description=(open('README.rst').read() + '\n\n' +
                      open('HISTORY.rst').read() + '\n\n' +
                      open('AUTHORS.rst').read()),
    url='http://github.com/hynek/pem/',
    license='MIT',
    author='Hynek Schlawack',
    author_email='hs@ox.cx',
    py_modules=['pem'],
    include_package_data=True,
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Intended Audience :: Developers',
        'Natural Language :: English',
        'License :: OSI Approved :: MIT License',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.6',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
        'Topic :: Software Development :: Libraries :: Python Modules',
    ],
)

In the simplest case, you just import setup from setuptoolsand run it with some keyword arguments.

Before I go into these arguments, I want to point out one of the most neglected and yet most important one: license.Always set a license! Otherwise nobody can use your module, which would be a pity, right?

I take a few shortcuts here which you may want to copy too:

  • I love DRY, therefore my long description is just a concatenation of my README, change log and author credits.
  • I use bumpversion for manipulating my version strings.

An essential field is py_modules which you'll use to denote the relevant code for packaging, if your package is actually just a single module like in pem’s case. If you have a full-blown package (i.e. a directory), you’ll go for:

1
2
3
4
5
6
7

from setuptools import setup, find_packages
 
setup(
...
    packages=find_packages(exclude=['tests*']),
...
)
from setuptools import setup, find_packages

setup(
...
    packages=find_packages(exclude=['tests*']),
...
)

The exclude makes sure that a top-level tests package doesn’t get installed (it’s still part of the source distribution) since that would wreak havoc.

You can also specify them by hand like I used to fordoc2dash’s setup.py. However, unless you have good reasons, just use find_packages() and be done.

The classifiers field’s usefulness is openly disputed, nevertheless pick them from here. PyPI will refuse to accept packages with unknown classifiers, hence I like to use "Private :: Do Not Upload" for private packages to protect myself from my own stupidity.

One icky thing are dependencies. Unless you really know what you’re doing, don’t pin them (specifying minimal version your package requires to work is fine of course) or your users won’t be able to install security updates of your dependencies:

1
2
3
4
5

setup(
...
    install_requires=['Django', 'django-annoying'],
...
)
setup(
...
    install_requires=['Django', 'django-annoying'],
...
)

Rule of thumb: requirements.txt should contain only ==,setup.py the rest (>=, !=, <=, …).

Non-Code Files

Every Python project has a “Fix MANIFEST.in” commit. Look it up, it’s true.

But it’s not really that hard: you add all files and directories that are not packaged due to py_modules or packages in your setup.py.

For “pem”, it’s just

include *.rst LICENSE

For more commands, have a look at the MANIFEST.in docs – especially recursive-include is used widely. There’s also a handy tool to validate your MANIFEST.in: check-manifest.

Important: If you want the files and directories fromMANIFEST.in to also be installed (e.g. if it’s runtime-relevant data), you will have to set include_package_data=True in yoursetup() call as in the example above!

Configuration

For our minimal Python-only project, we’ll only need two lines in setup.cfg:

1
2

[wheel]
universal = 1
[wheel]
universal = 1

These will make wheel build a universal wheel file (e.g.pem-0.1.0-py2.py3-none-any.whl) and you won’t have to circle through virtual environments of all supported Python versions to build them separately.

Documentation

As I’ve hopefully established, every open source project needs a license. No excuses.

Additionally, even the simplest package needs a README that tells potential users what they’re looking at. Make itreStructuredText (reST) so PyPI can properly render it on your project page. As a courtesy to your users, also keep a change log so they know what to expect from your releases. And finally it’s good style to credit your contributors in a third file. I like to call them README.rst,HISTORY.rst, and AUTHORS.rst respectively, but there’s no hard rule.

My long project description and thus PyPI text is the concatenation of those three (and I’ve shamelessly stolen it from Kenneth Reitz).

If you host on GitHub, you may want to add aCONTRIBUTING.md that gets displayed when someone wants to open a pull request. Unfortunately it has to be a Markdown file because GitHub doesn’t support reST for this. Have a look at pem’s if you need inspiration.

Let’s Build Finally!

Building a source distribution (which is what is usually used for pure-Python project) is just a matter of

1

$ python setup.py sdist
$ python setup.py sdist

Because it’s 2013 and wheels are awesome, we’ll build one too! Fortunately, it’s just as easy (please note that you’ll need to install wheel as per above for this to work!):

1

$ python setup.py bdist_wheel
$ python setup.py bdist_wheel

Of course, the wheel package is optional, but your users will thank you for much faster installation times.

Now you should have a new directory called dist containing a source distribution file and a wheel file. For pem, it currently looks like this:

dist
├── pem-0.1.0-py2.py3-none-any.whl
└── pem-0.1.0.tar.gz

You can test whether both install properly before we move on to uploading:

$ rm -rf 27-sdist  # ensure clean state if ran repeatedly
$ virtualenv 27-sdist
...
$ 27-sdist/bin/pip install --no-index dist/pem-0.1.0.tar.gz
Ignoring indexes: https://pypi.python.org/simple/
Unpacking ./dist/pem-0.1.0.tar.gz
  Running setup.py egg_info for package from file:///Users/hynek/Projects/pem/dist/pem-0.1.0.tar.gz

Installing collected packages: pem
  Running setup.py install for pem

Successfully installed pem
Cleaning up...
$ 27-sdist/bin/python
Python 2.7.4 (default, Apr 21 2013, 09:35:41)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pem
>>> pem.__version__
'0.1.0'

and

$ rm -rf 27-wheel  # ensure clean state if ran repeatedly
$ virtualenv 27-wheel
...
$ 27-wheel/bin/pip install --use-wheel --no-index --find-links dist pem
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking pem
Installing collected packages: pem
Successfully installed pem
Cleaning up...
$ 27-wheel/bin/python
Python 2.7.4 (default, Apr 21 2013, 09:35:41)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pem
>>> pem.__version__
'0.1.0'

Please note the lack of “Running setup.py install for pem” in the second test. Yes, you can finally install packages without executing arbitrary code!

So you’re confident that your package is perfect? Let’s use the Test PyPI server to find out!

The PyPI Staging Server

Again, I’ll be using “pem” as the example project name to avoid <your project name> everywhere.

First, sign up on the test server, you will receive a user name and a password. Please note that this is independent from the live servers. Thus you’ll have to re-register both yourself and your packages. It also gets cleaned from time to time so don’t be surprised if it suddenly doesn’t know about you or your projects anymore. Just re-register.

Next, create a ~/.pypirc consisting of:

1
2
3
4
5
6
7
8

[distutils]
index-servers=
    test
 
[test]
repository = https://testpypi.python.org/pypi
username = <your user name goes here>
password = <your password goes here>
[distutils]
index-servers=
    test

[test]
repository = https://testpypi.python.org/pypi
username = <your user name goes here>
password = <your password goes here>

Then use

1

$ python setup.py register -r test
$ python setup.py register -r test

to register your project with the PyPI test server.

Finally, upload your distributions:

1
2

$ python setup.py sdist upload -r test
$ python setup.py bdist_wheel upload -r test
$ python setup.py sdist upload -r test
$ python setup.py bdist_wheel upload -r test

Please note that calling upload without building something in the same command line will give you a:

error: No dist file created in earlier command

Now test your packages again:

1

$ pip install -i https://testpypi.python.org/pypi pem
$ pip install -i https://testpypi.python.org/pypi pem

Everything dandy? Then lets tackle the last step: putting it on the real PyPI!

The Final Step

First, register at PyPI, then complete your ~/.pypirc:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

[distutils]
index-servers=
    pypi
    test
 
[test]
repository = https://testpypi.python.org/pypi
username = <your test user name goes here>
password = <your test password goes here>
 
[pypi]
repository = http://pypi.python.org/pypi
username = <your production user name goes here>
password = <your production password goes here>
[distutils]
index-servers=
    pypi
    test

[test]
repository = https://testpypi.python.org/pypi
username = <your test user name goes here>
password = <your test password goes here>

[pypi]
repository = http://pypi.python.org/pypi
username = <your production user name goes here>
password = <your production password goes here>

One last deep breath and let’s rock:

1
2

$ python setup.py sdist upload -r pypi
$ python setup.py bdist_wheel upload -r pypi
$ python setup.py sdist upload -r pypi
$ python setup.py bdist_wheel upload -r pypi

And thus, your package is only a pip install away for everyone! Congratulations, do more of that!

Next Steps

The information herein will probably get you pretty far but if you get stuck, the current canonical truths for Python packaging are:

Please don’t let your software rot on GitHub or launchpad! Share!

Thanks

This article has been kindly proof-read by Lynn Root,Donald Stufft, Alex Gaynor, Thomas Heinrichsdobler, andJannis Leidel. All mistakes are still mine though.

I’d be happy if you have any feedback to make this article even more straight-forward for people new to Python packaging!

文章出自：sharing-your-labor-of-love-pypi-quick-and-dirty

返回

作者:

该日志由 润物无声 于11年前发表在程序设计分类下， 你可以发表评论，并在保留原文地址及作者的情况下引用到你的网站或博客。
转载请注明: 分享Python模块到PyPI | 润物无声 +复制链接
关键字: ,
【上篇】
【下篇】

您可能还会对这些文章感兴趣！

  1. Python开发之性能优化
  2. 搭建基于Python的视频服务器
  3. Python模块发布文件setup.py vs requirements.txt
  4. Python函数调用的自动化内联机制
  5. python 脚本登录百度空间
  6. 混乱代码分享之 Python 程序设计
  7. 为什么每个程序设计者都应该学习 Python 或者 Ruby
  8. 如何成为Python高手
深度解读Android关机过程
jQuery Mobile 教程
探索GAE背后的奥秘之GAE简介

给我留言

留言无头像?


插入图片

返回首页

Copyright © 2011-2014 润物无声  保留所有权利.   基于 WordPress 技术创建   Theme by Robin   Modify by Carey   辽ICP备2021004474号
×
新浪微博
腾讯微博