Please open notebook rsepython-s3r4.ipynb
Once we’ve made a working program, we’d like to be able to share it with others.
A good cross-platform build tool is the most important thing: you can always have collaborators build from source.
Distribution tools allow one to obtain a working copy of someone else’s package.
Until recently windows didn’t have anything like brew install or apt-get.
You had to build an ‘installer’, but now there is: https://chocolatey.org.
When planning to package a project for distribution, defining a suitable project layout is essential.
%%bash
tree --charset ascii greetings -I "doc|build|Greetings.egg-info|dist|*.pyc"
greetings
|– CITATION.md
|– LICENSE.md
|– README.md
|– conf.py
|– greetings
| |– init.py
| |– pycache
| |– command.py
| |– greeter.py
| `– test
| |– init.py
| |– fixtures
| | `– samples.yaml
| `– test_greeter.py
|– index.rst
|– scripts
| `– greet
`– setup.py
5 directories, 13 files
We can start by making our directory structure.
You can create many nested directories at once using the -p switch on mkdir.
%%bash
mkdir -p greetings/greetings/test/fixtures
mkdir -p greetings/scripts
To make python code into a package, we have to write a setup.py file:
from setuptools import setup, find_packages
setup(
name="Greetings",
version="0.1.0",
packages=find_packages(exclude=['*test']),
)
We can now install this code with
pip install
And the package will then be available to use everywhere on the system.
from greetings.greeter import greet
greet("UCL","RITS")
‘Hey, UCL RITS.’
Of course, there’s more to do when taking code from a quick script and turning it into a proper module:
We need to add docstrings to our functions, so people can know how to use them.
%%writefile greetings/greetings/greeter.py
def greet(personal, family, title="", polite=False):
""" Generate a greeting string for a person.
Parameters
----------
personal: str
A given name, such as Will or Jean-Luc
family: str
A family name, such as Riker or Picard
title: str
An optional title, such as Captain or Reverend
polite: bool
True for a formal greeting, False for informal.
Returns
-------
string
An appropriate greeting
"""
greeting= "How do you do, " if polite else "Hey, "
if title:
greeting+=title+" "
greeting+= personal + " " + family +"."
return greeting
Overwriting greetings/greetings/greeter.py
import greetings
help(greetings.greeter.greet)
Help on function greet in module greetings.greeter:
greet(personal, family, title=’’, polite=False)
Generate a greeting string for a person.
Parameters
———-
personal: str
A given name, such as Will or Jean-Luc
family: str
A family name, such as Riker or Picard
title: str
An optional title, such as Captain or Reverend
polite: bool
True for a formal greeting, False for informal.
Returns
——-
string
An appropriate greeting
The documentation string explains how to use the function; don’t worry about this for now, we’ll revisit this later.
%%writefile greetings/greetings/command.py
from argparse import ArgumentParser
from .greeter import greet # Note python 3 relative import
def process():
parser = ArgumentParser(description = "Generate appropriate greetings")
parser.add_argument('--title', '-t')
parser.add_argument('--polite', '-p', action="store_true")
parser.add_argument('personal')
parser.add_argument('family')
arguments= parser.parse_args()
print(greet(arguments.personal, arguments.family,
arguments.title, arguments.polite))
if __name__ == "__main__":
process()
Overwriting greetings/greetings/command.py
We use the setup.py file to specify the packages we depend on:
setup(
name = "Greetings",
version = "0.1.0",
packages = find_packages(exclude=['*test']),
install_requires = ['argparse']
)
This allows us to create a command to execute part of our library. In this case when we execute greet on the terminal, we will be calling the process function under greetings/command.py.
%%writefile greetings/setup.py
from setuptools import setup, find_packages
setup(
name = "Greetings",
version = "0.1.0",
packages = find_packages(exclude=['*test']),
install_requires = ['argparse'],
entry_points={
'console_scripts': [
'greet = greetings.command:process'
]})
Overwriting greetings/setup.py
And the scripts are now available as command line commands:
%%bash
greet --help
usage: greet [-h] [–title TITLE] [–polite] personal family
Generate appropriate greetings
positional arguments:
personal
family
optional arguments:
-h, –help show this help message and exit
–title TITLE, -t TITLE
–polite, -p
%%bash
greet Terry Gilliam
greet --polite Terry Gilliam
greet Terry Gilliam --title Cartoonist
Hey, Terry Gilliam.
How do you do, Terry Gilliam.
Hey, Cartoonist Terry Gilliam.
We could now submit “greeter” to PyPI for approval, so everyone could pip install it.
However, when using git we don’t even need to do that, we can install directly from any git URL:
pip install git+git://github.com/ucl-rits/greeter
%%bash
greet Lancelot the-Brave --title Sir
Hey, SirLancelot the-Brave.
For example:
%%writefile greetings/README.md
Greetings!
==========
This is a very simple example package used as part of the UCL Research Software Engineering with Python course: [http://rits.github-pages.ucl.ac.uk/research-se-python/index.html](http://rits.github-pages.ucl.ac.uk/research-se-python/index.html).
Usage:
Invoke the tool with `greet <FirstName> <Secondname>`
Overwriting greetings/README.md
For example:
%%writefile greetings/LICENSE.md
(C) University College London 2014
This "greetings" example package is granted into the public domain.
Overwriting greetings/LICENSE.md
e.g.:
%%writefile greetings/CITATION.md
If you wish to refer to this course, please cite the URL
http://rits.github-pages.ucl.ac.uk/research-se-python/index.html
Portions of the material are taken from Software Carpentry
http://swcarpentry.org
Overwriting greetings/CITATION.md
You may well want to formalise this using the codemeta.json standard or the citation file format - these doesn’t have wide adoption yet, but we recommend them.
%%bash
touch greetings/greetings/test/__init__.py
touch greetings/greetings/__init__.py
Separating the script from the logical module made this possible:
%%writefile greetings/greetings/test/test_greeter.py
import yaml
import os
from ..greeter import greet
def test_greeter():
with open(os.path.join(os.path.dirname(__file__),
'fixtures','samples.yaml')) as fixtures_file:
fixtures=yaml.load(fixtures_file)
for fixture in fixtures:
answer=fixture.pop('answer')
assert greet(**fixture) == answer
Overwriting greetings/greetings/test/test_greeter.py
Add a fixtures file:
%%writefile greetings/greetings/test/fixtures/samples.yaml
- personal: UCL
family: RITS
answer: "Hey, UCL RITS."
- personal: UCL
family: RITS
polite: True
answer: "How do you do, UCL RITS."
- personal: UCL
family: RITS
title: Dr
answer: "Hey, Dr UCL RITS."
Overwriting greetings/greetings/test/fixtures/samples.yaml
%%bash
py.test
============================= test session starts ==============================
platform darwin – Python 3.5.2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /Users/ucl-rits/devel/rsdt/rsd-engineeringcourse/ch04packaging, inifile:
collected 1 items
greetings/greetings/test/test_greeter.py .
=========================== 1 passed in 0.11 seconds ===========================
However, this hasn’t told us that also the third test is wrong! A better approach is to parameterize the test as follows:
%%writefile greetings/greetings/test/test_greeter.py
import yaml
import os
import pytest
from ..greeter import greet
def read_fixture():
with open(os.path.join(os.path.dirname(__file__),
'fixtures',
'samples.yaml')) as fixtures_file:
fixtures = yaml.load(fixtures_file)
return fixtures
@pytest.mark.parametrize("fixture", read_fixture())
def test_greeter(fixture):
answer = fixture.pop('answer')
assert greet(**fixture) == answer
Overwriting greetings/greetings/test/test_greeter.py
Now when we run pytest, we get a failure per element in our fixture and we know all that fails.
%%bash
py.test
============================= test session starts ==============================
platform linux – Python 3.6.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/dvd/Documents/Work/RIST/Teaching/rsd-engineeringcourse/ch04packaging, inifile:
collected 3 items
greetings/greetings/test/test_greeter.py .FF [100%]
=================================== FAILURES ===================================
__________ test_greeter[fixture1] __________
fixture = {‘family’: ‘Chapman’, ‘personal’: ‘Graham’, ‘polite’: True}
@pytest.mark.parametrize(“fixture”, read_fixture())
def test_greeter(fixture):
answer = fixture.pop(‘answer’)
> assert greet(**fixture) == answer
E AssertionError: assert ‘How do you d…aham Chapman.’ == ‘How do you do…aahm Chapman.’
E - How do you do, Graham Chapman.
E ? -
E + How do you do, Graahm Chapman.
E ? +
greetings/greetings/test/test_greeter.py:16: AssertionError
__________ test_greeter[fixture2] __________
fixture = {‘family’: ‘Palin’, ‘personal’: ‘Michael’, ‘title’: ‘CBE’}
@pytest.mark.parametrize(“fixture”, read_fixture())
def test_greeter(fixture):
answer = fixture.pop(‘answer’)
> assert greet(**fixture) == answer
E AssertionError: assert ‘Hey, CBE Michael Palin.’ == ‘Hey, CBE Mike Palin.’
E - Hey, CBE Michael Palin.
E ? ^^^ -
E + Hey, CBE Mike Palin.
E ? ^
greetings/greetings/test/test_greeter.py:16: AssertionError
====================== 2 failed, 1 passed in 0.13 seconds ======================
If you modify your source files, you would now find it appeared as if the program doesn’t change.
That’s because pip install copies the file.
If you want to install a package, but keep working on it, you can do:
pip install --editable
If you’re working in C++ or Fortran, there is no language specific repository. You’ll need to write platform installers for as many platforms as you want to support.
Typically:
dpkg for apt-get on Ubuntu and Debianrpm for yum/dnf on Redhat and Fedorahomebrew on OSX (Possibly macports as well)msi installer for Windows.Homebrew: A ruby DSL, you host off your own webpage.
See an installer for the cppcourse example: http://github.com/jamespjh/homebrew-reactor.
If you’re on OSX, do:
brew tap jamespjh/homebrew-reactor
brew install reactor
Next: Reading - Documentation