Why Python package?
Python supports all types of execution; you can run Python code directly into a shell, or put your code into file and run later.
Sometimes to start a new Python project is very hard; Write a script? Write a module? Write a package?
The best choice is micropiecies pattern: write a script, so rewrite that in a module and so rewrite in a package.
This pattern permits you to don't re-inventing the wheel every day and you reuse the code in the future.
Python package structure
Python package has this structure:
pkg
├── __init__.py
├── module1.py
└── subpkg
├── __init__.py
├── __main__.py
└── module2.py
The folder pkg is a package, because contains __init__.py module. Also the folder subpkg is a package; is a subpackage of pkg.
module1.py and module2.py are a modules of their packages.
The __main__.py module permits the execution of package.
Only here? Other things?
If you will become a Python developer, then you use other usually tools.
In order, you follow this steps every piece of code you write:
- Write Python code into your package
- Track your modifications
- Test all code that you write
- Put your code into environment that you test it
- Push your code in remote repository
- Build your package for distribution
- Upload your package into PyPi
Pipelines
Every change in your code, may introduces possible bugs. To discard this, every time we need test own package in the correct environment.
To do this, some tools are needed over Python itself, like git, docker and make.
Documentation, license and other common files
It is not enough to simply create a Python package and make it immediately available to everyone. You also have to think about how to document it, explain it briefly to other people, license it and explain how to integrate into the project.
This requires developing files like README, LICENSE, CODE_OF_CONDUCT and CONTRIBUTING.
Maybe adding a CHANGELOG to keep and have others keep track of the changes made to each version.
Create project in a few minutes
To realize all parts of a Python project, serves some hours or days.
But exists a tool for this purpose: psp.
After we follow the installation instructions:
[test@ubuntu ~] sudo apt install -y python3 python3-pip git curl
[test@ubuntu ~] curl -L https://github.com/MatteoGuadrini/psp/releases/download/v0.1.0/psp.deb -o psp.deb
[test@ubuntu ~] sudo dpkg -i psp.deb
run it:
[test@ubuntu ~] psp
Welcome to PSP (Python Scaffolding Projects): 0.1.0
> Name of Python project: app
> Do you want to create a virtual environment? Yes
> Do you want to start git repository? Yes
> Select git remote provider: Gitlab
> Username of Gitlab: test_user
> Do you want unit test files? Yes
> Install dependencies: flask
> Select documention generator: MKDocs
> Do you want to configure tox? Yes
> Select remote CI provider: CircleCI
> Do you want create common files? Yes
> Select license: Gnu Public License
> Do you want to install dependencies to publish on pypi? Yes
> Do you want to create a Dockerfile and Containerfile? Yes
Python project `app` created at app
Now, check the Python project that has been created:
[test@ubuntu ~] ls -lah app
total 88K
drwxrwxr-x 9 test test 440 Dec 20 14:48 .
drwxrwxrwt 29 root root 680 Dec 20 14:49 ..
drwxrwxr-x 2 test test 60 Dec 20 14:47 .circleci
drwxrwxr-x 7 test test 200 Dec 20 14:47 .git
-rw-rw-r-- 1 test test 381 Dec 20 14:47 .gitignore
drwxrwxr-x 4 test test 80 Dec 20 14:47 .gitlab
-rw-rw-r-- 1 test test 127 Dec 20 14:48 CHANGES.md
-rw-rw-r-- 1 test test 5.4K Dec 20 14:48 CODE_OF_CONDUCT.md
-rw-rw-r-- 1 test test 1.1K Dec 20 14:48 CONTRIBUTING.md
-rw-rw-r-- 1 test test 190 Dec 20 14:48 Containerfile
-rw-rw-r-- 1 test test 190 Dec 20 14:48 Dockerfile
-rw-rw-r-- 1 test test 35K Dec 20 14:48 LICENSE.md
-rw-rw-r-- 1 test test 697 Dec 20 14:48 Makefile
-rw-rw-r-- 1 test test 177 Dec 20 14:48 README.md
drwxrwxr-x 2 test test 60 Dec 20 14:47 docs
-rw-rw-r-- 1 test test 19 Dec 20 14:47 mkdocs.yml
-rw-rw-r-- 1 test test 819 Dec 20 14:48 pyproject.toml
-rw-rw-r-- 1 test test 66 Dec 20 14:47 requirements.txt
drwxrwxr-x 2 test test 80 Dec 20 14:47 tests
-rw-rw-r-- 1 test test 213 Dec 20 14:47 tox.ini
drwxrwxr-x 2 test test 80 Dec 20 14:46 app
drwxrwxr-x 5 test test 140 Dec 20 14:46 venv
Start develop the package
Start develop the package that psp command has created for our project.
[test@ubuntu ~] cd app/ && ls -lh app/
total 8.0K
-rw-rw-r-- 1 test test 162 Dec 20 14:46 __init__.py
-rw-rw-r-- 1 test test 204 Dec 20 14:46 __main__.py
[test@ubuntu ~] vim app/core.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello_world():
return "<p>Wow, this is my app!</p>"
Now, import our hello_world function into __main__.py file:
[test@ubuntu app] vim app/__main__.py
#! /usr/bin/env python3
# -*- encoding: utf-8 -*-
# vim: se ts=4 et syn=python:
# Generated by psp (https://github.com/MatteoGuadrini/psp)
from .__init__ import __version__
print(f'app {__version__}')
from .core import app
app.run(debug=True)
Run our package
You have write a simple, but organized and powerful package that is ready to production and distribution.
Test our package.
[test@ubuntu app] venv/bin/python -m app
app 0.0.1
* Serving Flask app 'app.core'
* Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
* Restarting with watchdog (inotify)
app 0.0.1
* Debugger is active!
* Debugger PIN: 352-398-739
127.0.0.1 - - [20/Dec/2024 15:21:27] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [20/Dec/2024 15:21:28] "GET /favicon.ico HTTP/1.1" 404 -
^C
And result is:
Execute unit tests on package
Now tests also the Python code on the package, throught tests folder:
[test@ubuntu app] ls -l tests
total 8
-rw-rw-r-- 1 test test 141 Dec 20 14:59 __init__.py
-rw-rw-r-- 1 test test 465 Dec 20 14:59 test_app.py
[test@ubuntu app] venv/bin/python3 -m unittest
Test all app successfully!
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Save our works
Now you can save the development of your web app.
[test@ubuntu app] git status
On branch master
No commits yet
Untracked files:
(use "git add <file>..." to include in what will be committed)
.circleci/
.gitignore
.gitlab/
CHANGES.md
CODE_OF_CONDUCT.md
CONTRIBUTING.md
Containerfile
Dockerfile
LICENSE.md
Makefile
README.md
app/
docs/
mkdocs.yml
pyproject.toml
requirements.txt
tests/
tox.ini
nothing added to commit but untracked files present (use "git add" to track)
[test@ubuntu app] git add -A && git commit -m "chore: hello app"
[master (root-commit) 96db65f] chore: hello app
23 files changed, 1126 insertions(+)
create mode 100644 .circleci/config.yml
create mode 100644 .gitignore
create mode 100644 .gitlab/issue_templates/bug.md
create mode 100644 .gitlab/issue_templates/feature.md
create mode 100644 .gitlab/merge_request_templates/merge.md
create mode 100644 CHANGES.md
create mode 100644 CODE_OF_CONDUCT.md
create mode 100644 CONTRIBUTING.md
create mode 100644 Containerfile
create mode 100644 Dockerfile
create mode 100644 LICENSE.md
create mode 100644 Makefile
create mode 100644 README.md
create mode 100644 app/__init__.py
create mode 100644 app/__main__.py
create mode 100644 app/core.py
create mode 100644 docs/index.md
create mode 100644 mkdocs.yml
create mode 100644 pyproject.toml
create mode 100644 requirements.txt
create mode 100644 tests/__init__.py
create mode 100644 tests/test_app.py
create mode 100644 tox.ini
[test@ubuntu app] git log
author: test_user <*******>
Date: Fri Dec 20 15:35:10 2024 +0100
chore: hello app
(END)
[test@ubuntu app] git push --set-upstream origin master && git push
...
Test environment
Simulate with docker your production environment:
[test@ubuntu app] docker build . -t app:0.0.1
STEP 1/6: FROM python:3
STEP 2/6: COPY app /app/app
--> cf0db0178706
STEP 3/6: COPY pyproject.toml /app
--> 363ca0249ca4
STEP 4/6: WORKDIR /app
--> 6167fc048a28
STEP 5/6: RUN pip install .
Processing /app
...
--> baf94c942fb9
STEP 6/6: CMD [ 'python', '-m', 'app' ]
COMMIT app:0.0.1
--> e8afe180c81d
Successfully tagged localhost/app:0.0.1
e8afe180c81d0897173885edf4abaf8c92e60e0fbd9548d1f5a40711085421c1
[test@ubuntu app] docker run -it --rm localhost/app:0.0.1 python -m app
app 0.0.1
* Serving Flask app 'app.core'
* Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
* Restarting with stat
app 0.0.1
* Debugger is active!
* Debugger PIN: 115-031-071
^C
And the result is the same:
Build the pipeline with make
Now, after the next development, you can use the pipeline with Makefile:
[test@ubuntu app] make clean test run
rm -fr build/
rm -fr dist/
rm -fr .eggs/
find . -name '*.egg-info' -exec rm -fr {} +
find . -name '*.egg' -exec rm -f {} +
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -fr {} +
venv/bin/python3 -m unittest
Test all app successfully!
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
venv/bin/python3 -m app
app 0.0.1
* Serving Flask app 'app.core'
* Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
* Restarting with watchdog (inotify)
app 0.0.1
* Debugger is active!
* Debugger PIN: 352-398-739
^C
Publish the package on PyPi
Now, if you want, you are ready to publish youp Python package on PyPi:
[test@ubuntu app] python -m build && python -m twine upload dist/*
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
- setuptools
- wheel
* Getting build dependencies for sdist...
running egg_info
writing app.egg-info/PKG-INFO
writing dependency_links to app.egg-info/dependency_links.txt
writing requirements to app.egg-info/requires.txt
writing top-level names to app.egg-info/top_level.txt
reading manifest file 'app.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'app.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing app.egg-info/PKG-INFO
writing dependency_links to app.egg-info/dependency_links.txt
writing requirements to app.egg-info/requires.txt
writing top-level names to app.egg-info/top_level.txt
reading manifest file 'app.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'app.egg-info/SOURCES.txt'
running check
creating app-0.0.1
creating app-0.0.1/app
creating app-0.0.1/app.egg-info
creating app-0.0.1/tests
copying files to app-0.0.1...
copying LICENSE.md -> app-0.0.1
copying README.md -> app-0.0.1
copying pyproject.toml -> app-0.0.1
copying app/__init__.py -> app-0.0.1/app
copying app/__main__.py -> app-0.0.1/app
copying app/core.py -> app-0.0.1/app
copying app.egg-info/PKG-INFO -> app-0.0.1/app.egg-info
copying app.egg-info/SOURCES.txt -> app-0.0.1/app.egg-info
copying app.egg-info/dependency_links.txt -> app-0.0.1/app.egg-info
copying app.egg-info/requires.txt -> app-0.0.1/app.egg-info
copying app.egg-info/top_level.txt -> app-0.0.1/app.egg-info
copying tests/test_app.py -> app-0.0.1/tests
copying app.egg-info/SOURCES.txt -> app-0.0.1/app.egg-info
Writing app-0.0.1/setup.cfg
Creating tar archive
removing 'app-0.0.1' (and everything under it)
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
- setuptools
- wheel
* Getting build dependencies for wheel...
running egg_info
writing app.egg-info/PKG-INFO
writing dependency_links to app.egg-info/dependency_links.txt
writing requirements to app.egg-info/requires.txt
writing top-level names to app.egg-info/top_level.txt
reading manifest file 'app.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'app.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build/lib/app
copying app/__init__.py -> build/lib/app
copying app/__main__.py -> build/lib/app
copying app/core.py -> build/lib/app
running egg_info
writing app.egg-info/PKG-INFO
writing dependency_links to app.egg-info/dependency_links.txt
writing requirements to app.egg-info/requires.txt
writing top-level names to app.egg-info/top_level.txt
reading manifest file 'app.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'app.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/app
copying build/lib/app/__init__.py -> build/bdist.linux-x86_64/wheel/./app
copying build/lib/app/__main__.py -> build/bdist.linux-x86_64/wheel/./app
copying build/lib/app/core.py -> build/bdist.linux-x86_64/wheel/./app
running install_egg_info
Copying app.egg-info to build/bdist.linux-x86_64/wheel/./app-0.0.1-py3.12.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/app-0.0.1.dist-info/WHEEL
creating '/tmp/app/dist/.tmp-bf8qnjot/app-0.0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'app/__init__.py'
adding 'app/__main__.py'
adding 'app/core.py'
adding 'app-0.0.1.dist-info/LICENSE.md'
adding 'app-0.0.1.dist-info/METADATA'
adding 'app-0.0.1.dist-info/WHEEL'
adding 'app-0.0.1.dist-info/top_level.txt'
adding 'app-0.0.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built app-0.0.1.tar.gz and app-0.0.1-py3-none-any.whl
Uploading distributions to https://upload.pypi.org/legacy/
Uploading app-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 kB • 00:00 • 11.3 MB/s
...
Conslusion
In less than five minutes, you have created a Python project where the development of the package itself is the only thing you have to worry about.
Tools used on this article:
psp: repository -- docs
git: repository -- docs
docker: repository -- docs
make: repository -- docs
python: repository -- docs
Top comments (0)