Julien Tanay

Posted on Nov 22, 2017 • Originally published at Medium

Pipfile and Pipenv : the future of Python dependencies management

#python #pipfile #pipenv #dependencies

Yes, I heared you. pip is a great tool and has been around for quite a long time. But for 3 years or so, people (contributors) have been looking for a way to enhance our packages management experience. Think about the superpowers of composer, npm (or better, yarn) in your favorite tool.

What they offer is (more or less) a replacement for the age-old requirements.txt file : the Pipfile.

How I learn to stop worrying and love pip (again)

The new Pipfile differs from the old-style requirements file in several ways :

It uses TOML syntax to allow more configuration;
It declares dependencies groups (i.e. a default one plus a development one for... development packages) so you don't have to split your requirements into several files.
It comes with a Pipfile.lock file which pins the versions of your packages, in addition to some security bonus.

Let's have a look at a minimal, simple Pipfile:

[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

[requires]
python_version = '3.6'

[packages]
flask = '*'

[dev-packages]
pytest = '*'

What do we have here :

First we declare the source that pip is going to use. Typically, it will be pypi.
Then we declare our Python version requirement.
Finally, we declare our packages and development packages.

You got it ? Now here is the example Pipfile from the official repository on GitHub :

[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

[requires]
python_version = '2.7'

[packages]
requests = { extras = ['socks'] }
records = '>0.5.0'
django = { git = 'https://github.com/django/django.git', ref = '1.11.4', editable = true }
"e682b37" = {file = "https://github.com/divio/django-cms/archive/release/3.4.x.zip"}
"e1839a8" = {path = ".", editable = true}
pywinusb = { version = "*", os_name = "=='nt'", index="pypi"}

[dev-packages]
nose = '*'
unittest2 = {version = ">=1.0,<3.0", markers="python_version < '2.7.9' or (python_version >= '3.0' and python_version < '3.4')"}

See ? With the TOML syntax, you can be very specific about the version, source or os-variant you want to use. Note that all packages versions are not pinned, even if it was the preferred way of declaring dependencies.

Pip is going to support this specification in the furure with a syntax like pip install -p Pipfile. Let's see how we can use all this magical stuff today.

Back to the Future

Meet Pipenv (yep, that's another project from Kenneth Reitz, the guy behind httpbin, pep8.org, and, yes, request).

Julien Tanay

@djiit

the fuck when you realize @kennethreitz 's pipenv just dropped a 🎃 in your shell. Made my day.

13:14 PM - 31 Oct 2017

Pipenv is the current preferred implementation of Pipfile and allows you to manage your virtualenvs like a boss. It aims to ease and enhance how we manage our python environments (a la virtualenvwrapper, but with pip skills).

Here is a simple workflow :

1- In your project directory, create your virtualenv. Pipenv will store it in a centralized place instead of your project root directory :

pipenv --three # Yeah, 3 is the way to go

2- Install some dependencies :

pipenv install flask
pipenv install --dev pytest # Notice the --dev flag

3- Use your virtualenv. It will open a new shell :

pipenv shell

Note that we were not in our virtualenv before this command. Everything is handled for us by Pipenv.

From this point, you are good to go. Pipenv will help you manage your environment throughout your development process.

Finally, you shouldn't be worried about compatibility issues with your deployment server (which is surely still using pip as you are reading this) : you can generate a good old requirements.txt by running :

pipenv lock -r > requirements.txt

Pipenv handles your dependencies and it even adds some security reinforcements by specifying each package's hash.

That's it. Go test Pipenv and tell me what you think !

Cheers

NOTE: This post was originally posted on Medium.

Top comments (11)

Raphael Dumas • Nov 28 '17 • Edited

I wanted to have pipenv shell be the default integrated terminal in VSCode on Windows. Here are the settings I modified:

    "terminal.integrated.shell.windows": "C:/Anaconda3/Scripts/pipenv.exe",
    "terminal.integrated.shellArgs.windows": ["shell"]

This works when an integrated terminal is opened. However, there doesn't seem to be saved history within a pipenv shell session (there would normally be history saved for a cmd session).

Is there a way to set up a command history so you can up-arrow for previous commands?

Julien Tanay • Dec 5 '17

Hi there! On Unix you could try something like SHELL=/usr/bin/zsh pipenv shell. Sadly I don't know the Windows equivalent, but the idea is : force-set your shell when running the pipenv cmd.

Cheers

Raphael Dumas • Dec 5 '17

Oops, my comment was incomplete. I meant it works in VSCode in Windows, but my outstanding issue is a lack of command history in the pipenv shell

rhymes • Nov 22 '17

Loving pipenv (+ pyenv for multiple Python versions) so far.

The only thing that seems to be lacking is I can't find an easy way to know which packages have new available versions (like "bundle outdated" or "yarn outdated").

Did you find a solution?

Julien Tanay • Nov 22 '17

There is pipenv check which uses the safety package from the guys at pyup.io and checks if there is any security update on major libraries -- but it won't cover all the packages updates.