My Python environment

Virtualenv for the win

After trying to use so many things for this, including conda, Linux in a VM + operating system packages, pyenv and everything else that is out there, for years, I finally settled on venv, requirements.txt and a single virtualenv on my laptop.

Why?

If you have tried to use some of these you might have run into some issues. Conda for example is absolutely nondeterministic, works some times and fails most of the time. I created a ticket about it but the response is to try to turn it off and on again. These developers really like It Crowd.

After wrestling with many of similar issues, finally I decided to go for dead simple approach.

I install Python3 with brew. Create a venv in my home folder and source it.

brew install python3
cd
python3 -m venv venv
. ~/venv/bin/activate.fish

Any time I work with a Python project I only use Python3 (RIP Python2 projects) and install all the packages into that single virtual environment. If there is a conflict I delete the entire directory and install the dependencies of the current project. I usually work on one Python project at a time. This gives me exactly what I need with an easy path to resolve local state issues, dependency problems and only using 1 directory instead of littering around all of my repositories and making backup slower. In production I do the same, I use venv to install the environment and install dependencies there. It does not matter if I use a Docker container or a non-containerized deployment the venv is specific to a single application. This gives my the best of both worlds, developer happiness and optimal environment in production. There are minor problems with this, Python development of AWS Lambda functions is not as easy because the architecture differences between Mac vs AWS Linux. This problem exists in Python which virtual environments have nothing to do with. In that case I usually package up software in a VM + Docker or use something else than Python, something that truly supports multi-platform development: Rust or F# for example.