I often use Conda environments when working on my Python projects, as it helps me manage dependencies for projects outside of just pure Python packages. In porting some of these projects to Codespaces and Dev Containers, I have found some tricks to getting the fastest and most reliable experience with Conda and Codespaces.
tl;dr Check out this template repo for a sample Dev Container that is fast and reduces Codespace resource consumption.
Base image size matters
When you start a Codespace for a project, it will try to use whatever Dev Container you have specified in your repo, else it will try to use a kitchen sink container.
That default kitchen sink can be way too much and so if you will be working with Conda environments with an Anaconda or Miniconda Dev Container template instead.
- Tip 1: To use less of your Codespaces resources start with a smaller image like Miniconda or Miniforge and install only what you need.
It is worth noting as well that if you use one of these template docker images or start with the same original docker image on the FROM
line, the memory for those layers is pre-cached in Codespaces and seems to download faster.
Create new environments, don't install into base
The default Anaconda/Miniconda templates in VS Code will look for an environment.yml
file in your project root and then install those requirements in the base environment.
This can run into issues when there are conflicts between what is already in the base Conda environment and what you are trying to install.
-
Tip 2: Change your Dockerfile to
conda create
a new environment as opposed to merging your environment with the base environment to avoid conflicts:
RUN if [ -f "/tmp/conda-tmp/environment.yml" ]; then umask 0002 && /opt/conda/bin/conda env create -f /tmp/conda-tmp/environment.yml; fi \
&& rm -rf /tmp/conda-tmp
Conda process uses lots of memory, gets killed
The other challenge I ran into sometimes was that if I was running a lower memory/storage Codespace instance, when I tried to use Conda from the command line to modify environments, the process would be killed after a few seconds.
This turns out to be related to some performance issues Conda has that make it consume a lot of memory when trying to work with the conda-forge installation channel.
You can always then just increase the size of the Codespace your are working with (just go to your Codespaces list and use the triple dots to change the settings for a Codespace).
There is another option that will fix the performance issues so that it can be run on smaller Codespaces.
Mamba is a faster dependency solver that can wrap Conda and resolve some of these performance issues.
You use all the same Conda syntax for things except use the command mamba
and if it's not something Mamba improves on, it just drops back to conda
.
To use mamba in your Dev Container to speed up working with environments, you will need to install it into your base Conda profile, and the use mamba
to create the Conda environment for your environment.
-
Tip 3: Install and use
mamba
to speed up the Conda environment creation process.
The snippet below is a minimal example of a Dockerfile for your dev containers and Codespaces with all the tips included:
FROM mcr.microsoft.com/devcontainers/miniconda:0-3
RUN conda install -n base -c conda-forge mamba
COPY environment.yml* .devcontainer/noop.txt /tmp/conda-tmp/
RUN if [ -f "/tmp/conda-tmp/environment.yml" ]; then umask 0002 && /opt/conda/bin/mamba env create -f /tmp/conda-tmp/environment.yml; fi \
&& rm -rf /tmp/conda-tmp
You can find a template repo where I have added all of these files into a blank repo that might help test some dev containers and Codespaces yourself!
Other Troubleshooting
Conda not working from the start in the terminal
Sometimes after the Codespace or Dev Container has started, the shell will not have been configured to use the conda
or mamba
commands. If you try to run either conda
or mamba
the error message will prompt you to run conda init
and that should fix it. Make sure after you run the fix that you close that terminal and open a new one for the shell profile to be reloaded.
Conda environments don't show up in the terminal/don't auto activate the right Conda environment
Depending on the loading order of the extensions in VS Code, if it has not seen any Python file or tool in use yet it will not necessarily load the terminal settings for environments right away.
If I open a Jupyter notebook or any Python file that will usually prompt the extension to load the terminal parts and then I just open another terminal window and it's all fixed.
Anyway, hope this helps you get the best Conda experience in Codespaces and Dev Containers! If you have any other questions, tips, or tricks, feel free to reach out to me on Mastodon or in the comments below! đź’–
Top comments (0)