We are going to use poetry to structure our Machine Learning project.
Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs, and can build your project for distribution.
Installation
You should note that:
Poetry should always be installed in a dedicated virtual environment to isolate it from the rest of your system. It should in no case be installed in the environment of the project that is to be managed by Poetry. This ensures that Poetry’s own dependencies will not be accidentally upgraded or uninstalled. (Each of the following installation methods ensures that Poetry is installed into an isolated environment.) In addition, the isolated virtual environment in which poetry is installed should not be activated for running poetry commands.
pip install poetry
Basic Usage
First, let’s create our new project, let’s call it poetry-demo
poetry new poetry-demo
This will create the poetry-demo directory with the following content:
poetry-demo
├── pyproject.toml
├── README.md
├── poetry_demo ← — — — — All you script file are written here
│ └── init.py
└── tests
└── init.py
The pyproject.toml file is what is the most important here. This will orchestrate your project and its dependencies. For now, it looks like this:
[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = ""
authors = ["Sébastien Eustace <sebastien@eustace.io>"]
readme = "README.md"
packages = [{include = "poetry_demo"}]
[tool.poetry.dependencies]
python = "^3.8"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
If you want to add any dependencies, you can use the simple command below:
poetry add pendulum
If you are using notebooks you can add a folder named notebook and alter you project to look like this :
poetry-demo
├── pyproject.toml
├── README.md
├── poetry_demo ← — — — — All you script file are written here
│ └── init.py
└── tests
└── init.py
└── notebook
└── notebook.ipynb
Dockerization
Dockerization, also known as “containerization,” refers to the process of packaging an application and its dependencies into a standardized container called a “Docker container.”
For out case we prepare a Dockerfile
# Use an official Python Runtime as a parent image
FROM Python:3.9-slim
#set the working directory in the container
WORKDIR /app
# Copy the current directory contents into the container /app
COPY ..
#Install sytem dependencies
RUN apt-get update && apt-get install -y \ curl \ && rm -rf /var/lib/apt/lists/*
# Install poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
# Use poetry to install python dependencies
RUN /root/.poetry/bin/poetry config virtualenvs.create false \ && /root/.poetry/bin/poetry install --no-interaction --no-ansi
# Specify the command to run on start
CMD ['python', 'poetry-demo/train.py']
poetry-demo
├── pyproject.toml
├── Dockerfile
├── README.md
├── poetry_demo ← — — — — All you script file are written here
│ └── init.py
└── tests
└── init.py
└── notebook
└── notebook.ipynb
Then run
docker build -t IMAGE-NAME
to build your image then run your docker image
docker run IMAGE-NAME
These is How to structure you Machine Learning project, give me a clap or incase you have any question or something to add up reply it here and I will reach out Thank you.
Top comments (0)