Manage Python dependencies

You can manage Python packages in different ways. Python images come with pip installed, but they’re flexible enough so you can choose what package manager you want. This article describes how to configure major package management tools.

This package management is different from global dependencies (packages available as commands), which you can add in your app configuration. See more about managing global dependencies.

Pip

pip is the primary package installer for Python and comes installed on every Python container. You can use it to install packages from the Python Package Index and other locations.

To manage packages with pip, commit a requirements.txt file with all of the dependencies needed for your app. Then install the packages in your build hook, such as by running the following command: pip install -r requirements.txt.

The following sections present ideas to keep in mind to ensure repeatable deployments on Upsun.

pip version

The version of pip on Python containers gets updated regularly. But it isn’t guaranteed to be the latest version or the version that matches your local environment. You might want to define a specific version of pip in your deployments to further enforce repeatable builds.

To do so, modify your app configuration, as in the following examples:

applications:
    # The name of the app container. Must be unique within a project.
    myapp:
        # The location of the application's code.
        source:
            root: "myapp"
       type: 'python:3.9'
        hooks:
            build: |
                # Fail the build if any errors occur
                set -eu
                # Download the latest version of pip
                python3.9 -m pip install --upgrade pip
                # Install dependencies
                pip install -r requirements.txt                

applications:
    # The name of the app container. Must be unique within a project.
    myapp:
        # The location of the application's code.
        source:
            root: "myapp"
       type: 'python:3.9'
        variables:
            env:
                PIP_VERSION: '22.3.1'
        hooks:
            build: |
                # Fail the build if any errors occur
                set -eu
                # Download a specific version of pip
                python3.9 -m pip install pip==$PIP_VERSION
                # Install dependencies
                pip install -r requirements.txt                

pip freeze

You can write requirements.txt files in various ways. You can specify anything from the latest major to a specific patch version in a requirement specifier. Use pip freeze before committing your requirements to pin specific package versions. This ensures repeatable builds on Upsun with the same packages.

Pipenv

Pipenv is a package manager for Python that creates and manages a virtual environment for Python projects. Dependencies are tracked and defined within a Pipfile. It also generates a Pipfile.lock file to produce repeatable installs.

You can specify the latest or a specific version of Pipenv in your deployments to ensure repeatable builds. Because Pipenv depends on pip, you might want to also specify the pip version.

applications:
    # The name of the app container. Must be unique within a project.
    myapp:
        # The location of the application's code.
        source:
            root: "myapp"
       type: 'python:3.9'
        dependencies:
            python3:
                pipenv: '*'
        hooks:
            build: |
                # Fail the build if any errors occur
                set -eu
                # Download the latest version of pip
                python3.9 -m pip install --upgrade pip
                # Install dependencies
                # Include `--deploy` to fail the build if `Pipfile.lock` isn't up to date
                pipenv install --deploy                

applications:
    # The name of the app container. Must be unique within a project.
    myapp:
        # The location of the application's code.
        source:
            root: "myapp"
       type: 'python:3.9'
        variables:
            env:
                PIP_VERSION: '22.3.1'
        dependencies:
            python3:
                pipenv: '2022.12.19'
        hooks:
            build: |
                # Fail the build if any errors occur
                set -eu
                # Download a specific version of pip
                python3.9 -m pip install pip==$PIP_VERSION
                # Install dependencies
                # Include `--deploy` to fail the build if `Pipfile.lock` isn't up to date
                pipenv install --deploy                

Poetry

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and manages them for you. Poetry offers a lock file to ensure repeatable installs and can build your project for distribution. It also creates and manages virtual environments to keep project work isolated from the rest of your system.

To set up Poetry on Upsun, follow these steps:

1. Configure your virtual environment by setting two variables in your app configuration.

   • POETRY_VIRTUALENVS_IN_PROJECT: Setting this to true places the virtual environment at the root of the app container: /app/.venv.
   • POETRY_VIRTUALENVS_CREATE: Setting this to true ensures that the same virtual environment created during the build hook is reused in subsequent steps.

   Set the variables as follows:

applications:
    # The app's name, which must be unique within the project.
    app:
        type: 'python:3.9'
        variables:
            env:
                POETRY_VIRTUALENVS_IN_PROJECT: true
                POETRY_VIRTUALENVS_CREATE: true

2. Install Poetry. You can specify the latest or a specific version of Poetry in your deployments to ensure repeatable builds.

   • Latest version
   • Specific version

   .upsun/config.yaml

   applications:
       # The name of the app container. Must be unique within a project.
       myapp:
           # The location of the application's code.
           source:
               root: "myapp"
           type: 'python:3.9'
           dependencies:
               python3:
                   poetry: '*'
           variables:
               env:
                   POETRY_VIRTUALENVS_IN_PROJECT: true
                   POETRY_VIRTUALENVS_CREATE: true
           hooks:
               build: |
                   # Fail the build if any errors occur
                   set -eu
                   # Download the latest version of pip
                   python3.9 -m pip install --upgrade pip
                   # Install dependencies
                   poetry install                

   .upsun/config.yaml

   applications:
       # The name of the app container. Must be unique within a project.
       myapp:
           # The location of the application's code.
           source:
               root: "myapp"
           type: 'python:3.9'
           dependencies:
               python3:
                   poetry: '>=1.8'
           variables:
               env:
                   POETRY_VIRTUALENVS_IN_PROJECT: true
                   POETRY_VIRTUALENVS_CREATE: true
           
           hooks:
               build: |
                   # Fail the build if any errors occur
                   set -eu
                   # Download the latest version of pip
                   python3.9 -m pip install --upgrade pip
                   # Install dependencies
                   poetry install                   

3. Make Poetry available outside the build hook. Although step 2 updated the PATH to make Poetry available during the build hook, it isn’t enough to make it available at subsequent stages.

   To use Poetry in a start command, a deploy hook, or during SSH sessions, update the PATH in a .environment file.

   .environment

   # Updates PATH when Poetry is used, making it available during deploys, start commands, and SSH.
   if [ -n "$POETRY_VERSION" ]; then
       export PATH="/app/.local/bin:$PATH"
   fi