Table of Contents
Part 2: From Docker Compose to Podman Quadlet: A Practical Guide Using Immich
Posted on :: Linux Immich Podman Quadlets Docker Compose

This is the second part on Podman. In my first post, I described the basics of setting up Quadlets by hand.

In this follow-up I describe how to convert a conventional docker-compose.yaml file into a Quadlet setup by using Immich as an example.

Preface

Docker Compose is the de facto standard for deploying multi-container applications. Although Docker Compose and Podman Quadlet are deliberately very similar in their configuration, the conversion is not yet fully automatic.

I think the main reason for this is that Podman is a fairly new project that not many people know about (and thus far fewer people work on it). However, within the Linux world, I consider Podman to be the far superior approach.

Although the fundamentals are already rock solid, it still lacks a tiny bit of the semantic sugar and automations here and there that Docker Compose offers – but this will eventually be fixed at some point.

Just Go

If you just want to “execute” a compose file, you can simply do that using podman compose up – as simple as that. It’s a drop-in replacement for docker compose up.

podman compose is a thin wrapper that enables external compose providers to communicate with the Podman socket. The default compose providers are docker-compose and podman-compose whereat docker-compose takes precedence as it is the original implementation of the Compose specification. podman-compose on the other hand is a reimplementation of the Compose specification.

Anyway, this is nice for immediate one-shot execution but not suited for long-term deployment.

Preparation

There is a project called podlet to convert Docker Compose files into the Quadlet syntax.1 Although we may still have to manually intervene, it already gets us most of the way.

cat cat

You find everything packed as an Ansible role at the very end.

Step 1 – Setup the Environment Variables

Since Docker is the de-facto deployment standard, we are solely offered a docker-compose.yaml file.

We download the two files from the documentation:

wget https://github.com/immich-app/immich/releases/latest/download/{example.env,docker-compose.yml}

Looking into example.env we find

# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v2.1.0"
IMMICH_VERSION=v2

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

where we configure the timezone and the DB password, e.g. using:

tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c 30; echo

Step 2 – Prepare the Compose File

Have a look at the following two snippets from the Compose file:

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}

There are now two issues we have to take care of:

  1. Podlet rejects the simultaneous definition of tag and digest of images for good reason while Docker Compose allows for it.

So we have to choose between tag (here 14-vectorchord0.4.3-pgvectors0.2.0) and digest/hash (here sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23).

I will go with tags and by looking at ghcr.io/immich-app/postgres, I see that the given version is actually heavily out-dated. Hence I choose to use the current 18-vectorchord0.5.3 instead.

  1. Podlet not yet supports variable replacement (work is on-going).

The variables are simple BASH-style definitions so that we could go with GNU envsubst most of the way

env $(grep -v '^#' example.env | xargs) envsubst < docker-compose.yml

but envsubst solely supports plain variables so that we would need to tweak some of them by hand – e.g. when defaults are used.

I would rather encourage to use the above mentioned podman-compose as it aims to support the full Compose specification:

podman-compose --env-file example.env -f docker-compose.yml config

One final remark: The compose file declares

    env_file:
      - .env

several times but we downloaded the file as example.env. So, either we rename it to .env as the Immich guide suggests or we adapt the docker-compose.yaml file (which is what I will do).

1

We can speculate about whether it goes a similar path to Quadlet and is going to be integrated into Podman at some point …

Simple Deployment

The simplest possible deployment is to put everything together in a pod. This is a direct equivalent of what’s being done in the “official” compose file.

We use

podlet --unit-directory compose --pod docker-compose.yml

and it creates all the relevant files in ~/.config/containers/systemd.

Now just use

systemctl --user daemon-reload
systemctl --user enable --now immich-immich-machine-learning.container

and Immich should start.

However, this is something I don’t want to deploy on my server.

For example, have a look at the following:

    volumes:
      - ./postgres:/var/lib/postgresql/data

  1. The supplied compose file declares relative folders:

    1. Never use relative paths.
    2. This would resolve to ~/.config/containers/systemd/postgres, what is definitely the wrong location for container data.
    3. This fails if the folder does not exist or doesn’t have the right permissions and ownership. You never want to take care of something like this manually.
    4. I want my container data to be well-structured in separate volumes.
    5. (It turns out that the recommended mounting location is meanwhile /var/lib/postgresql.)

  2. I don’t want databases to be accessible from the internet.

  3. I want to outsource the environment variables in a separate file (they contain sensitive information and you can set tighter access-permissions on them).

  4. (In my previous post I wrote that I want to stick to the podman naming convention/refrain from using name declarations, like ContainerName.)

More Advanced Deployment

While still pursuing the “as simple as possible” approach, we can still modify some aspects to get a slightly more sophisticated setup.

Point 1 – Setup systemd Volumes

In the final compose file, replace the volumes – e.g.

  immich-server:
    volumes:
      - ./library:/data

with named Quadlet volume files, like:

  immich-server:
    volumes:
      - immich-data.volume:/data

and manually create the corresponding .volume files:

immich-data.volume

[Unit]
Description=Immich Data Volume

immich-mlcache.volume

[Unit]
Description=Immich Machine Learning Cache Volume

immich-postgres.volume

[Unit]
Description=Immich Postgres DB Volume

Now create the .container files using:

podlet --unit-directory compose docker-compose.yml

The resulting files are:

  • database.container
  • immich-machine-learning.container
  • immich-server.container
  • redis.container

(I prefer a coherent naming convention. --> See point 4)

Point 2 – Setup Networks

In a pod, all services share the same network. I will thus not use a pod (drop the --pod argument) but again create two distinct networks:

immich-external.network

[Unit]
Description=External Network for Immich, Immich ML inference and Postgres DB

immich-internal.network

[Unit]
Description=Internal Network for Immich, Immich ML inference and Postgres DB

[Network]
Internal=true

All resulting .container files will hence get the line

[Container]
Network=immich-internal.network

and only immich-server.container and immich-machine-learning.container also get:

Network=immich-external.network

This way all four containers are interconnected, but only the two containers immich-server.container and immich-machine-learning.container can connect to the outside world.

Unfortunately, the machine learning container needs internet access as it wants to download models from time to time.

Point 3 – Unify Location of Environment Variables

Notice that the Postgres .container file has

Environment=POSTGRES_DB=immich POSTGRES_INITDB_ARGS=--data-checksums POSTGRES_PASSWORD=SECRETPASSWORD POSTGRES_USER=postgres

but all others (but Redis):

EnvironmentFile=example.env

There is an important thing to notice here: There is an inconsistency in the naming scheme: Postgres expects e.g. POSTGRES_PASSWORD (created in the docker-compose.yaml) but all others expect DB_PASSWORD (compare example.env). When we outsource the environment variables to a single file we thus have to include these as well.

example.env

TZ=Europe/Berlin

DB_PASSWORD=SECRETPASSWORD
POSTGRES_PASSWORD=SECRETPASSWORD

DB_USERNAME=postgres
POSTGRES_USER=postgres

DB_DATABASE_NAME=immich
POSTGRES_DB=immich

And adapt the postgres container config:

EnvironmentFile=example.env

Remark: Personally, I prefer to keep ~/.config/containers/systemd clean of everything that is not a quadlet or .service file. Therefore I put all auxiliary files, like example.env, in a separate location. E.g.

EnvironmentFile=%h/.config/containers/immich/immich.env

(Point 4) – Change Container Names

Everything so far was pretty straightforward. This optional section is a bit more entwined.

I wrote that I like to stick to the podman naming conventions: To do so, we have to remove the container_name fields from the Compose file:

yq 'del(.services[].container_name)' docker-compose.yml

This way our services are automatically named after the .service files. I furthermore rename the resulting files

  • database.container --> immich-postgres.container
  • immich-machine-learning.container --> immich-machine-learning.container
  • immich-server.container --> immich.container
  • redis.container --> immich-redis.container

and collect them into a separate folder ~/.config/containers/systemd/immich.

Immich, however makes heavy use of naming defaults. Changing the service names hence implies adding a few variables to example.env to declare the new locations of the respective services:

DB_HOSTNAME=systemd-immich-postgres
REDIS_HOSTNAME=systemd-immich-redis
IMMICH_MACHINE_LEARNING_URL=http://systemd-immich-machine-learning:3003

Finalize

To be able to reach the container from the outside world, you can again just drop in a Caddy configuration file

/etc/caddy/Caddyfile.d/immich.caddyfile

www.ourdomain.com {
    redir https://ourdomain.com{uri} permanent
}

ourdomain.com {
    # activate zstd compression (with gzip fallback) for text-based responses (HTML, CSS, JS, JSON, SVG, etc.)
    encode

    reverse_proxy 127.0.0.1:2283

    header {
        Strict-Transport-Security "max-age=31536000;"
    }
}

and restart Caddy:

sudo systemctl restart caddy

Appendix

Again, you find everything as an Ansible role here.