Debian NAS and Docker compose

I just want to start with I am not a Docker expert. I just use Docker as a tool that I use to help me accomplish some tasks. This post is about how I utilize Docker containers to self-host some services that I use for my homelab and home services. Also, how I backup my containers configuration and the Docker compose files.

I also want to add that this is not best practice and it is a method that I use because it makes sense to me. I use SnapRAID to protect my data and mergerfs to simplify the storage management. Docker containers do not like spreading its config file and SnapRAID is not ideal for files that constantly changing. The topology shown in Figure 1 how I have things configured.

In Figure 1, DISK4 is outside of SnapRAID and mergerfs. I am using the DISK4 to store my docker-compose.yml and appdata files. I also plan to use this DISK4 to store VMs, but VM is outside of the scope of this post. Keeping the disk outside of SnapRAID and mergerfs, I addressed the issue with SnapRAID and mergerfs.

I run several Docker containers on my NAS. Instead of keeping a single Docker compose file. I created each container its own directory and its own docker compose file as shown in Figure 2.

Starting and stopping the containers

I have to cd to each /mnt/disk4/docker_compose/<container-name> and execute the command docker-compose up -d. The snippet below how I bring each container up.

cd /mnt/disk4/docker_compose/syncthing
docker-compose up -d

There are instances that I need to stop all the containers. To stop all the containers, I use this bash script.

#!/bin/bash
echo "Stopping the Docker containers..."
echo ""
for i in $(find /mnt/disk4/docker_compose/ -iname docker-compose.yml)
do
  docker-compose -f $i stop
done

This is how I start all the containers manually using a bash script.

#!/bin/bash
echo "Starting the Docker containers..."
echo ""
for i in $(find /mnt/disk13/docker_compose/ -iname docker-compose.yml)
do
  docker-compose -f $i up -d
done

Docker compose files

Now, if you noticed in Figure 2 under the docker_compose directory, I have two files which are the docker-compose.yml and .env files. I use the .env file to store the variables that I use for my docker-compose.yml. Below is a sample of the content of the .env file.

APPDATA=/mnt/disk4/appdata/filebot
DOWNLOADSDIR=/mnt/pool/downloads/
MEDIADIR=/mnt/pool/media/
PUID=1001
PGID=1001
RDPPORT=3390
WEBPORT=8082

Below is a sample of a content of the docker-compose.yml for Filebot container. The sixth line, env_file: .env, basically points the docker-compose file to the location of the .env file.

version: '3'
services:
  filebot:
    container_name: filebot
    image: coppit/filebot
    env_file: .env
    environment:
      - WIDTH=1920
      - HEIGHT=1080
      - USER_ID=${PUID}
      - GROUP_ID=${PGID}
    ports:
      - '${RDPPORT}:3389'
      - '${WEBPORT}:8080'
    volumes:
      - '${APPDATA}:/config:rw'
      - '${DOWNLOADSDIR}:/downloads:rw'
      - '${MEDIADIR}:/media:rw'
    restart: unless-stopped

Backing up the Docker config and compose files

Since the appdata and docker-compose.yml are not protected by the SnapRAID. I have to write a bash script to create a backup of the appdata and docker_compose directories and store these data to the array so that they are protected by SnapRAID. Below is the bash script that I wrote to keep accomplish this task.

#!/bin/bash

# Backing up Docker containers' appdata and docker-compose files
# Stop all the Docker containers
echo "Stopping Docker containers..."

for i in $(find /mnt/disk4/docker_compose/ -iname docker-compose.yml)
  do
    docker-compose -f $i stop
done 

sleep 5s

# Backup docker-compose files
echo "Creating docker-compose backup..."

backup_dc="$(date +'$Y-$m-%d_%H:%M:%S')_docker_compose.tar.gz"

tar --force-local -zcvf $backup_dc /mnt/disk4/docker_compose

cp $backup_dc /mnt/pool/docker_backup/$backup_dc

orig_dc_tar=($(md5sum $backup_dc))
cp_dc_tar=($(md5sum /mnt/pool/docker_backup/$backup_dc))

echo "Comparing MD5 hash with the $backup_dc"
echo $orig_dc_tar
echo $cp_dc_tar
echo "Removing the original tar file..."

if [ $orig_dc_tar = $cp_dc_tar ]
then
  rm $backup_dc
fi

sleep 5s

# Backup appdata
echo "Creating appdata backup..."
echo "This may take a while..."
echo ""

backup="$(date +'%Y-%m-%d_%H:%M:%S')_appdata.tar.gz"

tar --force-local -zcvf $backup /mnt/disk4/appdata

cp $backup /mnt/pool/docker_backup/$backup

orig_tar=($(md5sum $backup))
cp_tar=($(md5sum /mnt/pool/docker_backup/$backup))

echo "Comparing MD5 hash with the $backup"
echo $orig_tar
echo $cp_tar
echo "Removing the original tar file..."

if [ $orig_tar = $cp_tar ]
then
  rm $backup
fi

sleep 5s

# Start Docker contaners
echo "Starting Docker containers..."
echo ""

for i in $(find /mnt/disk4/docker_compose/ -iname docker-compose.yml)
  do
    docker-compose -f $i start
done

The script’s flow goes like this.

1. Stop all the running container
2. Tar and compress the docker_compose directory
3. Copy the tarball file to the array, so that it would be protected by the SnapRAID
4. Compare the MD5 hash of the original tarball and the copied tarball
5. If the hash are the same, delete the original tarball
   1. If the hash doesn’t match for some reason, do nothing and continue with the script
6. Tar and compress the appdata directory
7. Copy the appdata tarball file to the array
8. Compare the MD5 hash of the original tarball and the copied tarball
9. If the hash are the same, delete the original tarball
   1. If the hash doesn’t match for some reason, do nothing and continue with the script
10. Start all the containers

This is it. I hope you will find this useful.

Cheers!