From Google Analytics to Matomo Part 3

In Part 1 we talked about why we switched from Google Analytics to Matomo. In Part 2, we discussed how we designed the architecture. Finally, here in Part 3 we will look at the Matomo specific changes necessary to support our architecture.

First, we modified the Dockerfile so that we could run commands as part of the container startup. This allows classdojo_entrypoint.sh to run, but the process that the container ultimately creates is the long running apache2-foreground:

# The matomo version here must exactly match the version in the matomo_plugin_download.sh script
FROM matomo:4.2.1
ADD classdojo_entrypoint.sh /classdojo_entrypoint.sh
ADD ./tmp/SecurityInfo /var/www/html/plugins/SecurityInfo
ADD ./tmp/QueuedTracking /var/www/html/plugins/QueuedTracking
ADD ./tmp/dbip-city-lite-2021-03.mmdb /var/www/html/misc/DBIP-City.mmdb
RUN chmod +x /classdojo_entrypoint.sh
ENTRYPOINT ["/classdojo_entrypoint.sh"]
CMD ["apache2-foreground"]

Next, we wrote a script to download plugins and geolocation data, to bake into the Docker image:

#!/bin/sh
set -e

MATOMO_VERSION="4.0.2"

rm -rf ./tmp
mkdir ./tmp/
cd ./tmp/

# This script downloads and unarchives plugins.  These plugins must be activated in the running docker container
# to function, which happens in matomo_plugin_activate.sh
curl -f https://plugins.matomo.org/api/2.0/plugins/QueuedTracking/download/${MATOMO_VERSION} --output QueuedTracking.zip
unzip QueuedTracking.zip -d .
rm QueuedTracking.zip
curl -f https://plugins.matomo.org/api/2.0/plugins/SecurityInfo/download/${MATOMO_VERSION} --output SecurityInfo.zip
unzip SecurityInfo.zip -d .
rm SecurityInfo.zip

curl -f https://download.db-ip.com/free/dbip-city-lite-2021-03.mmdb.gz --output dbip-city-lite-2021-03.mmdb.gz
gunzip dbip-city-lite-2021-03.mmdb.gz

cd ..

Then we write the entrypoint file itself. Since we overwrote the original entrypoint, our entrypoint needs to unpack the Matomo image and fix some permissions first, but then we activate plugins that we want to include:

#!/bin/sh
set -e

if [ ! -e matomo.php ]; then
  tar cf - --one-file-system -C /usr/src/matomo . | tar xf -
  chown -R www-data:www-data .
fi

mkdir -p /var/www/html/tmp/cache/tracker/
mkdir -p /var/www/html/tmp/assets
mkdir -p /var/www/html/tmp/templates_c
chown -R www-data:www-data /var/www/html
find /var/www/html/tmp/assets -type f -exec chmod 644 {} \;
find /var/www/html/tmp/assets -type d -exec chmod 755 {} \;
find /var/www/html/tmp/cache -type f -exec chmod 644 {} \;
find /var/www/html/tmp/cache -type d -exec chmod 755 {} \;
find /var/www/html/tmp/templates_c -type f -exec chmod 644 {} \;
find /var/www/html/tmp/templates_c -type d -exec chmod 755 {} \;

# activate matomo plugins that were downloaded and added to the image
/var/www/html/console plugin:activate SecurityInfo
/var/www/html/console plugin:activate QueuedTracking

exec "$@"

We tie it together with a Makefile to build and publish these Docker images:

build-img:
	sh ./matomo_plugin_download.sh
	docker build . -t classdojo/matomo
	rm -rf ./tmp

push-img:
	docker tag classdojo/matomo:latest xxx.dkr.ecr.us-east-1.amazonaws.com/classdojo/matomo:latest
	docker tag classdojo/matomo:latest xxx.dkr.ecr.us-east-1.amazonaws.com/classdojo/matomo:${BUILD_STRING}
	aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin xxx.dkr.ecr.us-east-1.amazonaws.com
	docker push xxx.dkr.ecr.us-east-1.amazonaws.com/classdojo/matomo:latest
	docker push xxx.dkr.ecr.us-east-1.amazonaws.com/classdojo/matomo:${BUILD_STRING}

Inside our Nomad job specifications, we inject a config.ini.php file. This contains the customized config.ini.php for Matomo. It is a copy of the original Matomo config.ini.php file, but with some important changes:

[General]
proxy_client_headers[] = "HTTP_X_FORWARDED_FOR"
force_ssl = 1
enable_auto_update = 0
multi_server_environment=1
browser_archiving_disabled_enforce = 1

Proxy_client_headers and force_ssl are used as part of our SSL setup. Enable_auto_update prevents containers from updating separately, so that we can coordinate updates across all containers. Multi_server_environment prevents plugin installation from the UI and disables UI changes that write to the config.ini.php file. Browser_archiving_disabled_enforce ensures that the archiving job is the only job that can run archiving, and that archiving won’t happen on demand.

For our non-frontend ingestion containers, we also set:

; Maintenance mode disables the admin interface, but still allows tracking
maintenance_mode = 1

Another major change is that the Docker command for the queue processor is changed to:

        command    = "/bin/sh"
        args       = ["-c", "while true; do /var/www/html/console queuedtracking:process; done"]

This allows the job to run in a loop, continuously processing the items in the queue.

Similarly, the archive job is changed to:

        command    = "/var/www/html/console"
        args       = ["core:archive"]

Which runs the archiving job directly. The admin and ingestion containers all use the default docker command and arguments.

That’s the end of our current journey from Google Analytics to Matomo. There’s more work we have to do around production monitoring and making upgrades easier, but we’re very happy with the performance of Matomo at our scale, and its ability to grow with ClassDojo.