A addon for Plone which allow you to track requests from AI Chatbots to the server in Matomo.
Tracks the requests of AI bots in Matomo, also when Varnish serves them from its cache, without slowing them down:
- Varnish classifies AI bots by User-Agent as
user(an AI assistant fetching a page for a user, like Claude-User or ChatGPT-User),search(AI search crawlers) ortraining(training data crawlers), and writes that to its log. - varnishncsa writes these requests to a log file, and a small shipper sends them in batches to Plone. When Plone or Matomo is down, tracking is delayed, not lost.
- Plone forwards them to Matomo's bulk tracking API, with the original time of the
request. Requests for the resources of pages, like images, styles and scripts, are
skipped, and documents like PDFs are tracked as downloads:
userrequests go to Matomo's AI Chatbots report (Matomo 5.8 or later).- Optionally all AI bot requests go to a separate Matomo site as visits, with the bot category, bot name and the Varnish cache status (hit, miss, pass) as custom dimensions.
The control panel "Matomo AI Chatbot Tracking" has the Matomo settings. See the add-on's README for setting up Varnish, varnishncsa and the shipper.
Install collective.matomoaitracker with pip:
pip install collective.matomoaitrackerAnd to create the Plone site:
make create-siteThis Addon expects you to run a varnish, varnishnsca and shipper Docker image. Examples can be found in the subfolder for each service.
- An operating system that runs all the requirements mentioned.
- uv
- Make
- Git
- Docker (optional)
-
Clone this repository, then change your working directory.
git clone git@github.com:collective/collective.matomoaitracker.git cd collective.matomoaitracker -
Install this code base. This also builds the Docker images for the full stack.
make install
The full stack runs in Docker:
requests: Varnish (8001) -> Nginx VirtualHostMonster (8002) -> Plone site "Plone" (8003)
tracking: Varnish log -> varnishncsa -> ai-bots.log -> shipper -> Plone -> Matomo (8004)
Varnish's configuration (varnish/default.vcl) is based on the one of Plone's
cookieplone project templates, tuned for plone.app.caching: Plone purges changed
content from Varnish, logged-in users are not cached. For the tracking, Varnish only
classifies AI bots and writes the result to its log, it makes no HTTP calls. varnishncsa writes the AI bot requests to a log file, which the shipper sends
in batches to Plone, and Plone forwards them to Matomo. By default that is a fake
Matomo, which records what it receives for the tests.
make stack-startOn start the Plone site is created, the Matomo settings are applied and the service
user for the shipper is created. Settings can be overridden with environment
variables or in a .env file, see .env.example:
cp .env.example .envSimulate an AI chatbot visit, and see what reached the fake Matomo a few seconds later:
curl -A "Claude-User/1.0" http://localhost:8001/
curl http://localhost:8004/_requestsTest the VCL and the varnishncsa format with varnishtest:
make varnish-testTest the whole tracking chain against the running stack with the fake Matomo:
every AI bot is tracked once in the right category, also from the cache, other
requests are not, bots are not slowed down, and outages of Matomo or the shipper
delay tracking without losing it. These are the pytest tests in tests/stack, which
make test skips because they need the stack:
make stack-testThe tests run with Plone's caching policy, and again with pages cached in Varnish for
60 seconds (moderateCaching), restoring the policy afterwards. They log in to Plone's
REST API as admin:admin for that, set PLONE_ADMIN=user:password for other
credentials.
Other targets: make stack-logs, make stack-stop and make stack-remove-data.
This package provides markers as strings (<!-- extra stuff goes here -->) that are compatible with plonecli and bobtemplates.plone.
These markers act as hooks to add all kinds of subtemplates, including behaviors, control panels, upgrade steps, or other subtemplates from plonecli.
To run plonecli with configuration to target this package, run the following command.
make add <template_name>For example, you can add a content type to your package with the following command.
make add content_typeYou can add a behavior with the following command.
make add behaviorYou can check the list of available subtemplates in the [`bobtemplates.plone` `README.md` file](https://github.com/plone/bobtemplates.plone/?tab=readme-ov-file#provided-subtemplates).
See also the documentation of [Mockup and Patternslib](https://6.docs.plone.org/classic-ui/mockup.html) for how to build the UI toolkit for Classic UI.
The project is licensed under GPLv2.
Generated using Cookieplone (2.0.0) and cookieplone-templates (fa8eca4) on 2026-09-30 17:16:12.816436. A special thanks to all contributors and supporters!