Watchful generates a lot of data about the WordPress and Joomla websites added to your account. This saved information, with a variable retention time, forms the activity logs.

Next week we will release a revamped logging system. This post describes the change and what to expect as a Watchful user.

How we store logs currently

Log data is used to generate notifications, for debugging site errors, and for building client reports. Let’s use a software update as an example log entry.

When a plugin is updated on a Watchful-powered website, a record is made of the type of event (software update in this case), the date and time the update was detected, the name of the plugin, and the version information.

Here is what this might look like in the log viewer here at Watchful:

Using the first item in the list, we would save the log entry as plain text exactly as shown above:

Akeeba Backup has been updated (8.0.3 => 8.0.4)

This creates a lot of wasted space in our database as the phrase has been updated is repeated millions of times. Multiplying this by all of the different log types makes the storage problem that much worse.

Additionally, the saved logs contain any language string errors present at the time of creation. Fixing these errors retroactively is not possible in the current system.

Revamped logging will reduce storage & add flexibility

We have been working on a revamped logging system for a few months and plan to launch it on Wednesday, 28-April. The new system will no longer store explicit log entries as described above. Instead, the logs will store variables such as software_name and event_type.

When needed, the variables combine in real time to generate the same output shown above. In most cases, we expect the revamped logs to be identical (or nearly so), though there are some exceptions (see below).

In addition to saving a lot of storage space, the logs are more flexible and can be easily modified or translated into new languages.

Changes for non-English accounts

Watchful has a rather inconsistent history with multi-language support. We currently offer both French and English, but in the past we also offered Italian, Spanish and Russian.

As a result, users who have ever used a non-English language in Watchful have mixed-language logs that look something like this:

The revamped logging system will fix this mixed-language problem. However, all logs (including existing ones) will be English-only.

Timeline

To prepare for release, we have been processing terabytes of data behind the scenes to make it compatible with the new system. With that task complete, here is the schedule for the roll-out:

• 21-April-2021 — Announce upcoming changes (website & newsletter).
• 28-April-2021 — Release changes for all Watchful users.
• 28-April-2021 through 5-May-2021 — Review logs for errors and adjust as needed.
• 23-June-2021 — Modify retention times.

Feedback

If you have any questions or concerns regarding this change, please get in touch using the support link in your account.