A quick-but-not-so-quick tip for Linux users: beware of systemd journal logs and backup.
Recently, I had a server running out of space often and because of that, MySQL and other services crashing. When checking the reason, I found out that both systemd journal logs and one of the servers websites backups were hogging disk space.
Cleaning up systemd journal logs space with journalctl
In the case of systemd journal logs, it’s a good “system hygiene” to clean it up often. Just don’t clean it up too much, because Logs are useful for security audits.
To do that, just use the journalctl command with the either the --vacuum-time or --vacuum-size parameters with a sensible setting. The first one will clean the journal logs and leave just the ones from the time set. For example, journalctl –vacuum-time=6months will keep only the lost from the last 6 months. The seconde one will clean logs until it achieves the specified size. That is, journalctl --vacuum-size=500M will keep only 500 megabytes of data.
For more information on how journalctl, a great tool, works check its manual page: https://www.man7.org/linux/man-pages/man1/journalctl.1.html
Make sure you delete old sites backups.
Another often overlooked reason for running out of disk space is old backups not being deleted. Specially when you have some sort of automatic backup in place (e.g. using cron jobs) but they are not automatically deleted. In my case, one of the cron jobs to delete old backups of a client website were not properly set and it was not working. So, remember to check your backups often and make sure all your cron jobs are working properly.
