Overview
Upgrading a production mail server during a major Security Issue can be daunting. When configuration files change drastically, you often spend hours—sometimes days—digging through documentation to find the single small change that broke your entire mail flow.
During my recent upgrade from Debian 12 (Bookworm) to Debian 13 (Trixie), I encountered significant breaking changes in Dovecot. This article documents my journey and provides a solution for anyone facing similar issues with virtual user lookups in Dovecot 2.4.
Sometimes when you update your system and there is a major version upgrade of some packages where the config files drastically changed, you have to read the whole documentation, try to understand everything and sometimes it takes very long beforeyou found the little change that broke the whole stuff.
Also Debian allows you to compare and to merge your configuration with the default configuration.
This time no merge succeeded, only conflicts. I knew I will have do dive.
I did an upgrade from debian 12 (Bookworm) to debian 13 (Trixie) about a year ago.
The big change between Bookworm & Trixie (from my point of view) is the use of the apt deb822 format change which seems easier to use. But Trixie is working with both format, so you don’t have to hurry. And at some point you decide to do the change.
File: /etc/apt/sources.list.d/debian.sources
Types: deb
URIs: https://deb.debian.org/debian
Suites: trixie trixie-updates
Components: main non-free-firmware
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
Types: deb
URIs: https://security.debian.org/debian-security
Suites: trixie-security
Components: main non-free-firmware
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpgCode language: HTTP (http)Simple isn’t it. Then you remove the old /etc/apt/sources.list and you update / upgrade
sudo apt update
sudo apt upgrade
sudo apt full-upgrade
sudo apt autoremoveAnd because it was for the php upgrade I removed the old version of non patched php that I didn’t use anymore
sudo find /etc/nginx/ -type f -exec sed -i 's/php8\.2\-fpm\.sock/php8.4\-fpm\.sock/g' {} \;
sudo systemctl stop php8.2-fpm
systemctl reload nginxCode language: JavaScript (javascript)sudo apt purge "php8.2-*"
sudo apt purge "php7.4-*"
sudo rm -rf /var/run/php/php8.2-fpm.sock
sudo rm -rf /etc/php/7.4/Code language: JavaScript (javascript)I don’t know why, all the stories start like this. I did just want to upgrade to the last php version because of the security issue CVE-2026-109618 & CVE-2026-109619…
So, I needed the last security version, which was in trixie-security, and while wanting to add trixie-security I thought it would be easier to upgrade everything to the last secure version (which is always a good idea). I also decided to change to deb822 apt format and do a full-upgrade…
I use dovecot, especially Debian packages:
dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql dovecot-pop3d dovecot-sieveBut Dovecot did change a lot since Bookworm. Dovecot 2.4 has a lot of breaking changes in the config files.
The biggest one is the way it handle the virtual users and pass from '%u' to '%{user | domain}'
Also linked to the default use of a database which look like this:
username | domain | password | home | uid | gid
Where I have multiple domain in a virtual_domain table, multiple real user in virtual_user and multiple alias for my virtual users in a table virtual_alias.
Also I’m using a vmail system user who has access to all databases as everything is virtual and a /var/mail/vhosts/<domain>/<user> structure with Maildir
By installing the last dovecot everything changed and it took me 24h to solve it, so I hope that if someone has the same issue, one find this article.
I won’t do the whole history just focus on what to do
From Postfix 3.7 to Postfix 3.10
No big change because of http://www.postfix.org/COMPATIBILITY_README.html.
Everything is virtual including the domain where the server is running. Imagine that the machine is mail.sbw.be. I don’t want to have mailbox for my real users but only virtual mailbox for everyone.
compatibility_level = 2
myhostname = mail.sbw.be
mydomain = sbw.be
myorigin = mail.sbw.be
mydestination = localhost, localhost.localdomain
relayhost =
virtual_transport = lmtp:unix:private/dovecot-lmtp
# Virtual domains, users, and aliases
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_domains =
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf,Code language: PHP (php)Check virtual domains and users
To check if your virtual_mailbox works there is a nice command line utility.
sudo postmap -q sbw.be mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
sudo postmap -q xxx@sbw.be mysql:/etc/postfix/mysql-virtual-mailbox-maps.cfCode language: JavaScript (javascript)Should return ‘1’ if the domain is found and nothing if the domain is not there
sudo postmap -q xxx@sbw.be mysql:/etc/postfix/mysql-virtual-alias-maps.cfCode language: CSS (css)Should return the main e-mail to deliver to, based on the alias.
Form Postfix to mailbox via LMTP
Now that postfix can check e-mail destination it can send them od Dovecot through lmtp.
Check lmtp access
~$ ls -al /var/spool/postfix/private/dovecot-lmtp
srw-rw-rw- 1 postfix postfix 0 Oct 23 17:56 /var/spool/postfix/private/dovecot-lmtpCode language: PHP (php)If everything works well you can restart postfix
systemctl restart postfixForm Dovecot 2.3 to Dovecot 2.4
Once the mail are sent to the dovecot-lmtp unix socket it need to pass all the Dovecot rules to be delivered to the right mail boxes.
lmtp
Ensure that the protocols lmtp is enabled /etc/dovecot/dovecot.conf
protocols = imap pop3 lmtpYou can also enable debugging in this file by adding this. It will be needed in the end if we have issue.
mail_debug = yes
auth_verbose = yes
auth_debug = yes
log_debug = (category=auth OR category=mail OR category=ssl)Ignore local auth and use virtual users in /etc/dovecot/conf.d/10-auth.conf
auth_mechanisms = plain login
#!include auth-system.conf.ext
!include auth-sql.conf.extCode language: PHP (php)Define the mail generic organisation in /etc/dovecot/conf.d/10-mail.conf
mail_driver = maildir
mail_home = /var/mail/vhosts/%{user | domain}/%{user | username}
mail_path = ~/
mailbox_list_layout = Maildir++
mail_privileged_group = vmailCode language: PHP (php)Define users to run services and open the ports for imap, pop3 and lmtp in /etc/dovecot/conf.d/10-master.conf
service imap-login {
inet_listener imap {
port = 0
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0666
user = postfix
group = postfix
}
}
service auth {
unix_listener auth-userdb {
mode = 0666
user = vmail
}
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix
}
user = dovecot
}
service auth-worker {
user = vmail
}Code language: PHP (php)Define mailbox in /etc/dovecot/conf.d/15-mailboxes.conf
namespace inbox {
type = private
separator = .
prefix =
inbox = yes
mailbox INBOX {
auto = subscribe
}
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Trash {
special_use = \Trash
}
mailbox Spam {
auto = subscribe
special_use = \Junk
}
mailbox Archive {
auto = subscribe
special_use = \Archive
}
mailbox Sent {
special_use = \Sent
auto = subscribe
}
mailbox "Sent Messages" {
special_use = \Sent
}
}Code language: JavaScript (javascript)Here is where I finally found that the domain was removed form the username variable before the mail delivery causing a lot of trouble to drop the mail in the right mailbox: /etc/dovecot/conf.d/20-lmtp.conf
protocol lmtp {
# auth_username_format = %{user | username | lower}
}Code language: PHP (php)Enable SQL authentication in /etc/dovecot/conf.d/auth-sql.conf.ext
sql_driver = mysql
mysql 127.0.0.1 {
user = HardCodeYourDBUserNameHere
password = HardCodeYourDBPasswordHere
dbname = HardCodeYourDBNameHere
port = 3306
}In the same file we will define the way password are validated and the way we search where to store the mail based on the e-mail address.
So you will need to create a query that retrieve the password for a user: passdb sql.
passdb sql {
default_password_scheme = SHA512-CRYPT
query = \
SELECT email AS user, password \
FROM virtual_users vu, virtual_aliases va \
WHERE va.destination = vu.email \
AND va.source = '%{user}' OR vu.email='%{user}' LIMIT 1
}Code language: PHP (php)And a query that tells dovecot where to drop the mail based on the destination e-mail: userdb sql.
userdb sql {
driver = sql
query = \
SELECT CONCAT('/var/mail/vhosts/', domain, '/', username) AS home, \
5000 as uid, \
5000 as gid \
FROM ( \
SELECT SUBSTRING_INDEX(destination,'@',1) AS username, \
SUBSTRING_INDEX(destination,'@',-1) AS domain \
FROM virtual_aliases AS va JOIN virtual_domains AS vd ON vd.id=va.domain_id \
WHERE va.source = '%{user}' \
UNION \
SELECT SUBSTRING_INDEX(email,'@',1) AS username, \
SUBSTRING_INDEX(email,'@',-1) AS domain \
FROM virtual_users WHERE email='%{user}') t\
LIMIT 1
}Code language: PHP (php)That once was not that easy and I know the join on virtual_domains looks useless but it’s an “integry check” with the JOIN I’m at least sure there is a domain associated with the e-mail in the domain table.
Of course if you use the database table proposed by default by dovecot, you can just uncomment the lines of the userdb sql section and thus have easier SQL queries.
Once everything is ready let’s test and debug.
Debugging and testing
For debugging we need to enable full log in dovecot and in postfix
In /etc/dovecot/dovecot.conf add:
mail_debug = yes
auth_verbose = yes
auth_debug = yes
log_debug = (category=auth OR category=mail OR category=ssl)In /etc/postfix/master.cf add a ‘-v’ at the end of the lmtp line
lmtp unix - - y - - lmtp -vEverything will be available inside /var/log/mail.log
Check dovecot config
dovecot -n will check the dovecot config files, if there is an error it will highlight the file and the line number.
In a terminal run sudo tail -f /var/log/mail.log and wait.
In another terminal use swaks to send a mail.
install swaks and send a mail
sudo apt install swaks
swaks --to xxx@sbw.be \
--from yyy@sbw.be \
--body "Testing LMTP handoff" \
--server localhostCode language: JavaScript (javascript)The journey of the mail
Mail Transport to the destination
| Step | What could go wrong ? |
The mail will be handled by postfix first and send through lmtp socket: /var/spool/postfix/private/dovecot-lmtp. | file rights file does not exists |
| Dovecot will use the mysql query userdb sql to get all the information (home, uid, gid, …) needed to know where the mail should be written. | query error query find nothing variable are modified variable not parsed query missing some information bad concatenation to create the ‘home’ path alias are create as real addresses |
| Dovecot will write the “new” mail in the right Maildir folder | access rights |
Mail from the mailbox to the imap/pop client
So greate, no the mail is on the server. We still need to retrieve it.
I recommand to setup an autoconfig. Exemple: autoconfig.sbw.be. And anyway a mail client.
The username is the full e-mail. For instance xxx@sbw.be.
The port have been delcared in /etc/dovecot/conf.d/10-master.conf
I recommend a reverse proxy to tunnel everyhting within SSL and with a clean certificate.
I created an nginx domain:
server {
listen 443 ssl; # managed by Certbot
listen [::]:443 ssl; # managed by Certbot
listen 80;
listen [::]:80;
server_name mail.sbw.be;
server_name smtp.sbw.be;
server_name imap.sbw.be;
root /var/www/domains/be.sbw/mail/public_html/;
# ...
}Code language: PHP (php)This will allow to use Certbot for the certificate generation and to reuse the create fullchain.pem and privkey.pem within dovecot.
Configuring the SSL
In /etc/dovecot/conf.d/10-ssl.conf add the .pem file generated by certbot for nginx (or apache or …)
ssl_server_cert_file = /etc/letsencrypt/live/mail.sbw.be/fullchain.pem
ssl_server_key_file = /etc/letsencrypt/live/mail.sbw.be/privkey.pemCode language: JavaScript (javascript)Testing with a mail client
Name: xxx@sbw.be
Password: the password stored in the database and associated with the user.
To setup a password you need the hash of the password. Simply use:
sudo doveadm pw -s sha512-cryptAnd then use the output as a password in the database.
Then test the user agains the password you just created
~$ sudo doveadm auth test xxx@sbw.be SomePassword
passdb: xxx@sbw.be auth succeeded
extra fields:
user=xxx@sbw.beThis allows you to check the passdb query.
Conclusion
The most difficult part was to find out why the username was trimmed from the domain name and the SQL queries part.
Enjoy!
Feel free to ask questions.