I’ve been running my own mail server for years, using the ispmail setup, and recently I have been looking into replacing it with a slightly more comprehensive setup (with a modern webmail, admin ui), ideally without putting in all the manual work. Here is what I came up with:
Open Source options
sovereign – ansible based, lots of unrelated stuff, but no admin ui for mail.
MailCow – Docker-based.
Mailu – Docker-based.
iRedmail – seems capable, but no cool factor.
modoboa – Administration only
Kite (developer now an engineer at nylas.com)
Nylas N1 – a Desktop solution now, but using web tech could presumably run in the browser.
Interesting Commercial Mail solutions
FastMail – uses Sieve for filters.
atmail – provides a self-install version; a web interface I really like; pricing seems unclear
JMAP – IMAP successor.