Home
Products
Download
Buy Now
Support
Company

This page is out of date, please use our new website https://surgemail.com

Help Index

 

Mailserver Migration

SurgeMail has a number of features to make the migration from an existing mail server to your new SurgeMail installation easier. These can be combined to your choice to make your mail migration as painless as possible for both you and your users. However migrating from one mail server to another should always be done carefully and should be fully controlled. Important considerations are factors such as simplicity of upgrade procedure, length of downtime and ability to rollback.

Unless you have specific requirements, the recommended method for upgrading to SurgeMail is POP / IMAP intercept mode. We aim to make the migration as simple and safe as possible. If you have any suggestions on how we can improve the migration process please let us know:
surgemail-support@netwinsite.com

Note: In all cases we recommend you start by making a backup copy of your existing mail server's mailbox files :-)

Migration using POP / IMAP intercept mode (recommended)

This is a simple zero downtime method for migrating all active accounts from any arbitraty mailserver to SurgeMail. This method configures SurgeMail to be the new mailserver for all users. When a user logs into SurgeMail and a local account does not exist, SurgeMail will login to the old server check the account is valid. If the account is valid SurgeMail will create an account and retrieve mail for this account storing it locally. From now on the user will use SurgeMail as their primary mailserver and mail for them will be delivered locally. Mail delivered to SurgeMail for users that have not yet logged in will be forwarded on to your old mail server.

  1. Backup your existing mail server's mailbox files :-)
  2. Install SurgeMail onto your new server and configure to host your existing domains
  3. Setup the options old_POPhost / old_IMAPhost and fallback_relay for the domain you are migrating (on domain administration page). The fallback_relay setting will pass on all mail for accounts that not yet have been created locally to your old server and the old_pophost setting will allow SurgeMail to retrieve account information and email from the old mailserver.
  4. Change your DNS and MX records to point to surgemail
  5. Whenever a user checks their mail an account with the same username and password will be created in SurgeMail and all outstanding mail will be retrieved.

Detailed examples and documentation can be found here on using this method. Note! migration cannot be used in conjunction with CRAM-MD5 as that prevents surgemail from storing the old password. CRAM-MD5 is best not used as SSL offers better security.

You can also use this command line tool to import a single account or test importing an account. Examine migration.log if it is not successful: 

tellmail imap_import newuser@domain oldimaphost olduser oldpass delete|keep create|nocreate
e.g. to import an account, without deleting the messages from the old one, use: 
tellmail imap_import test@xyz.com mailsever.xyz.com test@xyz.com SECRET keep create

If you wish to test an account by logging in to it, and then you wish to delete the account and test it again you would need to issue this command so that surgemail will know it needs to retry:

tellmail pstat_delete "user@domain.name"

To make it do all accounts again use: 

tellmail pstat_delete "*"

(this can result in duplicate messages, normally you would only do this if you had been testing and you wanted to delete the mailbox folders manually and start again!)

 

Migration from Non SurgeMail server

SurgeMail has a number of features to make the migration from an existing non SurgeMail mailserver to SurgeMail easier:

  • Parallel operation / gradual migration
  • Authentication database import
  • Delivered Mail conversion
  • Complete configuration upgrade (DMail only)

Parallel operation / gradual transfer

SurgeMail has the option "fallback_relay" (configurable per domain) that allows mail to be delivered to a different host if the user does not exist locally. This can be used whilst testing that a system is operational and if desired for the gradual migration of users from an existing server to the current server. The two ways of configuring this are as follows:

1. Configure SurgeMail to handle the domain served by your existing server. Keep your MX record pointing at your existing server. On the existing server cc (or redirect) mail to your SurgeMail configuration. For the account(s) in question start using SurgeMail to retrieve and send mail. In this configuration rollback is easy by simply switching your mail client back to the original server.
Note: If you are sending to servers that do a reverse lookup you will need to get SurgeMail to relay outbound through your existing server.

2. Configure SurgeMail to handle the domain served by your existing server. Switch your MX record to the SurgeMail server. Now if the account in question is defined in the SurgeMail user database, SurgeMail will process mail for this user and the users mail client will need to point to SurgeMail. If the account is not present all mail will be forwarded to your existing mailserver and the users mail client will need to point to your existing mail server.
Note: Again if you are sending to servers that do a reverse lookup you will need to get your existing server to relay outbound mail through the SurgeMail server.

Authentication database import

There probably is no need to is no need to import your existing user database as SurgeMail is very likely to have an authentication module that will use your existing database directly. Having said that many administrators do choose to convert their existing user database to NWAuth. This is especially so if using system based authentication as NWAuth does not require a UNIX system account for every user in your authentication database.

If converting your sendmail configuration your existing /etc/passwd + /etc/shadow account database can be imported to NWAuth format without users having to change their password or the encrypted password being decoded either by manually copying the fields or using the following script xferusers.pl.

Delivered Mail Conversion

SurgeMail uses maildir format to store delivered mail. SurgeMail has the inbuilt ability to convert standard drop files to maildir format. In addition SurgeMail will convert dmail bin files and mbx files to SurgeMail mdir format. The settings that control this process are the surgemail.ini vdomain settings: dmail_drop_path, dmail_bin_path and dmail_hash.

eg: If you were converting a Windows DMail configuration the upgrade settings would be as follows, these are automatically added if you do SurgeMail install based on a dmail.conf.

dmail_bin_path "D:\somewhere\binfiles"
dmail_drop_path "D:\somewhere\dropfiles"
dmail_hash "0"

Note: Hash level: 0 = no hashing, 1= dmail specific hashing, 2=generic double level hashing (mail/f/r/fred), 3= dmail specific hashing.

eg So unhashed UNIX Sendmail drop files can be converted as follows:

dmail_drop_path "/var/spool/mail"
dmail_hash "0"

The default SurgeMail behaviour is to check for mail to convert when a user logs in using POP or IMAP. This means that normally the conversion load is evenly spread over the timeframe that users login. However larger accounts may see a delay in checking their mail while the mail conversion takes place. This can be prevented by doing the conversion of all "outstanding old mail" in batch mode using the following command:

tellmail convert_dmail

This command requires dmail_bin_path and dmail_drop_path and dmail_hash settings to have been setup correctly prior to running the command.

DROPFILE format (pine)

For pine format unix drop files (Each folder is contained in a single file with each message starting with the line 'From ...', you can use this command to import mail folders (other than the inbox which the dmail conversion commands will handle) This commands requires SurgeMail 3.7c4 or later.

tellmail dropfile_import john@your.domain /home/john/mail

Complete Configuration Upgrade

As each mail server configuration tends to be unique and specifically configured we do not support a generic upgrade ability from an arbitrary mail server. We aim to supply the capability to easily migrate the necessary elements of your configuration to surgemail.

Having said that, SurgeMail does have the ability to upgrade an existing DMail configuration. It does this by generating a surgemail.ini configuration based on the dmail.conf configuration. This reuses the authentication database that DMail used and adding the settings described in "Delivered Mail Conversion" above. Then when a user logs in mail existing mail is converted to the correct SurgeMail location.

Moving a SurgeMail configuration

SurgeMail has been designed to make migration between two SurgeMail servers of similar or of different platforms very easy but again there are several different ways to migrate.  We recommend the Mirroring migration method, see instructions here.

NOTE:  If you upgrade to the latest surgemail version in the process of transferring to a new server, then you will need to purchase updates (if you don't currently have updates).  Alternatively, you can install a matching 'old' version of surgemail on the new system. (Old releases are available here.

Migration by copying directory tree

The easiest and currently the recommended way of moving SurgeMail between servers of the same platform is by copying the whole SurgeMail directory tree. To do this, install a default configuration on the new server, then copy the entire SurgeMail directory tree (plus /etc/surgemail.ini) to the new server, move your license key from one host to the other using tellmail deactivate / activate, and update your mx record.

"Moving" your SurgeMail key:

  1. If you want to temporarily run your two servers side by side, you can run with the normal temporary trial key until you transfer your license.
  2. Run "tellmail deactivate N##### myemail@mydomain.com" on the old server to disable your paid key on your old server
  3. Run "tellmail activate N##### myemail@mydomain.com" on the new server to enable your paid key on your new server

Migration by copying data files

Alternatively the relevant data files can be copied. This is the recommended way when moving between platforms or you wanting a clean install on the same platform. To do this you will need to install a default configuration on the new server, then copy the following configuration, mail and data files from the old installation to the new installation, move your license key, and update your MX record.

WE STRONGLY RECOMMEND USING MIRRORING INSTEAD WHEN MOVING BETWEEN PLATFORMS.

To copy SurgeMail data directories you will need to copy:

  1. surgemail.ini (c:\winnt for windows or in /etc for unix) : main SurgeMail configuration
  2. surgemail_direcotry\mydomain.com : delivered mail for each domain.
  3. surgemail_direcotry\webmail_work : WebMail options (and folders if running WebMail in POPmode)
  4. surgemail_directory\scripts\webmail.ini : WebMail configuration file
  5. surgemail_directory\nwauth.txt + nwauth.add : user database (if using NWAuth)
  6. surgemail_directory\*.ini *.dat : variety of configuration files
  7. surgemail_directory\work (optional) : undelivered mail queue
  8. surgemail_directory\recYYMM (optional) : delivery record
  9. surgemail_directory\ssl (optional) : if signed certificates have been added these will need to be copied
  10. surgemail_directory\bull and dlist (optional) : configuration of bulletins and mailing lists

On UNIX be sure to correct the file ownerships to 'mail' if you coppied the files as root! (chown -R mail /usr/local/surgemail)

Zero downtime migration using Mirrorring (RECOMMENDED)

See this page for detailed instructions

In both the above two cases neither the old or new SurgeMail installations should not be running whilst copying files. If this is not acceptable in terms of interruption of service you can use migration using pop intercept mode or migration using mirrorring. The optimal procedure is still being finalised for zero downtime upgrades so please test your chosen migration path on a non live system first and again suggestions for improvement are welcome.

The mirrorring feature can be used to migrate user accounts and mail from one server to another using zero downtime. To do this you need to Setup your new server in the same configuration as your old server by copying the configuration files and install a temporary second license key. Enable mirrorring on both systems. Add an MX record with higher priority for your new server. At this stage both servers will be live and mail can be delivered to either server and it will be mirrored to the other server. When you are confident the new server is working as expected you remove the mx record for the old system, wait for any remaining mail to be mirrored to the new server and take the old server offline.

Several things should be carefully noted. Firstly mirroring only duplicates delivered mail, more specifically files that get mirrored are the files stored in the surgemail/mydomain.com maildir directories for each domain. This means that certain settings or files will not get mirrored as follows:

  1. If you are using WebMail in POP mode mail stored in WebMail folders will not be synchronised. If you are using WebMail in IMAP mode (since recently the default) mail in WebMail folders will be copied.
  2. WebMail user configuration settings, DList information will not be copied.

Miscellaneous Mail Conversion / Import / Export

Moving mail in WebMail folders to surgemail IMAP folders

When running WebMail in POP mode the mail stored in folders is stored in the WebMail work area. When running WebMail in IMAP mode this mail is stored in the SurgeMail folders and is accessible using other IMAP clients. To convert this mail do the following:

  1. You need WebMmail version 3.0r or later
  2. Ensure you have a manager password configured in your webmail.ini file "managers_password mypassword"
  3. Run "webmail.exe -manager", do option "m" and enter "*" for converting all users.

Importing old mail from other Maildir implementations

Maildir implementations vary slightly in their implementation. SurgeMail is able to import other Maildir formats by running the command "tellmail maildir_convert surgemail_domain.com source_directory" where source directory is a fully specified path with $HASH1 / $HASH2 and $USER, where expansion of directories is required.

tellmail maildir_convert mydomain.com '/home/$USER/Mail'
tellmail maildir_convert mydomain.com '/home/$HASH2/$USER/Mail'
tellmail maildir_convert mydomain.com '/var/spool/mail/$HASH1/$USER/Mail'

Where
$HASH1 = /fr (for fred)
$HASH2 = /f/r (for fred)
$USER = wild card scan of usernames

eg: Converting all mail stored in the format /var/spool/mail/user1/Mail to /usr/local/surgemail/mydomain.com/bb/wj/user1/

tellmail maildir_convert '/var/spool/mail/$USER/Mail'

Virtual User Table - Translating username to username@domain.name

Using 5.1 or later you can create a file called 'virtusertable.dat' to translate user names given without a username during pop/imap login into a user and domain name pair, this should only be used when converting users from an old sendmail system where you need to translate the users to multiple different domains, e.g.

fred -> fred@domain1.com

bob -> bob@domain2.com

joe -> joe@domain3.com

 

The file format is as follows:

fred fred@domain1.com
bob bob@domain2.com
joe joe@domain3.com