XQM Manual 1.29

Welcome
Querying the Queue
The Inspector
| Spoolfiles
| Slogfiles
The Eraser
The Defroster
The Flusher
The Exporter
Start and Stop
Configuration
| Profile Management
| Local Mode
| Remote Mode (XQM Agent Configuration and Security)
XMail SMTP Error Codes
Bugs ?

Welcome to XMail Queue Manager ! |

| XMail Forum

Join the community brainpool ... You are a newbie to all that SMTP things? Stunned by the XMail Server manual? You are an expert level user migrating to XMail Server? Any Questions about XMail related tools? Join the community and share knowledge! Visit XMail Forum

At first: don't be scared about reading manuals! This doc is as short as possible and it will help you to make your life as an XMail Server Admin easier! So let's go ...

XMail Server handles all its incoming and outgoing mail in a single folder called spool.

When you look inside that directory on a standard installation, you see 23 sub folders, numbered from 0 to 22 and 2 other folders called temp and local. Each of the 23 numbered folders has again 23 sub folders inside (0-22). Each of this 2nd level folders ends up in 8 folders named mess, rsnd, froz, slog etc ...

This makes up a total of (23 * 23 * 8) + 2 = 4.234 folders, which XMail Server uses as a queue database for multiple threads.

So XQM was created to have a tool that

- allows you to flatten this multilevel nested structure to a single, easy readable list,
- to view a snapshot of the incoming and outgoing mail flow,
- to inspect messages for detailed content and error diagnostics,
- to monitor message- and queue size,
- to do Remote Queue Management of multiple non-GUI machines
  which run XMail Server ...

Querying the Queue |

This function makes a snapshot of the current queue content. XQM spawns multiple threads,. each of them collecting the information of one or more sub tree structures in parallel. When all threads are finished, the spoolfile list is displayed.

The list is sortable by each column header field. Every line starts with an icon that represents the message's current status:

| Sending means that it is currently delivered, exactly in this moment.
| Resending means that the message could not be delivered due to a temporary delivery error and is tried again.
| Frozen : If a message was resent a certain number of times, it changes its state to frozen. Frozen messages are not resent anymore.

When you right-click an entry, you'll get a popup menu, which displays all message related functions of the toolbar for quick access.

Diagnosis

The maybe most important column is the rightmost one called "Diagnosis". It contains a short hint (message tag), that tell you why a message wasn't delivered. E.g. "QUOTA", if the recipient's mailbox was full or "DNS_FAILED?" when a DNS problem is assumed etc.

This Analyzer function is extremely helpful for classifying a bulk of messages quickly at a glance. By sorting the column, you can easily seperate malformed mails from only temporary errors. There is need to dig through each message with the Inspector.

If you are interested in creating user defined message tags, then have a look at the file parse.lib in XQM's main directory. The file can be viewed with any ASCII viewer or editor.

The Inspector |

The queue inspector helps you to track messages inside your spool directory. It gives you a detailed view of the currently selected mail (=Spoolfile), its corresponding send log (=SLOG File) and some deeper diagnosis info.

Best practice:

- First check the rightmost column called "Diagnosis". In most cases it contains a hint about the reason why a message wasn't delivered. If you need more details or if the column is cleared, continue with the Inspector
- Leave one inspector dialog open and click through the list to display the currently selected message dynamically.
- Open multiple inspectors when you want to compare some messages with each other.
- You can also envoke this tool by double clicking any list entry or by using the right-button popup menu on the listview.

Spoolfiles

Spoolfiles are RFC822-conform mails with an additional header that contains XMail specific data. Their structure is as follows:

| Header structure up to version 1.13

1: SMTP Domain 2: SmtpMessageID 3: MAIL FROM:<...> 4: RCPT TO:<...> 5: <<MAIL-DATA>> 6: The RFC822-conform mail starts here ...

| Header structure since version 1.14

1: Remote_IP:Port;Local_IP:Port;Datestamp 2: SMTP Domain 3: SmtpMessageID 4: MAIL FROM:<...> 5: RCPT TO:<...> 6: <<MAIL-DATA>> 7: The RFC822-conform mail starts here ...

SLOG Files

When a message is in the resending or frozen state, XMail creates an individual send log file (=SLOG) for that message. The SLOG contains detailed informations about each try to deliver the mail.

| Example

[PeekTime] 1046264879 : Wed, 26 Feb 2003 14:07:59 +0100
<<
ErrCode = -40
ErrString = Invalid server address
ErrInfo = marketmix.co
SMAIL SMTP-Send FF = "marketmix.co" SMTP = "tbird" From = "mailmaster@testdomain.de" To = "test@marketmix.co" Failed !
SMTP-Error = "417 Temporary delivery error"
>>
[PeekTime] 1046265771 : Wed, 26 Feb 2003 14:22:51 +0100
<<
ErrCode = -40
ErrString = Invalid server address
ErrInfo = marketmix.co
SMAIL SMTP-Send FF = "marketmix.co" SMTP = "tbird" From = "mailmaster@testdomain.de" To = "test@marketmix.co" Failed !
SMTP-Error = "417 Temporary delivery error"
>>
...

"Reading these files looks much more complicated, than it is. Really!" :-)

With every delivery attempt, XMail adds a new entry to the SLOG file. The latest report is always at the bottom of the file.

A SLOG entry

- starts with the identifier [PeekTime], followed by a timestamp.
- The delivery result is enclosed in << ... >> symbols.

The delivery result

- contains a numeric ErrCode,
- a verbose description of the error, called ErrString
- and eventually additional information called ErrInfo.

After that you get the delivery details,

- like e.g. the delivery agent (LMAIL for local delivey and SMAIL for all other mails),
- the delivery mode (SMTP-Send or SMTP-Receive),
- the mail addresses of sender and recipient,
- and the SMTP Error Code.

All further deliver attempts are sequentially logged in exaclty the same manner.

So let's parse our example:
At exactly 26.Feb 2003, 14:07:05 XMail tried to deliver a mail to the domain "marketmix.co". The delivery failed, because the domain name could not be resolved by the name server (= invalid server address). This is a temporary delivery attempt, because maybe it is only a temporary name server problem. The next try was at 14.22.51. All the relevant things were taken from the red marked parts.

You see, it's as easy as promised. In most cases its enough to quickly inspect ErrString and the SMTP-Error message.

The Eraser |

Deletes messages by their state or selection.

If a message still exists after erasing, then it was locked by XMail. Just try again a little later in that case.

The Defroster |

This very handy function changes the state of frozen messages to sending messages. You won't miss that tool, if you once made configuration changes to your XMail Server, your DNS or network setup and some of your mails got stuck in the queue.

The Flusher |

When messages are resent, XMail sticks to a clearly defined scheduling. On a standard installation, the sending plan for each messages looks like this:

| Resend Scheduling Plan

`>> zinc.pl 480 16 32
01 send-time = 00000 (00:00:00) next-try = 0480 (00:08:00)
02 send-time = 00480 (00:08:00) next-try = 0510 (00:08:30)
03 send-time = 00990 (00:16:30) next-try = 0541 (00:09:01)
04 send-time = 01531 (00:25:31) next-try = 0575 (00:09:35)
05 send-time = 02107 (00:35:07) next-try = 0611 (00:10:11)
06 send-time = 02719 (00:45:19) next-try = 0649 (00:10:49)
07 send-time = 03369 (00:56:09) next-try = 0690 (00:11:30)
08 send-time = 04059 (01:07:39) next-try = 0733 (00:12:13)
09 send-time = 04793 (01:19:53) next-try = 0779 (00:12:59)
10 send-time = 05573 (01:32:53) next-try = 0828 (00:13:48)
11 send-time = 06401 (01:46:41) next-try = 0880 (00:14:40)
12 send-time = 07281 (02:01:21) next-try = 0935 (00:15:35)
13 send-time = 08216 (02:16:56) next-try = 0993 (00:16:33)
14 send-time = 09210 (02:33:30) next-try = 1055 (00:17:35)
15 send-time = 10265 (02:51:05) next-try = 1121 (00:18:41)
16 send-time = 11387 (03:09:47) next-try = 1191 (00:19:51)
17 send-time = 12579 (03:29:39) next-try = 1266 (00:21:06)
18 send-time = 13845 (03:50:45) next-try = 1345 (00:22:25)
19 send-time = 15190 (04:13:10) next-try = 1429 (00:23:49)
20 send-time = 16620 (04:37:00) next-try = 1518 (00:25:18)
21 send-time = 18139 (05:02:19) next-try = 1613 (00:26:53)
22 send-time = 19752 (05:29:12) next-try = 1714 (00:28:34)
23 send-time = 21467 (05:57:47) next-try = 1821 (00:30:21)
24 send-time = 23288 (06:28:08) next-try = 1935 (00:32:15)
25 send-time = 25224 (07:00:24) next-try = 2056 (00:34:16)
26 send-time = 27281 (07:34:41) next-try = 2185 (00:36:25)
27 send-time = 29466 (08:11:06) next-try = 2321 (00:38:41)
28 send-time = 31787 (08:49:47) next-try = 2466 (00:41:06)
29 send-time = 34254 (09:30:54) next-try = 2620 (00:43:40)
30 send-time = 36875 (10:14:35) next-try = 2784 (00:46:24)
31 send-time = 39660 (11:01:00) next-try = 2958 (00:49:18)
32 send-time = 42618 (11:50:18) next-try = 3143 (00:52:23)`

"For further informations about tweaking the scheduler, see chapter COMMAND LINE, paragraph [SMAIL] in the XMail Manual."

If you flush the queue, XMail starts to resend all messages immediately, without taking care for their precalculated scheduling.

Beware:
When using this function, you should keep 2 things in mind:

- Every flush() increases XMail's resend counter. If the counter exceeds 32 (in our upper example), the messages are frozen.
- If not running, XMail Server is started immediately when pressing the flush button.

Internally flushing is done, by opening a CTRL session and sending an ETRN command to the SMTP service.

CTRL access has to be configured in the Networking dialog if you manage a local queue. When doing Remote Queue Management, the XQM Agent takes care for CTRL access. This assures that password informations are never sent across a network connection.

The Exporter |

The exporter exports the list content as a plain ASCII list for use with other applications. Makes a mailing list manager's life easier.

Just decide which message type and which fields to export. Futher you can define a field seperator and quoting character. If you need the column names in the 1st line of your export file, check 'with column headers'. Check 'strip <> from addresses', if your export processor doesn't like <recipient@domain.com> type addressing.

Start and Stop |

Starts and stops XMail Server.

Configuration |

XQM knows two networking modes, "local" and "remote". As in its name, "local" is limited to your local machine and "remote" means across a network, e.g. your LAN or even the internet. So the first configuration step is to decide the networking mode. You can create multiple config profiles in this dialog.

Profile Management |

Select a profile:
All saved profiles are available in the pulldown menu, which allows quick switching in a very comfortable way. The list is sorted alphabetically and pressing some key will directly jump to the first entry starting with that letter.

If you manage multiple remote queues, you can also start multiple instances of XQM with different profiles to query all Agents in parallel.

Add a profile:
After your have finished your config settings, you can save your configuration as a profile by clicking the ADD button.
Delete a profile:
To delete a profile, just select it with the pulldown menue and click the DEL button.

Rename a profile:
Select the profile, click DEL, click ADD.

Edit a profile:
All changes are automatically saved to the currently selected profile, when you click the Apply button of the configuration dialog.

Updating happens without further notification. So if you generate a new profile based on an already existing one, remember to click the ADD button to assign a new name to the new profile before pressing Apply. Pressing Apply directly, would update the currently selected profile.

Local Mode |

Force MailRoot
During startup, XQM tries to determine the MailRoot path from its environment or the registry (on win32 platform). If you enter a path here, all environment settings are ignored. This is handy when managing multiple queues on the same machine or when no environment variable is set. Enter the path without a slash at the end or use the browse dialog.

Force service name (only on win32 platform)
If you use a non-standard name for the XMail Server service, then enter it here.This is useful, when managing multiple servers running on the same machine.

A note about multiple installations ...
Running multiple servers requires all further instances to be a XMail Server >=1.16.
Running multiple instances of XQM Agent requires all further instances to be >=v1.19. Forcing remote MailRoot and servicename is done in the Agent's .conf file.
There is no need to install multiple versions of XQM, since it is capable of profile management.

CTRL access
XQM needs access to the CTRL protocol in order to apply the Queue Flush Function. If you don't manage a remote host, you'll have to configure IP and port of the CTRL service, that runs on your local machine. When managing a remote host, this part is configured in the Agent's .conf file.

It's good practice to allow CTRL access only from your local machine, e.g. on interface 127.0.0.1. So your sensible CTRL account information is never sent across any network connection.

Number of threads
XQM spawns up to 25 threads. Each of them collects the information from one or more segments of the nested spoolfolder structure in parallel.

Since threads consume resources (like RAM, CPU power ...), the number of threads was made scaleable to allow running XQM on slower machines with the best possible result.

Just experiment with the slider to find out which value fits best for your individual environment. Keep in mind, that the first query is always slower, because the spoolfolder structure is cached for further queries.

Remote Mode |

Remote Queue Management allows you to manage a remote host on the same or another platform. You can e.g. run XQM on Win2000, managing a Linux machine or the other way around. The setup for Remote Queue Management just needs the IP Address or NetBIOS name of the remote host + the Agent's port number (e.g. 8000)

Additionally you need XQM Agent running on the host. XQM and its Agent talk to each other with the XML/RPC (Remote Procedure Call) protocol, which is based on HTTP-requests. That means it can be routed through any firewall, which allows the user defined communication port.

Security:
Generally no password informations are sent across a network connection. Only predefined, trusted IP-addresses, subnets or machine names are allowed to speak to the agent. All other requests are rejected. These trustees can be mixed and are defined in the file xqmagentwin.conf, which resides on the remote host. Here are some config examples:

Scenario | XQM Networking Dialog | The Agent's xqmagentwin.conf (or xqmagent.conf on Linux)
Only 2 clients (192.168.0.10 and 192.168.0.11) should be able to manage the server (192.168.0.1) | Remote: 192.168.0.1 Port: 8000 | THREADS=25 AGENT_HOST=192.168.0.1 AGENT_PORT=8000 AGENT_DYNIP=0 TRUSTED_CIENTS=192.168.0.10,192.168.0.11 CTRL_HOST=127.0.0.1 CTRL_PORT=6017
All clients in the server's subnet should be able to manage the server. | Remote: 192.168.0.1 Port: 8000 | THREADS=25 AGENT_HOST=192.168.0.1 AGENT_PORT=8000 AGENT_DYNIP=0 TRUSTED_CIENTS=192.168.0.* CTRL_HOST=127.0.0.1 CTRL_PORT=6017

Your Server is located somewhere in the internet. You want to manage it from your LAN. Since client and server lease dynamic IP addresses from their ISPs, you use a free dynamic name service, e.g. no-ip.com . The Agent should take care for IP-changes automatically.

Remote:
servername.no-ip.com
Port: 8000

THREADS=25
AGENT_HOST=server.no-ip.com
AGENT_PORT=8000
AGENT_DYNIP=1
TRUSTED_CIENTS=clientname.no-ip.com
CTRL_HOST=127.0.0.1
CTRL_PORT=6017

The Agent and Dynamic IP Addresses:
You should keep in mind, that a change of you IP address makes the agent's service temporary unavailable. The agent binds itself to the new IP, as soon he detects a change. But XQM will only be able to reconnect, when your dynamic nameservice client has updated your remote host's DNS cache. This behavior cannot be influenced or accellerated by the agent itself. This can lead to a delay between 1 and approx 15 minutes.

The Agent and Threads:
The number of threads used by the Agent can be scaled with the THREADS varliable in the .conf file. If it takes very long until XQM gets the results from its Agent, you should experiment with this value. A thread number between 2 and 8 is recommended for slow machines, or machines with less RAM. Remember to restart the Agent to apply .conf file changes.

Multiple Agents:
The Agent also understands forced mailroot folders and forced service names (on Win32 platform), which allows to run multiple Agents in parallel. See the Agent's .conf file for more details. No - this does not apply to Agent Smith (Matrix Reloaded ;-)

XMail SMTP Error Messages |

This is a list of the SMTP error messages, that XMail Server uses to report delivery problems. It will help you when tracking undelivered or delayed mails.

Error codes starting with a 4 are temporary errors. In that case XMail retries to deliver the mail. Codes starting with a 5 are final errors without further delivery attempts. SLOG files might also contain error codes of the recipient's SMTP, which are build up in a similar way (see RFC821).

Log entries are not added to the SLOG files, but to the logs which XMail writes to the MailRoot/logs folder. They are listed here for completeness and for your comfort.

My thanks for the original list go to dario@henry.it , who did a lot of work by filtering this informations out of the XMail Server source code.

| 421 Service not available, closing transmission channel

Possible Reason: The SMTP session reached its timeout (see command line: -St timeout) Xmail cannot create a new SMTP session. E.g. maximum number of threads reached (see command line: -SX nthreads) or out of system resources.

| 451 Requested action aborted: local error in processing

Possible Reason: Xmail cannot create message file or set the related parameters/arguments. Possible causes: disk full, hardware failure, network error ....

| 452 Mailbox full

Possible Reason: User Account exceeded quota (reached MaxMBSize in USER.TAB) or insufficient disk space to receive the message Log Entry: RCPT=EFULL

| 500 Syntax error, command unrecognized

Possible Reason: The sender issued a command that Xmail doesn’t understand

| 501 Command not accepted

Possible Reason: Negative response to the VRFY command (AllowSmtpVRFY = 0 or not set at all in SERVER.TAB). Negative Response to the ETRN command (AllowSmtpETRN = 0 or not set at all in SERVER.TAB Log Entry: VRFY=EACCESS or ETRN=EACCESS

| 501 Syntax error in forward path

Possible Reason: The sender made a syntax error in the RCPT TO:<xxx> command. Log Entry: RCPT=ESYNTAX

| 501 Syntax error in return path

Possible Reason: Negative response to the MAIL FROM:<> command if "Null Sender" is not allowed (AllowNullSender = 0 or not set at all in SERVER.TAB). The return path (MAIL FROM:<xxx>) has syntax errors or is too long. Log Entry: SNDR=EEMPTY or SNDR=ESYNTAX

| 503 Bad sequence of commands

Possible Reason: The sender issued a bad sequence for one of the following commands: HELO, EHLO, MAIL FROM, RCPT TO, DATA or AUTH Log Entry: SNDR=EEMPTY or SNDR=ESYNTAX

| 504 Unknown authentication

Possible Reason: Xmail doesn’t understand the authentication method used by the sender

| 504 You are registered as spammer

Possible Reason: The sender's IP address is listed in SPAM-ADDRESS.TAB. Log Entry: SNDR=ESPAM

| 505 Y our domain has not DNS/MX entries

Possible Reason: No SMTP server was found after issuing the MAIL FROM:<xxx> command (only when CheckMailerDomain = 1 is set in SERVER.TAB). Log Entry: SNDR=ENODNS

| 550 Relay denied

Possible Reason:

- The recipient's domain (RCPT TO:<xxx>) is not listed in Xmail’s DOMAINS.TAB,
- the sender's IP address is not listed in SMTPRELAY.TAB,
- there is no (or an incomplete) user folder inside /domains/domainname/,
- or your .tab file indexes are corrupted. To fix this, stop XMail Server, delete MailRoot/tabindex/* and restart XMail.

Log Entry: RCPT=ERELAY

| 550 Account disabled

Possible Reason: The recipient's account is disabled (ReceiveEnable = 0 in USER.TAB) Log Entry: RCPT=EDSBL

| 550 Mailbox unavailable

Possible Reason: The recipient’s domain is local, but no user or alias matches the username part of the recipient's address (USER.TAB or CMDALIASES sub directory) Log Entry: RCPT=EAVAIL

| 550 String does not match anything

Possible Reason: Negative response if VRFY is enabled (AllowSmtpVRFY = 1 in SERVER.TAB)

| 551 Server use forbidden

Possible Reason: Receiving mail is locked because Xmail requires authentication from any sender (mail-auth=1 in SERVER.TAB)

| 550 Server access forbidden by your IP

Possible Reason: The sender's domain is listed in SMTP-RDNSCheck or in CustMapsList (SERVER.TAB). Log Entry: SNDRIP=ERDNSSNDRIP=EIPMAP

| 550 Too many recipients

Possible Reason: Exceeded the maximum number of recipients (RCPT TO:). See command line: -Sr maxrcpts. Log Entry: RCPT=ENBR

| 552 Message exceeds fixed maximum message size

Possible Reason: Exceeded the maximum message size (MaxMessageSize in USER.TAB or SERVER.TAB). Log Entry: SIZE=EBIG

| 557 Message exceeds fixed maximum message size

Possible Reason: The User cannot post to a mailing list (check MLUSER.TAB RW attributes). Log Entry: RCPT=EACCESS

Any Bugs ? |

If you have any additional informations that you'd like to see in that document or if you find any bugs, please send a mail to h_schneider@marketmix.com

All the best,
Harald