Mailscanner vesus Exiscan performance Options

[pairote]

Aric, I give up on MailScanner. Too much problem running with cPanel, and I am sure cPanel will break server that don't run MailScanner again and again.

On old MailScanner conf, I enable Virus Scan, Spamassassin3.0.0+Razor,+Pyzor+DCC+URIDNSBL+ few custom rules from rulesemporium.com on Xeon 2.8 GHz w/ 1 GB RAM. It cannot handle email at 25,000 mail/day.

I just changed to exiscan+clamd+Spamassassin3.0.0+Razor,+Pyzor+DCC+URIDNSBL+ few custom rules from rulesemporium.com. It looks a lot better than MailScanner, but still overload the server.

Latest Spamassassin3.0.0 spawn 5 spamd to handle email scanning + URIDNSBL that query to at least 5 list (not sure), it consume a lot of RAM. Our server run smoothly before upgrade to 3.0.0. Now I Just upgrade the RAM to 2 GB.. If it still overload the server, I will fire it.

[SarcNBit]

The cPanel changelog does mention a change in 9.9.0 build 64 in support of Spamassasin 3. I do not know if the change was simply to correct the subject rewrite problem or something more indepth (they do a poor job documenting changes in their changlog).

[Aric]

According to the bug report filed on this, it only fixes the SA 3 configuration changes.

It doesn't seem to affect anything else.

As for MailScanner, some official usage stats can be found here:

http://www.mailscanner.biz/maq/#rulespost

(scroll down to the bottom)

Pairote, it's probably not realistic to expect any CPANEL server (especially if it is busy) to be able to process a ton of mail each day with no performance hit, especially considering the large number of plugins you use. If you drop back to ONLY MailScanner (or exiscan), clamAV and SA3 with a basic set of rules you will get better performance.

Frankly, with that much mail daily, you'd probably be better served by a modest server running ONLY as a mail server rather than trying to process all that mail on your CPANEL server.

Just my opinion.

I can tell you that I've seen a Dual Xeon 2.4 w/ 2 GB RAM, and 15,000 RPM Ultra SCSI drives process more than 1.5 million messages in 24 hours when it is running MailScanner, ClamAV and SpamAssassin (again, only a mail server, no CPANEL).

Aric

[catwalk]

Pairote, how did you get on with the hardware upgrade? Did it work out better for you?

[pairote]

It is a lots better. No more overloading. In the next few days, I will release a document to configure Exim to run Spam & Virus Protecion. It is 18 pages, content is here.

1. Install required software and scripts
2. Virus Protection
2.1 Configure Exim to reject virus at SMTP time
2.2 Configure Exim to reject virus + sender whitelist + receiver whitelist
3. RBL and blacklists
3.1. Sender blacklist and remote mail server blacklist
3.2. RBL setting + sender whitelist + receiver whitelist + remote mail server whitelist
4. Spam Protection + sender whitelist + receiver whitelist
5. Integrate into user's cPanel allowing user enable/disable server-wide Virus and Spam Protection
6. Testing

[catwalk]

That's great news.

Will you be publishing your document in this forum or at Cpanel's?

[pairote]

The document can be found here:
http://www.rvskin.com/index.php?page=public/antispam

[Guest_Prixone_*]

I am getting this all the time;

2005-09-09 10:19:06 1EDimX-0006W9-V4 Error in system filter: unknown filtering command "and" near line 85 of filter file

what could be ?

can you please send the WHM -> exim configuration advanced ? (all the box already with changes.

i think all the installation where fine just the changes into that parts i had missing something.

[pairote]

Your exim system filter is wrong.
Download this file:
http://rvskin.com/download/antivirusandspam.exim.virus.spam
And fix your exim system filter.

[Guest_Guest_*]

THX you save my day, all working fine now.

By the way, i love this combination that why im trying it.

one last thing just to be sure i am ok now.

i am using the follow configs at: Main >> Service Configuration >> Exim Configuration Editor

am i missing anything or did anything wrong ?

First box(#!!# cPanel Exim 4 Config):

av_scanner = clamd:/var/clamd

acl_smtp_connect = acl_check_host

acl_smtp_mail = acl_check_sender

domainlist rv_rbl_receiver_domain_whitelist = lsearch;/usr/local/cpanel/base/eximacl/rv_rbl_receiver_domain_whitelist

hostlist rv_rbl_server_ip_whitelist = net-iplsearch;/usr/local/cpanel/base/eximacl/rv_rbl_server_ip_whitelist

addresslist rv_rbl_sender_address_whitelist= lsearch*@;/usr/local/cpanel/base/eximacl/rv_rbl_sender_address_whitelist

addresslist rv_spam_sender_address_whitelist= lsearch*@;/usr/local/cpanel/base/eximacl/rv_spam_sender_address_whitelist

hostlist rv_relay_hosts = net-iplsearch;/etc/relayhosts

log_selector = +address_rewrite+all_parents+arguments+connection_reject+delay_delivery+delivery
size+dnslist_defer+incoming_interface+incoming_port+lost_incoming_connection+que
e_run+received_sender+received_recipients+retry_defer+sender_on_delivery+size_re
ect+skip_delivery+smtp_confirmation+smtp_connection+smtp_protocol_error+smtp_syn
ax_error+subject+tls_cipher+tls_peerdn

The first three set of box(begin acl):

First BOX:

#!!# This ACL is used at the start of an incoming connection.
#!!# The tests are run in order until the connection is
#!!# either accepted or denied.
acl_check_host:

##
# Reject email sent from mail server IP listed in the blacklist
##
deny message = Host $sender_host_address is blocked
hosts = /usr/local/cpanel/base/eximacl/rv_server_ip_blacklist
delay = 20s

accept

#!!# This ACL is used for the MAIL FROM: command in an
#!!# incoming SMTP transaction. The tests are run in order until the
#!!# sender address is either accepted or denied.
acl_check_sender:

##
# Reject email sent from sender listed in the blacklist
##
deny message = Sender $sender_address is blocked
senders = /usr/local/cpanel/base/eximacl/rv_sender_address_blacklist
delay = 20s

accept

Middle BOX:

#!!# ACL that is used after the DATA command
check_message:
require verify = header_sender

##
# Reject messages with serious MIME container errors
##
deny message = This message contains malformed MIME ($demime_reason).
demime = *
condition = ${if >{$demime_errorlevel}{2}{1}{0}}

##
# Reject messages attach attach a file with a CLSID in the name
# which causes Windows to hide the file extension.
##
deny message = Hiding of file extensions(CLSID hidden) is not allowed.
regex = ^(?i)Content-Disposition:.*?)filename=\\s*"+((\{[a-hA-H0-9-]{25,}\})|((.*?)\\s{10,}(.*?)))"+\$

##
# Reject messages attach illegal extension files
##
deny message = We do not accept ".$found_extension" attachments here. If you \
meant to send this file then please package it up as a zip file and resend it.
# You might need to remove some of these extensions if you want to allow your user get these files
demime = ade:adp:bas:bat:chm:cmd:com:cpl:crt:eml:exe:hlp:hta:inf:ins :isp:jse:lnk:mdb:mde:msc:msi:msp:pcd:pif:reg:scr:sct:shs:url:vbs:vbe:wsf:wsh:wsc

##
# Reject email contains Virus
##
deny message = This message contains a virus or other harmful content ($malware_name)
demime = *
malware = */defer_ok

##
# Add X-Scanned Header
##
warn message = X-Antivirus-Scanner: Appears clean mail though you should still use an Antivirus

accept

[prixon]

sorry if i dont use my login here its bcz i just register and i cant receive e-mails hehe

another error detected was: unknown ACL verb in "check_recipient"
while i tryed to sent to myself a message with XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

that virus test thing

[pairote]

Your middle box is not correct. It should be like this. Be careful copy and paste from this forum. some words might be stripped. Some are sepearated to multi lines instead put in the single line.

#!!# ACL that is used after the RCPT command
check_recipient:
  # Accept if the source is local SMTP (i.e. not over TCP/IP). We do this by
  # testing for an empty sending host field.
  accept  hosts = :

    drop hosts = /etc/exim_deny
        message = Connection denied after dictionary attack
        log_message = Connection denied from $sender_host_address after dictionary attack

    drop message = Appears to be a dictionary attack
        log_message = Dictionary attack (after $rcpt_fail_count failures)
        condition = ${if > {${eval:$rcpt_fail_count}}{3}{yes}{no}}
        condition = ${run{/etc/exim_deny.pl $sender_host_address }{yes}{no}}
        !verify = recipient


  # Deny if the local part contains @ or % or / or | or !. These are rarely
  # found in genuine local parts, but are often tried by people looking to 
  # circumvent relaying restrictions.

  deny    local_parts  = ^.*[@%!/|]
        message = I've never seen @, %, !, /, or | in an e-mail. Neither should you!

  # Accept mail to postmaster and abuse in any local domain, regardless of the source,
  accept  local_parts  = postmaster:abuse
                  domains      = +local_domains

# Accept bounces to lists even if callbacks or other checks would fail
  warn    message      = X-WhitelistedRCPT-nohdrfromcallback: Yes
          condition    = \
          ${if and {{match{$local_part}{(.*)-bounces\+.*}} \
                    {exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
                {yes}{no}}

  accept  condition    = \
          ${if and {{match{$local_part}{(.*)-bounces\+.*}} \
                    {exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
                {yes}{no}}


  # Accept bounces to lists even if callbacks or other checks would fail
  warn    message      = X-WhitelistedRCPT-nohdrfromcallback: Yes
          condition    = \
          ${if and {{match{$local_part}{(.*)-bounces\+.*}} \
                    {exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}_${lc:$domain}/config.pck}}} \
                {yes}{no}}

  accept  condition    = \
          ${if and {{match{$local_part}{(.*)-bounces\+.*}} \
                    {exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}_${lc:$domain}/config.pck}}} \
                {yes}{no}}

  #if it gets here it isn't mailman
                                                                                                                                         

##
# Be polite and say HELO. Reject anything from hosts that havn't given
# a valid HELO/EHLO to us.
##
deny message = Bad HELO: Empty HELO, please see RFC 2821 section 4.1.1.1
condition = ${if eq{$sender_helo_name}{}{yes}{no}}
delay = 3s

##
#  Forged hostname -HELOs as one of my own IPs
##
# Forged HELO (our ip/hostname)
deny message = Forged HELO: you are not $sender_helo_name as that is our IP Address and you are not allowed to use it in HELO/EHLO as per RFC Standards.
!hosts = @[]
!hosts = +rv_relay_hosts
!authenticated = *
condition = ${if eq{$sender_helo_name}{$interface_address}{yes}{no}}
delay = 3s

##
# Forged hostname - HELOs as my own hostname or domain
##
deny message = Forged HELO: you are not $sender_helo_name our local domain and you are not allowed to use as per RFC Standards.
# accept helo which is in local_domain if we relay or had smtp auth
!hosts = @[]
!hosts = +rv_relay_hosts
!authenticated = *
condition = ${if match_domain{$sender_helo_name}{+local_domains}{yes}{no}}
delay = 3s

##
# Hacked HELO (DOMAIN.com) (constructed by viruses)
##
deny message = Hacked HELO: you are not $sender_helo_name
condition = ${if match {$sender_helo_name}{\N^[A-Z0-9]+\.[a-z]+$\N}{yes}{no}}
condition = ${if match {$sender_helo_name}{\N^[0-9]+\.[a-z]+$\N}{no}{yes}}
!hosts = @[]
!hosts = +rv_relay_hosts
!authenticated = *
delay = 3s

#sender verifications are required for all messages that are not sent to lists                                                                                                                                         
require verify = sender

##
# Reject email sent from server listed in DNS blacklists.
##
deny message = Message rejected because $sender_fullhost is blacklisted at $dnslist_domain see $dnslist_text
!hosts = @[]
!hosts = +rv_relay_hosts
!authenticated = *
# RBL Bypass Local Domain List
!domains = +rv_rbl_receiver_domain_whitelist
# RBL Whitelist Incoming hosts
!hosts = +rv_rbl_server_ip_whitelist
# RBL Bypass Sender Domain List
!senders = +rv_rbl_sender_address_whitelist
# The following is a list of RBL to check for spam.
dnslists = list.dsbl.org : \
sbl.spamhaus.org : \
relays.ordb.org
delay = 3s
                                                                                                                                       
##
# If the receiver domain is on this server, accept only the receiver email addresses that exist.
# Default address for the receiver domain have to set to :fail: to work with this ACL.
# If the default address set to :blackhole: or /dev/null, Exim will always think that email exist
# and pass to lower ACL. Domains being attacked by dictionary attack spam are suggested to set
# default address to :fail:.
##
accept domains = +local_domains
      endpass
      message = "The recipient cannot be verified. Please check all recipients of this message to verify they are valid."
      log_message = unknown user
      verify = recipient

                                                                                                                                         
  accept  domains = +relay_domains


  warn  message = ${perl{popbeforesmtpwarn}{$sender_host_name}}
        hosts = +relay_hosts
  accept  hosts = +relay_hosts
                                                                               
  warn  message = ${perl{popbeforesmtpwarn}{$sender_host_address}}
        condition = ${perl{checkrelayhost}{$sender_host_address}}
  accept  condition = ${perl{checkrelayhost}{$sender_host_address}}

  accept  hosts = +auth_relay_hosts
          endpass
          message = $sender_fullhost is currently not permitted to \
                        relay through this server. Perhaps you \
                        have not logged into the pop/imap server in the \
                        last 30 minutes or do not have SMTP Authentication turned on in your email client.
          authenticated = *

  deny    message = $sender_fullhost is currently not permitted to \
                        relay through this server. Perhaps you \
                        have not logged into the pop/imap server in the \
                        last 30 minutes or do not have SMTP Authentication turned on in your email client.


#!!# ACL that is used after the DATA command
check_message:
require verify = header_sender
##
# Reject messages with serious MIME container errors
##
deny message = This message contains malformed MIME ($demime_reason).
demime = *
condition = ${if >{$demime_errorlevel}{2}{1}{0}}

##
# Reject messages attach illegal extension files
##
deny message = We do not accept ".$found_extension" attachments here. If you meant to send this file then please package it up as a zip file and resend it.
# You might need to remove some of these extensions if you want to allow your user get these files
demime = bat:cmd:com:cpl:pif:reg:scr

##
# Reject messages attach attach a file with a CLSID in the name
# which causes Windows to hide the file extension.
##
deny message = Hiding of file extensions(CLSID hidden) is not allowed.
regex = ^(?i)Content-Disposition:.*?)filename=\\s*"+((\{[a-hA-H0-9-]{25,}\})|((.*?)\\s{10,}(.*?)))"+\$

##
# Add a warning header if email contains illegal extension files but acccept the message
##
warn message = X-Antivirus-Filetype: Infected - $found_extension
# You might need to remove some of these extensions if you want to allow your user get these files
demime = ade:adp:bas:bat:chm:cmd:com:cpl:crt:eml:exe:hlp:hta:inf:ins:isp:jse:lnk:mdb:mde:
sc:msi:msp:pcd:pif:reg:scr:sct:shs:url:vbs:vbe:wsf:wsh:wsc

##
# Add a warning header if email contains Virus but acccept the message
##
warn message = X-Antivirus-Scanner: Infected - $malware_name
demime = *
malware = */defer_ok

# Messages larger than 200k are accepted without spam scanning to reduce spamd load
accept condition = ${if >{$message_size}{200k}{true}}

##
# Reject spam messages with score over 15.
# Keep in mind that $spam_score_int is the messages score multiplied by ten.
##
deny message = Spam score too high ($spam_score)
    # Bypass Sender that usually send a lot of emails to minimize spamd load
    !senders = +rv_spam_sender_address_whitelist
    spam = mailnull:true/defer_ok
    condition = ${if >{$spam_score_int}{150}{1}{0}}

##
# Add a warning header if email scored between 12 and 15.
# Delete email in the System Filter File, if sender or receiver is not listed in the whitelists.
##
warn message = X-Exiscan-SA-Spam: Yes
    # Bypass Sender that usually send a lot of emails to minimize spamd load
    !senders = +rv_spam_sender_address_whitelist
    spam = mailnull:true/defer_ok
    condition = ${if >{$spam_score_int}{120}{1}{0}}

##
# Rewrite subject if email scored between 9 and 15.
##
# Always put X-Spam-Score header in the message.
# It looks like this:
# X-Exiscan-SA-Score: 6.6 (++++++)
# When a MUA cannot match numbers, it can match for an
# equivalent number of '+' signs.
warn message = X-Exiscan-SA-Score: $spam_score ($spam_bar)\
# Put X-Spam-Report header in the message.
# This is a multiline header that informs the user
# which tests a message has "hit", and how much a
# test has contributed to the score.
              \nX-Exiscan-SA-Report: $spam_report\
# For the subject tag, we prepare a new subject header in the
# ACL, then swap it with the original Subject in the system filter.
              \nX-Exiscan-SA-New-Subject: *SPAM* $h_subject:
    # Bypass Sender that usually send a lot of emails to minimize spamd load
    !senders = +rv_spam_sender_address_whitelist
        spam = mailnull:true/defer_ok
    condition = ${if >{$spam_score_int}{90}{1}{0}}


accept

[prixon]

Thanks man, helped me a lot, all working 100% now.

i got a lot blind into that tutorial because all that parts to make changes where a lot confuse also some times they repeat so i get lost.

Would be very nice if you siplify it or allow your exim.conf and virus.spam file there for download check. just a hit...

thx for all the help.