Sender Verify Callout, Driving me crazy!!! Options

[Tomas]

Hi,

I installed the spam/virtus protection on all my servers, but now I'm getting emails from my customers about several emails not being delivered, upon investigation, I was able to see the following:

2006-11-24 10:44:42 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:2955 I=[69.65.106.226]:25 U=root
2006-11-24 11:39:07 H=(lnxsvr.rush.com.pe) [200.60.77.168]:3272 I=[69.65.106.226]:25 sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 11:39:07 H=(lnxsvr.rush.com.pe) [200.60.77.168]:3272 I=[69.65.106.226]:25 F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not complete
sender verify callout
2006-11-24 11:44:08 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:3272 I=[69.65.106.226]:25
2006-11-24 12:38:45 H=(lnxsvr.rush.com.pe) [200.60.77.168]:3664 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 12:38:45 H=(lnxsvr.rush.com.pe) [200.60.77.168]:3664 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout
2006-11-24 12:43:45 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:3664 I=[69.65.106.226]:25 U=root
2006-11-24 13:52:59 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4006 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 13:52:59 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4006 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout
2006-11-24 13:57:59 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:4006 I=[69.65.106.226]:25 U=root
2006-11-24 14:48:38 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4402 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 14:48:38 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4402 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout
2006-11-24 14:53:38 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:4402 I=[69.65.106.226]:25 U=root
2006-11-24 15:39:29 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4680 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 15:39:29 H=(lnxsvr.rush.com.pe) [200.60.77.168]:4680 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout
2006-11-24 15:44:29 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:4680 I=[69.65.106.226]:25 U=root
2006-11-24 16:38:55 H=(lnxsvr.rush.com.pe) [200.60.77.168]:1040 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 16:38:55 H=(lnxsvr.rush.com.pe) [200.60.77.168]:1040 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout
2006-11-24 16:43:55 SMTP command timeout on connection from (lnxsvr.rush.com.pe) [200.60.77.168]:1040 I=[69.65.106.226]:25 U=root
2006-11-24 17:38:31 H=(lnxsvr.rush.com.pe) [200.60.77.168]:1342 I=[69.65.106.226]:25 U=root sender verify defer for <czapata@rush.com.pe>: Could not complete sender verify callout
2006-11-24 17:38:31 H=(lnxsvr.rush.com.pe) [200.60.77.168]:1342 I=[69.65.106.226]:25 U=root F=<czapata@rush.com.pe> temporarily rejected RCPT <lpacheco@quickferr.com>: Could not c
omplete sender verify callout


What can I do to stop this?

Thanks!!

[pairote]

Operating sender verify callout is not easy. Either DNS or non-rfc of reciever server can cause this. You can disable sender verify callout in WHM/EXIM configuration. Uncheck the checkbox for sender verify callout.

[Tomas]

But if I disable that then I wont be able to know who's sending email right?

[pairote]

>> But if I disable that then I wont be able to know who's sending email right?

No. Your mail server will not be able to verify if the sender is a valid sender. And you will get more spam.

[Tomas]

Thanks for your replies Pairote!!

I have 2 options here:

1. Use callouts to verify the existence of email senders.
2. Verify the existence of email senders.

Which one is it?

Thanks!

[pairote]

Use callouts to verify the existence of email senders.

[jameshsi]

Use callouts to verify the existence of email senders.

Hi!
I don't quite understand, you mean we should not take these 2 option off, we should just take off the call out option, right ?

But after I check out the call out in WHM, I still see this in my Advanced Editor:

#if it gets here it isn't mailman

#sender verifications are required for all messages that are not sent to lists

require verify = sender/callout
accept domains = +local_domains
endpass

Is that means I still using callout ?

[Tomas]

I did as you said Pairote but exim started to spit out exim and spamd restarts according to chkservd so I activated the option again.

[pairote]

Verify the existance of email senders

This option will verify if the MX record of the sender is valid.

Use callouts to verify the existance of email senders.

This option will SMTP back to the sender server and asking sender server if the sender is a valid email address. It is a good idea but some servers doesn't response even if the sender is valid. In that case, email will be rejected.

But after I check out the call out in WHM, I still see this in my Advanced Editor:

QUOTE
#if it gets here it isn't mailman

#sender verifications are required for all messages that are not sent to lists

require verify = sender/callout
accept domains = +local_domains
endpass


Is that means I still using callout ?

Not sure. It seems cPanel strip it later even it appear in EXIM GUI if the checkbox for callout is not checked. You may verify it by run
grep callout /etc/exim.conf
If no result, you don't run it.

[Tomas]

I'm not understanding, I got this:

root@nova [/home/tomas]# grep callout /etc/exim.conf
require verify = sender/callout

hehe

[pairote]

It means you are running sender callout. Try to remove it in WHM / EXIM configuration.

[dev_cw]

I have been having the same problem. Is there a way to have a whitelist for this test as well. I don't really want to disable the verify function but I DO want to whitelist a few addresses/servers that are not getting through. Is this possible?

[dev_cw]

The main reason that the sender verify can fail is because the remote mail server does not conform to standard RFCs. They issue a RCPT TO command to our server but do not wait long enough for a response. RFC 2821 #4.5.3.2 states that the mail server should wait up to 5 minutes for a response. As part of our email harvesting protection system we sometimes perform a delay on incoming connections.

Could this be a solution? Where can we configure exim to wait a few minutes for a response? By watching the logs it looks like it is immediate.

[pairote]

Replace your simple verify callout to something like this on your EXIM editor.

CODE

deny message = From email address must be valid
      # do not check address for lists or bounces
      # or people in our company contact database
      !senders = ^.*-request@.*:\
                ^bounce-.*@.*:\
                ^.*-bounce@.*:\
                ^owner-.*@.*:\
                ^listmaster@.*:\
                ^root@.*:\
                ^anonymous@.*:\
                ^nobody@.*
          !domains = +rv_callout_receiver_domain_whitelist
          !sender_domains = +rv_callout_sender_domain_whitelist
      # do not check for DSN-ignorant domains
      # those that don't accept MAIL FROM:<>
      !dnslists = dsn.rfc-ignorant.org/$sender_address_domain
      !verify  = sender/callout=10s,defer_ok
##
# Check that there is a MX record for those that do not
# meet the deny statement requirements - ie bounces
# No cost as previous lookup is cached if executed
##
     deny  !sender_domains = +rv_sender_verify_domain_whitelist
          !verify = sender

And add 3 new whitelist files at the first box of WHM/EXIM advanced editor. Don't forget to create these files in /usr/local/cpanel/base/eximacl/.

CODE

domainlist rv_callout_sender_domain_whitelist = lsearch;/usr/local/cpanel/base/eximacl/rv_callout_sender_domain_whitelist
domainlist rv_callout_receiver_domain_whitelist = lsearch;/usr/local/cpanel/base/eximacl/rv_callout_receiver_domain_whitelist
domainlist rv_sender_verify_domain_whitelist= lsearch;/usr/local/cpanel/base/eximacl/rv_sender_verify_domain_whitelist

Last updated on March 12, 2007.

[dev_cw]

Thanks pairote, that is a cool setup. Plugged it right in and worked fine. I have not seen any messages beeing blocked yet but I have my eyes open.

[dazza]

Thanks pairote, that is a cool setup. Plugged it right in and worked fine. I have not seen any messages beeing blocked yet but I have my eyes open.

so dev_cw,

I've recently been through "Sender Failure" hell myself and came across this post. It's been a month since your post, how's it looking? any issues?

Looks like a good solution, I just need to becareful not to block the wrong emails.

And would I replace the * with domains to whitelist or just leave as is? not sure how to decipher this.

Thanks,

daz

[dev_cw]

dazza,

I have been happy so far, no complaints from clients. I am still blocking hundreds (or thousands) of messages that do not pass sender verification and all my messages seem to be getting trough. It must be working since no one has called to complain about missed messages.

I would still like to have a main whitelist to bypass sender verification.

[dazza]

cool, I'll give it a try.


I haven't tried this either, still all a bit new to me, but this may work as well. whitelist link


Thanks for the reply.

[dazza]

I'm still getting the following through dnsstuff.com email check whether or not I have "use callouts to verify" checked or not:

Could not connect: Got an unknown RCPT TO response: 550-Verification failed for <TestedFrom-71.198.77.160@DNSreport.com>
550-Previous (cached) callout verification failure
550 From email address must be valid


I've replaced "require verify = sender/callout" in my exim file with pairote's code above. Email seems to be getting through, I'm just concerned about the 550 messages.

[dazza]

Seems to be working now. Below is what I have done. Could you tell me if you think this looks ok? Much appreciated.

In WHM:
Verify the existence of email senders. <-- is checked
Use callouts to verify the existence of email senders. <--- is checked.

Modified exim file:
Removed: /callout from ---> require verify = sender/callout

Replaced: deny message = $sender_fullhost is currently not permitted to \
relay through this server. Perhaps you \
have not logged into the pop/imap server in the \
last 30 minutes or do not have SMTP Authentication turned on in your email client.

With: deny message = From email address must be valid
# do not check address for lists or bounces
# or people in our company contact database
!senders = ^.*-request@.*:\
^bounce-.*@.*:\
^.*-bounce@.*:\
^owner-.*@.*:\
^listmaster@.*:\
^root@.*:\
^anonymous@.*:\
^nobody@.*
# do not check for DSN-ignorant domains
# iow those that don't accept MAIL FROM:<>
!dnslists = dsn.rfc-ignorant.org/$sender_address_domain
!verify = sender/callout=10s,defer_ok

root@xxx [~]# grep callout /etc/exim.conf
!verify = sender/callout=10s,defer_ok