[Sysadmin] [ronr at linuxdude.com: bounces]

[David Kaufman]

Dean Mah <dmah at shaw.ca> wrote:
> ----- Forwarded message from Ron <ronr at linuxdude.com> -----
> Subject: bounces
>
> Regarding bounces,
> there seems to be an intermitant problem with resolving
> lists.evolt.org, I'm attaching the log entries. We have
> had no network issues and no down time that I'm aware of.
>
> There doesn't seem to be any other valid domains not
> resolving. If I can be of any assistance, let me know.
>
> Regards,
>
> Ron Robertson


I think the problem may be neuro's WATERS.COM dns server:

whois evolt.org:
  [snip most]
  Name Server:NS1.EVERYDNS.NET
  Name Server:NS2.EVERYDNS.NET
  Name Server:NS3.EVERYDNS.NET
  Name Server:NS.WASTERS.COM

and, further shooting the trouble...

  host lists.evolt.org NS1.EVERYDNS.NET:
    lists.evolt.org has address 216.40.227.23

  host lists.evolt.org NS2.EVERYDNS.NET:
    lists.evolt.org has address 216.40.227.23

  host lists.evolt.org NS3.EVERYDNS.NET:
    lists.evolt.org has address 216.40.227.23

is all good, but then:

  host lists.evolt.org NS.WASTERS.COM
    lists.evolt.org is a nickname for aa.houston.tx.us.evolt.org
    aa.houston.tx.us.evolt.org has address 216.40.227.23

NS.WASTERS.COM disagrees with the other name servers.  The IP address 
reported is the same, but (for some reason) it is reported as a CNAME 
alias for aa.houston.tx.us.evolt.org instead of an A record.  This might 
work sometimes but it can't be good.  When you try to do double-check 
that the reverse-ip resolves *back* to the name, it's not found using my 
nameserver, which is the way paranoid mailservers would perform this 
check, right?

  host aa.houston.tx.us.evolt.org
    Host not found.

it does resolve if I happen to ask NS.WASTERS.COM:

  host aa.houston.tx.us.evolt.org NS.WASTERS.COM
    aa.houston.tx.us.evolt.org has address 216.40.227.23

..but 3 dns requests out of 4 won't ask ns.waters.com, they'll ask 
EveryDNS and will get a not-found, since the other 3 EveryDNS servers 
haven't had aa.houston.tx.us.evolt.org setup on them.

William, I wasn't going to say anything, but why do you think we need to 
run our own DNS, anyway?  I've have all of my domains on EveryDNS for 
over a year and have been 100% satisfied with the top-notch service 
David runs there.  Is there some configuration option you can't do 
through the EveryDNS.net web interface?

If you just prefer to maintain (and back up) the configuration changes 
on a server that you control, the EveryDNS servers can be slaved to 
y/our DNS server *privately*, without modifying the public WHOIS record, 
so that EveryDNS slurps it's data off our locally-administered and 
security-hardened) server, but the bazillions of mailservers all over 
the net don't hit y/our machine directly -- they hit one of EveryDNS's 
very well connected, and geographically-, network- and 
registrar-redundant public dns servers.  But what are we hoping to gain 
by running our own DNS servers, other than more administration work for 
us?

I think we gotta remove this nameserver from the whois record, at least 
for now, as it could be causing bounces on more mail servers than just 
this one reported directly to Dean, and if we do put it back, not do 
that until after all of the nameservers are slaved properly, and 
generating consistent responses to queries.

A lot of mail servers have recently begun enforcing stricter checks on 
inbound SMTP connections to try to reduce spam and virus volumes.  My 
company just had an incident about 2 weeks ago where fully *half* of our 
major clients were suddenly rejecting mail from us, because we had no 
reverse IP setup for the external interface on our firewall.  Our 
mailserver has a publicly routable IP address, which worked for 
*inbound* connections, but we learned that it's *outbound* connections 
appeared to originate from our firewall.  The receiving mailservers were 
detecting what the actual origination IP address of our connections was 
(which was not who it claimed to be in its HELO message) and they 
apparently decided that IP addresses without reverse DNS mapping that 
lie in their HELO string similar enough to the typical sources of spam, 
phishing scams, viruses and so on, to reject all connections from.  The 
fact that this configuration change happened to many seemingly random 
external servers at once implies that some fairly common mailserver 
software out there was auto-updated that day, or a new release came out, 
that imposed this behavior by default.  Needless to say we had to setup 
reverse DNS for that IP address quickly.

Evolt's current DNS issue is more evil (in terms of ease of debugging), 
since that odd cname will only be reported to, and so its reverse ip 
will only be *required* by, one in four of the mailservers (that care to 
check), and those lookups will fail if they happen to be directed to any 
of the other three public DNS servers we publish.  Sticky, icky, and 
intermittent, indeed.

Please let's remove it from the public WHOIS forthwith.  If this 
continue to have to be manually maintained in two different places, the 
servers will inevitably eventually fall out of sync again, unless or 
until the servers are actually slaved.

-dave