[SunHELP] Sendmail Address Canonicalization on Solaris 10

[David Eisner]

On Wed, Jan 27, 2010 at 2:49 PM, David Eisner <deisner at gmail.com> wrote:
> I'm having a he.. of a time figuring this one out, and I believe it
> may have something to do with modifications to the version of sendmail
> that ships with Solaris.

I figured it out.  For anybody who runs into this in the future:

The difference between sendmail running on my Linux box and sendmail
running on my Solaris box is that on the Linux box, host
canonicalization begins by querying DNS.  On the Solaris box, it
begins with /etc/hosts:

Linux:
    [linux]$ /usr/sbin/sendmail -d38.20 -d8.2 -bt -Ctest.cf
    ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
    Enter <ruleset> <address>
    > Canon foo.bar.org
    Canon              input: foo . bar . org
    openmap()       host:host NULL: valid
    getcanonname(foo.bar.org), trying dns
    dns_getcanonname(foo.bar.org, trymx=1)
    getcanonname(foo.bar.org), found
    Canon            returns: foo . bar . org .

Sun:
    [sun]$ /usr/sbin/sendmail  -d38.20 -d8.2 -bt -Ctest.cf
    ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
    Enter <ruleset> <address>
    > Canon foo.bar.org
    Canon              input: foo . bar . org
    openmap()       host:host NULL: valid
    getcanonname(foo.bar.org), trying files
    text_getcanonname(foo.bar.org)
    getcanonname(baz.bar.org), found
    Canon            returns: baz . bar . org .

Note that on both the Linux and Sun box, /etc/nsswitch.conf has hosts
configured this way:

    hosts:      files dns

If you look at switch_map_find() in sendmail/conf.c, the difference is
probably that the _USE_SUN_NSSWITCH_ ifdef'd section is used under
Solaris, but not on Linux. I think the Linux code path winds up
punting to DNS.  I haven't carefully checked this, though.

In the sendmail source, It is text_getcanonname() in map.c that is
responsible for getting the canonical name from /etc/hosts, by way of
extract_canonname().  Long story short, the first hostname listed on
the /etc/hosts line that contains the searched-for hostname is
considered canonical (at least for FQDNs).

So by switching

    10.0.0.20    sun-internal.bar.org sun-external.bar.org

to

    10.0.0.20    sun-external.bar.org sun-internal.bar.org

in /etc/hosts, I was able to get the behavior I wanted.  Not sure what
you'd do if you didn't want address rewriting to always map to one
hostname.

-David

-- 
David Eisner     http://cradle.brokenglass.com