2008 - Squid authentication and user monitoring

ladislav.hajzer@employer.example
2008

Odporucana literatura: http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/

The task: monitor the combinations of client IP address and user ID as users of the internal network reach the Internet through the Squid proxy (http, https, ftp, ...).
An addition to the task: try to solve it without outside programs, using squid alone

        Mozne autentizacne moduly pre squid
        -----------------------------------
        NCSA: uses the NCSA style for the file of user names and passwords
        LDAP: pouziva LDAP (Lightweight Directory Access Protocol)
        MSNT: pouziva Windows NT autentifikacnu domenu
        PAM: pouziva Linuxovu schemu PAM (Pluggable Authentication Modules)
        SMB: uses an SMB server such as Windows NT or Samba
        getpwam: uses the old UNIX password file
        SASL: pouziva na overovanie SASL (Simple Authentication and Security Layer)
        NTLM: pouziva NT LAN Manager

        Autentizacne moduly v squide (Standardne dodane zo squidom v Debian Etch)
        -------------------------------------------------------------------------
        digest_pw_auth
        getpwnam_auth
        ldap_auth
        msnt_auth
        ncsa_auth
        ntlm_auth
        pam_auth
        smb_auth
        yp_auth

        Autentizacne metody v squide
        ----------------------------
        basic a digest ( http://www.ietf.org/rfc/rfc2617.txt )

        -----------------------------------
        autentizacne parametre v squid.conf
        -----------------------------------

        authenticate_ttl hodnota
        ------------------------
        How long squid remembers a client's authentication. This option
        means the client has to authenticate again after the given time. It loses much of its point
        kedze moderne prehliadace maju zabudovanu funkciu ukladania hesiel. Na zamedzenie ukladania
        passwords by users, Microsoft tools for managing workstation policy can be used
        (group policy).

        authenticate_ip_ttl value
        -------------------------
        How long a client authentication stays bound to a given IP address.
        The point of this parameter is to stop users sharing passwords. Take care
        in situations where the address changes as a matter of course (dialup).

        What it does: every user on the LAN can reach the internet through the proxy only with their login and password, and only from one chosen address.
        ------

        Fit for purpose: not usable in this environment, since the addresses are handed out by DHCP dynamically and at random.
        ---------

        Pouzite subsystemy:
        - ip_user - externe ACL, ktore sluzi na kontrolu parov login-ip_adresa
        - ncsa_auth - the module that authenticates users by login and password

-------------------
Konfiguracia squidu
-------------------

        # nano /etc/squid/squid.conf
        ----------------------------
        [1]  auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
        [2]  auth_param basic children 5
        [3]  auth_param basic credentialsttl 2 hours
        [4]  auth_param basic realm Overenie pre pristup k internetu
        [5]  external_acl_type IP_USER %SRC %LOGIN /usr/lib/squid/ip_user_check -f /etc/squid/ip_user.conf
        [6]  acl IP_USER external IP_USER %SRC %LOGIN
        [7]  Acl all src 0.0.0.0/0.0.0.0
        [8]  Acl trusted_ports port 21 80 443
        [9]  Acl OVERENY proxy_auth REQUIRED
        [10] http_access deny !trusted_ports
        [11] http_access allow IP_USER
        [12] http_access allow OVERENY
        [13] http_access deny all
        ----------------------------END->/etc/squid/squid.conf

        [1]  the basic method of authenticating, using the ncsa module with the passwords held in /etc/squid/squid_passwd
        [2]  autentizacny proces moze mat maximalne 5 potomkov
        [3]  uzivatelske meno a heslo je platne 2 hodiny (proti replay utokom)
        [4]  The login dialog shows the text "Authentication for internet access"
        [5]  definujeme externe ACL s nazvom IP_USER
        [6]  pouzijeme externe ACL IP_USER a nazveme ho tiez IP_USER
        [7]  ACL s nazvom 'all' specifikuje vsetky IP adresy
        [8]  ACL s nazvom 'trusted_ports' specifikuje porty, ktorym doverujeme (21,80,443)
        [9]  TRUE ak autentizacia je uspesna
        [10] deny access unless it is one of the trusted ports
        [11] povolime povolene kombinacie login-ip_adresa
        [12] povolime overenych uzivatelov
        [13] zakazeme vsetko ostatne

------------------
Pridanie uzivatela
------------------

        # htpasswd /chroot/squid/etc/squid/squid_passwd user1
        -----------------------------------------------------
        New password: ****
        Re-type new password: ****
        Adding password for user user1
        -----------------------------------------------------

        # cat /chroot/squid/etc/squid/squid_passwd
        ------------------------------------------
        user1:EVigNSsqcpqd2

-----------------------------
Pridanie paru login-ip_adresa
-----------------------------

        # nano /chroot/squid/etc/squid/ip_user.conf
        -------------------------------------------
        10.62.2.22 user1

Squid ma dost obmedzene moznosti logovania (v zmysle delenia sprav do osobitnych suborov)
both for the events coming from squid itself and for those from the other subsystems.
Logovanie runtime udalosti (cache.log) sa da zvysit upravou direktivy: debug_options
(viac info na http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+16.+Debugging+and+Troubleshooting/16.2+Debugging+via+cache.log/)

Logovanie:
----------
access.log - every HTTP and ICP request (the format is set by the logformat directive in squid.conf)
cache.log - spravy o cinnosti squida (treba upravit parameter debug_options pre ziskanie detailnejsich sprav, default: ALL,1 )
store.log - informacie a status o ulozenych [neulozenych] objektoch (Information and status of [not] stored objects)

Monitorovanie:
--------------
When checking the pair login:ip_address, the external ACL "IP_USER" does not log (cache.log) in a way that
po spravnom/nespravnom pare boli v jednom riadku nasledovne informacie: ip_adresa, login, status.
The status "TCP_DENIED" can however be read out of access.log, which says that some address broke
the policy for internet access. The drawback is that it will hold a TCP_DENIED status for
every request that broke the permitted rules.

Autentizacny mechanizmus BASIC ma moznost logovat priebeh autentizacie uzivatela, ale aj napriek tomu
nema v jednom riadku potrebne informacie: ip_adresa, login, status (co by bol idealny stav).
This mechanism can however write to cache.log the fact that a user was seen on a new address:

- authenticateAuthUserRequestSetIp: user 'user2' has been seen at a new IP address (10.62.2.22)

To umozni sledovat overenych uzivatelov (ty, ktory zadali spravny login a heslo), ale neumozni sledovat
users trying login and password combinations. That can only be found out from access.log and the matching
the status "TCP_DENIED", which every request breaking the proxy policy has.

        What it does: every user on the LAN can reach the internet through the proxy only after authenticating with a login and password.
        ------

        Fit for purpose: usable in this environment, since the addresses are handed out by DHCP dynamically and at random.
        ---------

        Pouzite subsystemy:
        - ncsa_auth - the module that authenticates users by login and password

-------------------
Konfiguracia squidu
-------------------

        # nano /etc/squid/squid.conf
        ----------------------------
        [1]  auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
        [2]  auth_param basic children 5
        [3]  auth_param basic credentialsttl 2 hours
        [4]  auth_param basic realm Overenie pre pristup k internetu
        [5]  Acl localnet src 10.62.0.0/255.255.0.0
        [6]  Acl all src 0.0.0.0/0.0.0.0
        [7]  Acl trusted_ports port 21 80 443
        [8]  Acl OVERENY proxy_auth REQUIRED
        [9]  http_access deny !trusted_ports
        [10] http_access allow OVERENY
        [11] http_access allow localnet
        [12] http_access deny all
        ----------------------------END->/etc/squid/squid.conf

        [1]  the basic method of authenticating, using the ncsa module with the passwords held in /etc/squid/squid_passwd
        [2]  autentizacny proces moze mat maximalne 5 potomkov
        [3]  uzivatelske meno a heslo je platne 2 hodiny (proti replay utokom)
        [4]  The login dialog shows the text "Authentication for internet access"
        [5]  ACL s nazvom 'localnet' specifikuje siet 10.62.0.0/16
        [6]  ACL s nazvom 'all' specifikuje vsetky IP adresy
        [7]  ACL s nazvom 'trusted_ports' specifikuje porty, ktorym doverujeme (21,80,443)
        [8]  TRUE ak autentizacia je uspesna
        [9]  deny access unless it is one of the trusted ports
        [10] povolime overenych uzivatelov
        [11] povolime pristup pre nasu lokalnu siet
        [12] zakazeme vsetko ostatne

------------------
Pridanie uzivatela
------------------

        # htpasswd /chroot/squid/etc/squid/squid_passwd user1
        -----------------------------------------------------
        New password: ****
        Re-type new password: ****
        Adding password for user user1
        -----------------------------------------------------

        # cat /chroot/squid/etc/squid/squid_passwd
        ------------------------------------------
        user1:EVigNSsqcpqd2

Squid ma dost obmedzene moznosti logovania (v zmysle delenia sprav do osobitnych suborov)
both for the events coming from squid itself and for those from the other subsystems.
Logovanie runtime udalosti (cache.log) sa da zvysit upravou direktivy: debug_options
(viac info na http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+16.+Debugging+and+Troubleshooting/16.2+Debugging+via+cache.log/)

Logovanie:
----------
access.log - every HTTP and ICP request (the format is set by the logformat directive in squid.conf)
cache.log - spravy o cinnosti squida (treba upravit parameter debug_options pre ziskanie detailnejsich sprav, default: ALL,1 )
store.log - informacie a status o ulozenych [neulozenych] objektoch (Information and status of [not] stored objects)

Monitorovanie:
--------------
Autentizacny mechanizmus BASIC ma moznost logovat priebeh autentizacie uzivatela, ale aj napriek tomu
nema v jednom riadku potrebne informacie: ip_adresa, login, status (co by bol idealny stav).
This mechanism can however write to cache.log the fact that a user was seen on a new address:

- authenticateAuthUserRequestSetIp: user 'user2' has been seen at a new IP address (10.62.2.22)

To umozni sledovat overenych uzivatelov (ty, ktory zadali spravny login a heslo), ale neumozni sledovat
users trying login and password combinations. That can only be found out from access.log and the matching
the status "TCP_DENIED", which every request breaking the proxy policy has.

Writing our own log parser to give a better view of the successful ones is no answer either
a neuspesnych pokusoch o overenie z tychto zaznamov v cache.log, kedze funkcia 'authenticateBasicHandleReply'
nevrati dodatocne informacie ako ip adresa, login a podobne, dokonca nezaloguje ani identifikator sedenia (0x8458ec8)
the authentication was started for.

2008/02/08 09:58:03| authenticateStart: auth_user_request '0x8458ec8'
2008/02/08 09:58:03| authenticateStart: 'user2:heslo'
2008/02/08 09:58:03| authenticateAuthUserRequestLock auth_user request '0x8458ec8'.
2008/02/08 09:58:03| authenticateAuthUserRequestLock auth_user request '0x8458ec8' now at '2'.
2008/02/08 09:58:03| authenticateBasicHandleReply: {ERR Wrong password}

Relying on the logging happening in the right order during authentication, that is:

1. authenticateStart: auth_user_request '0x8458ec8'
2. authenticateStart: 'user2:heslo'
3. authenticateBasicHandleReply: {ERR Wrong password}

is impossible with a large number of authentications, because the individual authentications could
v logoch prekryvali a tym padom by boli tieto informacie nepresne resp. zle.

The only way is to edit squid's source so that it writes one line for a successful or failed
overenie s tymyto informaciami: login, ip_adresa, status overenia
V pripade implementacie tejto moznosti by bolo nutne kazdu novu verziu squida patchovat takymto patchom.
To zas prinasa viac casu potrebneho na administrovanie samotneho squida.

Zaujimave pasaze zo zdrojakov:

ncsa_auth.c - wrong password atd.
---------------------------------
http://www.google.com/codesearch?hl=en&q=+squid-2.6+Wrong+password+show:2AJNB7Vn9l0:tXUcVRzpsmQ:fGlsHwOlIu4&sa=N&cd=1&ct=rc&cs_p=http://ftp.osuosl.org/pub/nslu2/sources/squid-2.6.STABLE5.tar.gz&cs_f=squid-2.6.STABLE5/helpers/basic_auth/NCSA/ncsa_auth.c#l146

authenticateBasicHandleReply
----------------------------
http://www.google.sk/codesearch?hl=en&q=+authenticateBasicHandleReply+squid+squid-2.6+show:-bJZluBasIg:XXLk_SXxBvY:nHEKvGFY8j8&sa=N&cd=1&ct=rc&cs_p=http://ftp.osuosl.org/pub/nslu2/sources/squid-2.6.STABLE9.tar.gz&cs_f=squid-2.6.STABLE9/src/auth/basic/auth_basic.c#l259

authenticateAuthUserRequestSetIp
--------------------------------
http://www.google.sk/codesearch?hl=en&q=+squid+squid-2.6+authenticateAuthUserRequestSetIp+show:07UGmdsjrdk:X3DabSbAlX8:dQVDblQELU4&sa=N&cd=1&ct=rc&cs_p=http://ftp.osuosl.org/pub/nslu2/sources/squid-2.6.STABLE9.tar.gz&cs_f=squid-2.6.STABLE9/src/authenticate.c#l257

struct _auth_user_ip_t
----------------------
http://www.google.com/codesearch?hl=en&q=+auth_user_ip_t+squid-2.6+squid+show:EYULPiNc0vA:pX3cmmdT-k8:hNOzfCRYXsk&sa=N&cd=2&ct=rc&cs_p=http://ftp.osuosl.org/pub/nslu2/sources/squid-2.6.STABLE5.tar.gz&cs_f=squid-2.6.STABLE5/src/structs.h#l109

cache.log
access.log
squid.conf
ip_user
squid_passwd

        !!! Tutorial: Squid. The definitive guide !!! <- super
        ---------------------------------------------
        http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/

        Windows NT autentifikacia
        -------------------------
        http://www.papercut.com.hk/kb/index.php?n=Main.InstallingAndConfiguringSquidNTProxy

        step by step using samba to join a windows domain
        -------------------------------------------------
        http://www.onlamp.com/pub/a/onlamp/2008/04/01/step-by-step-using-samba-to-join-a-windows-domain.html

        Setting up Squid for NTLM Auth
        ------------------------------
        http://www.flatmtn.com/article/setting-squid-ntlm-auth

        Quick HOWTO : Ch32 : Controlling Web Access with Squid
        ------------------------------------------------------
        http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch32_:_Controlling_Web_Access_with_Squid

        Squid ACL Proxy Authentication with External Programs (ncsa)
        ------------------------------------------------------------
        http://www.devet.org/squid/proxy_auth/

        Monitoring access to Server SQUID
        ---------------------------------
        http://www.howtoforge.com/monitoring_squid

        SquidGuard Howto Guide
        ----------------------
        http://www.aerospacesoftware.com/squidguard-howto.html

        Skripty pre squid
        -----------------
        http://www.squid-cache.org/Scripts