2011 - Monitoring the use of a Squid proxy server

category: howtoz/archive · date: 2011-02-16 · source: old.linuxor.sk/PROXY/MONITORING_VYUZIVANIA_PROXY_SERVERA_SQUID.txt

The Slovak original of this document: 2011 - Monitoring využívania proxy servera Squid (slovensky).

asciiart

MONITORING THE USE OF A SQUID PROXY SERVER
==========================================

        Daily/Weekly/Monthly statistics
        -------------------------------
        https://192.168.1.1/sarg

        RealTime statistics
        -------------------
        https://192.168.1.1/sqstat

 SARG - http://sarg.sourceforge.net/
=====================================

        Installing SARG
        ---------------
        # cd /tmp
        # wget "http://downloads.sourceforge.net/project/sarg/sarg/sarg-2.3.1/sarg-2.3.1.tar.gz?r=http%3A%2F%2Fsarg.sourceforge.net%2Fsarg.php&ts=1296214728&use_mirror=garr"
        # tar -xvf ./sarg-2.3.1.tar.gz
        # cd ./sarg-2.3.1
        # ./configure
        # make
        # make install

        Setting up the SARG script
        --------------------------
        # nano /usr/local/etc/sarg.conf

        Installing the helper script for SARG (sarg-reports)
        ----------------------------------------------------
        # cd /usr/local/bin
        # wget http://www.initzero.it/products/opensource/sarg-reports/download/sarg-reports
        # chmod +x ./sarg-reports

        Setting up sarg-reports
        -----------------------
        # nano /usr/local/bin/sarg-reports
        #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        ## Slovak Language
        SARG=/usr/local/bin/sarg
        CONFIG=/usr/local/etc/sarg.conf
        HTMLOUT=/var/www/sarg
        PAGETITLE="Statistiky Proxy servera $(hostname)"
        LOGOIMG=http://www.bigcorp.example/html/skin/image/02/common/ko_images/firma_logo.jpg
        LOGOLINK=http://www.bigcorp.example
        DAILY=Denny report
        WEEKLY=Tyzdenny report
        MONTHLY=Mesacny report
        EXCLUDELOG1="SARG: Zaznamov v subore: "
        EXCLUDELOG2="SARG: "
        #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

        Setting up CRON
        ---------------
        # nano /etc/crontab
        -------------------
        # SARG reports
        00 *       * * * root /usr/local/bin/sarg-reports today
        00 00      * * * root /usr/local/bin/sarg-reports daily
        00 01      * * 1 root /usr/local/bin/sarg-reports weekly
        30 02      1 * * root /usr/local/bin/sarg-reports monthly
        # SARG reports

        Restarting CRON
        ---------------
        # /etc/init.d/cron restart

 SQSTAT - http://samm.kiev.ua/sqstat/
======================================

        Installing SQSTAT
        -----------------
        # cd /tmp
        # wget http://samm.kiev.ua/sqstat/sqstat-1.20.tar.gz
        # tar -xvf ./sqstat-1.20.tar.gz
        # cd ./sqstat-1.20
        # mkdir /var/www/sqstat
        # cp /tmp/sqstat-1.20/* /var/www/sqstat
        # cp /var/www/sqstat/config.inc.php.defaults /var/www/sqstat/config.inc.php

        Configuring SQSTAT
        ------------------
        # nano /var/www/sqstat/config.inc.php
        -------------------------------------
        $squidhost[0]="127.0.0.1";
        $squidport[0]=8080;

 Setting up access to the statistics for selected users only
=============================================================
Required module: mod_auth_basic: http://httpd.apache.org/docs/2.2/mod/mod_auth_basic.html

# nano /etc/apache2/sites-enabled/default-ssl
---------------------------------------------
<Directory /var/www/sarg>
        Options Indexes MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
        AuthType Basic
        AuthName "pristup"
        AuthUserFile /etc/apache2/basic_passwords
        Require user statistiky
</Directory>

<Directory /var/www/sqstat>
        Options Indexes MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
        AuthType Basic
        AuthName "pristup"
        AuthUserFile /etc/apache2/basic_passwords
        Require user statistiky
</Directory>
---------------------------------------------

Creating the password for access to the directories "sarg" and "sqstat"
-----------------------------------------------------------------------
# htpasswd -c /etc/apache2/basic_passwords statistiky
------------------------------------------------------------
Adding password for statistiky in realm realm.
New password: <password>

LINKz
=====

        SARG - Squid Analysis Report Generator is a tool that allow you to view "where" your users are going to on the Internet.
        ------------------------------------------------------------------------------------------------------------------------
        http://sarg.sourceforge.net/sarg.php

        SARG-REPORTs - a helper script for generating daily, weekly and monthly reports
        -------------------------------------------------------------------------------
        http://www.initzero.it/products/opensource/sarg-reports/download/sarg-reports

        SqStat - realtime statistics of the Squid proxy server
        SqStat is a script which allows to look through active squid users connections. It use cachemgr protocol to get information from squid proxy server.
        ----------------------------------------------------------------------------------------------------------------------------------------------------
        http://samm.kiev.ua/sqstat/

Auxiliary LINKz
===============

        Setting the language
        --------------------
        http://tech.groups.yahoo.com/group/orso/message/4123

Current practice (checked 2026-10)

noteThe article above is kept as it was written in 2011. This section lists what has changed since and what to do instead today.

SARG installation: the article compiles 2.3.1 from a tarball fetched with wget over plain http and without any checksum. SARG's last release on SourceForge is 2.4.0 and Debian ships it as the package sarg, so install it with apt and let the distribution carry the fixes.
sarg-reports from a third-party site: the script is downloaded over http, made executable and run as root from cron every hour. Do not do that today; run the report job as an unprivileged account that can read access.log and write to the report directory.
sqstat: the project page at samm.kiev.ua no longer exists (the address now redirects to an unrelated site). sqstat talks to Squid with the old cachemgr protocol, and Squid 7 removed the cache_object: URI scheme together with cachemgr.cgi; the manager reports are now plain HTTP under /squid-internal-mgr/, see the command below.
Apache access control: Order allow,deny and Allow from all are Apache 2.2 syntax. In Apache 2.4 they work only through mod_access_compat and mixing them with Require is discouraged; here AuthType Basic with Require user statistiky alone is the whole rule, so delete the two old lines.
Password file: create it with htpasswd -c -B /etc/apache2/basic_passwords statistiky so the entry is a bcrypt hash.
What the reports show today: HTTPS requests reach the proxy as CONNECT host:443, so SARG lists host names and byte counts per user, not the pages visited.
Still right: serving the statistics only over https and only to a named user is correct, because the reports are personal data about the users of the proxy.

The live list of requests that sqstat used to show, on the proxy from the article (port 8080):

bash

$ curl http://127.0.0.1:8080/squid-internal-mgr/active_requests

Sources: