Skip to main content

Setting up HTTPS inspection (mitm) with Windows Squid

I needed a way to inspect HTTPS traffic on my home network. Some dodgy browsing going on and i wanted to see it all, HTTPS included. Solution? Setup a squid proxy with ssl-bump configured to handle HTTPS.

I'm a linux guy so my windows admin is meh, setting up a traditionally unix-based service on my windows server 2012 was a bit of a mission. But here are my notes from getting it (as far as i can tell) working.

caution

You can follow all these steps on the wiki but just as i wished someone had written some of the documentation a little bit clearer, so i leave this here.

Requirements#

Installation#

run the MS Squid MSI install GUI:

squid gui install

when you're done you should have a little menu in the task bar, and when you run services.msc you should see a 'Squid for Windows' service in 'running' status.

run the Cygwin setup-x86_64.exe you downloaded:

squid gui install

when this is done you should have a new desktop shortcut to crack open a cygwin terminal when you need one. also a 'squid' terminal, which is just a 'cmd' session in the squid home directory:

squid gui install

Create your Proxy ROOT CA certificate#

I use the desktop cygwin shortcut to open a cygwin terminal to run some openssl commands to create your proxy servers ROOT Certificate Authority (CA) certificate.

I opened a cygwin terminal, changed into the squid installation directory /cygdrive/c/Squid-3.5/etc/ssl and ran my openssl command there:

cd /cygdrive/c/Squid-3.5/etc/sslopenssl req -new -newkey rsa:2048 -sha256 -days 365 -nodes -x509 -extensions v3_ca -keyout caproxy.pem -out caproxy.pem

generate a certificate in a format (DER) you can import into client browsers/computers (via certificate import wizard)

openssl x509 -in caproxy.pem -outform DER -out caproxy.der

I imported this .der certificate into the 'Trusted Root Certification Authorities' folder on a client PC on my network.

Configure squid.conf on server#

I modified the following section from wiki.squid-cache.org to match my installation.

http_port 3128 ssl-bump \  cert=/etc/ssl/caproxy.pem \  generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
# For squid 3.5.xsslcrtd_program /lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1ssl_bump bump all

Errors when restarting Squid#

When restarting for first time with the ssl settings enabled in your squid.conf you're going to run into a few errors. in /var/log/cache.log you'll see this: (ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".

and squid will finally crash out with this (you'll see this error in EventViewer as well): FATAL: The ssl_crtd helpers are crashing too rapidly, need help!

The Fix#

As you can see from the error message, we need to initialize our ssl cer db directory with run "ssl_crtd -c -s /var/lib/ssl_db".

now, open a 'Squid Terminal' from your desktop shortcut in Windows Server 2012 and navigate to where the 'ssl_crtd.exe' program is (e.g. for me, C:\Squid-3.5\lib\squid)

and run:

C:\Squid-3.5\lib\squid\ssl_crtd.exe -c -s C:\Squid-3.5\var\lib\ssl_db

key thing to note here:

  • the dir 'ssl_db' must NOT ALREADY EXIST. (or else you're going to have a very bad time)

Don't be a dumbass like me and follow the error message ssl_crtd: Cannot create blah blah into a vortex of online forums about it that point to "squid with cygwin is broken and therefore can never do ssl bumping for https traffic".

Success! (finally)#

squid crtd success

Restart Squid service (again)#

If your install and configuration was successful, check logfile /var/log/cache.log, and it should look like this:

Squid Cache (Version 3.5.26): Terminated abnormally.CPU Usage: 0.203 seconds = 0.125 user + 0.078 sysMaximum Resident Size: 1304576 KBPage faults with physical i/o: 52022017/08/25 20:58:42 kid1| Set Current Directory to /var/cache/squid2017/08/25 20:58:42 kid1| Starting Squid Cache version 3.5.26 for x86_64-unknown-cygwin...2017/08/25 20:58:42 kid1| Service Name: squid2017/08/25 20:58:42 kid1| Process ID 29442017/08/25 20:58:42 kid1| Process Roles: worker2017/08/25 20:58:42 kid1| With 3200 file descriptors available2017/08/25 20:58:42 kid1| Initializing IP Cache...2017/08/25 20:58:42 kid1| parseEtcHosts: /etc/hosts: (2) No such file or directory2017/08/25 20:58:42 kid1| DNS Socket created at [::], FD 52017/08/25 20:58:42 kid1| DNS Socket created at 0.0.0.0, FD 62017/08/25 20:58:42 kid1| Adding nameserver 8.8.8.8 from squid.conf2017/08/25 20:58:42 kid1| Adding nameserver 208.67.222.222 from squid.conf2017/08/25 20:58:42 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes2017/08/25 20:58:42 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:42 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:42 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:43 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:43 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:43 kid1| Logfile: opening log daemon:/var/log/squid/access.log2017/08/25 20:58:43 kid1| Logfile Daemon: opening log /var/log/squid/access.log2017/08/25 20:58:43 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:43 kid1| Store logging disabled2017/08/25 20:58:43 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects2017/08/25 20:58:43 kid1| Target number of buckets: 10082017/08/25 20:58:43 kid1| Using 8192 Store buckets2017/08/25 20:58:43 kid1| Max Mem  size: 262144 KB2017/08/25 20:58:43 kid1| Max Swap size: 0 KB2017/08/25 20:58:43 kid1| Using Least Load store dir selection2017/08/25 20:58:43 kid1| Set Current Directory to /var/cache/squid2017/08/25 20:58:43 kid1| Finished loading MIME types and icons.2017/08/25 20:58:43 kid1| HTCP Disabled.2017/08/25 20:58:43 kid1| Squid plugin modules loaded: 02017/08/25 20:58:43 kid1| Adaptation support is off.2017/08/25 20:58:43 kid1| Accepting SSL bumped HTTP Socket connections at local=[::]:3128 remote=[::] FD 21 flags=92017/08/25 20:58:44 kid1| storeLateRelease: released 0 objects2017/08/25 20:59:25 kid1| Set Current Directory to /var/cache/squid2017/08/25 20:59:25 kid1| Starting Squid Cache version 3.5.26 for x86_64-unknown-cygwin...2017/08/25 20:59:25 kid1| Service Name: squid2017/08/25 20:59:25 kid1| Process ID 1442017/08/25 20:59:25 kid1| Process Roles: worker2017/08/25 20:59:25 kid1| With 3200 file descriptors available2017/08/25 20:59:25 kid1| Initializing IP Cache...2017/08/25 20:59:25 kid1| parseEtcHosts: /etc/hosts: (2) No such file or directory2017/08/25 20:59:25 kid1| DNS Socket created at [::], FD 52017/08/25 20:59:25 kid1| DNS Socket created at 0.0.0.0, FD 62017/08/25 20:59:25 kid1| Adding nameserver 8.8.8.8 from squid.conf2017/08/25 20:59:25 kid1| Adding nameserver 208.67.222.222 from squid.conf2017/08/25 20:59:25 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| Logfile: opening log daemon:/var/log/squid/access.log2017/08/25 20:59:25 kid1| Logfile Daemon: opening log /var/log/squid/access.log2017/08/25 20:59:25 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:59:25 kid1| Store logging disabled2017/08/25 20:59:25 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects2017/08/25 20:59:25 kid1| Target number of buckets: 10082017/08/25 20:59:25 kid1| Using 8192 Store buckets2017/08/25 20:59:25 kid1| Max Mem  size: 262144 KB2017/08/25 20:59:25 kid1| Max Swap size: 0 KB2017/08/25 20:59:25 kid1| Using Least Load store dir selection2017/08/25 20:59:25 kid1| Set Current Directory to /var/cache/squid2017/08/25 20:59:25 kid1| Finished loading MIME types and icons.2017/08/25 20:59:25 kid1| HTCP Disabled.2017/08/25 20:59:25 kid1| Squid plugin modules loaded: 02017/08/25 20:59:25 kid1| Adaptation support is off.2017/08/25 20:59:25 kid1| Accepting SSL bumped HTTP Socket connections at local=[::]:3128 remote=[::] FD 21 flags=92017/08/25 20:59:26 kid1| storeLateRelease: released 0 objects2017/08/25 21:01:41 kid1| Starting new ssl_crtd helpers...2017/08/25 21:01:41 kid1| helperOpenServers: Starting 1/32 'ssl_crtd' processes2017/08/25 21:01:41 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 21:30:14 kid1| Error negotiating SSL on FD 11: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed (1/-1/0)2017/08/25 21:30:15 kid1| Error negotiating SSL on FD 11: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed (1/-1/0)

now, this config could probably do with some more work and testing, but this is the bare bones of it and i wanted to get it down before i didn't care about it anymore lol.

Set client computers to use proxy#

on client side computer:

  1. open 'internet options'
  2. go to 'communications' tab
  3. click 'LAN settings'
  4. check 'Use a proxy server'
  5. Address: IP of your proxy server
  6. Port: 3128

Get browsing and everything should connect/display for the client, no cert errors/issues, fully transparent.

Troubleshooting#

Reference#

full squid.conf file#

## Recommended minimum configuration:#
# Example rule allowing access from your local networks.# Adapt to list your (internal) IP networks from where browsing# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal networkacl localnet src 172.16.0.0/12  # RFC1918 possible internal networkacl localnet src 192.168.0.0/16 # RFC1918 possible internal networkacl localnet src fc00::/7       # RFC 4193 local private network rangeacl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443acl Safe_ports port 80      # httpacl Safe_ports port 21      # ftpacl Safe_ports port 443     # httpsacl Safe_ports port 70      # gopheracl Safe_ports port 210     # waisacl Safe_ports port 1025-65535  # unregistered portsacl Safe_ports port 280     # http-mgmtacl Safe_ports port 488     # gss-httpacl Safe_ports port 591     # filemakeracl Safe_ports port 777     # multiling httpacl CONNECT method CONNECT
## Recommended minimum Access Permission configuration:#
# Only allow cachemgr access from localhosthttp_access allow localhost managerhttp_access deny manager
# Deny requests to certain unsafe portshttp_access deny !Safe_ports
# Deny CONNECT to other than secure SSL portshttp_access deny CONNECT !SSL_ports
# We strongly recommend the following be uncommented to protect innocent# web applications running on the proxy server who think the only# one who can access services on "localhost" is a local user#http_access deny to_localhost
## INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS#
# Example rule allowing access from your local networks.# Adapt localnet in the ACL section to list your (internal) IP networks# from where browsing should be allowedhttp_access allow localnethttp_access allow localhost
# And finally deny all other access to this proxy#http_access deny all
# Squid normally listens to port 3128#http_port 3128
http_port 3128 ssl-bump \  cert=/etc/ssl/caproxy.pem \  generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
# For squid 3.5.xsslcrtd_program /lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1ssl_bump bump all
# Uncomment the line below to enable disk caching - path format is /cygdrive/<full path to cache folder>, i.e.#cache_dir aufs /cygdrive/d/squid/cache 3000 16 256

# Leave coredumps in the first cache dircoredump_dir /var/cache/squid
# Add any of your own refresh_pattern entries above these.refresh_pattern ^ftp:       1440    20% 10080refresh_pattern ^gopher:    1440    0%  1440refresh_pattern -i (/cgi-bin/|\?) 0 0%  0refresh_pattern .       0   20% 4320
dns_nameservers 8.8.8.8 208.67.222.222
max_filedescriptors 3200

full error output in /var/log/squid/cache.log#

2017/08/25 20:58:33 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument(ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument(ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument(ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument(ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:33 kid1| Logfile: opening log daemon:/var/log/squid/access.log2017/08/25 20:58:33 kid1| Logfile Daemon: opening log /var/log/squid/access.log(ssl_crtd): Uninitialized SSL certificate database directory: /var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".2017/08/25 20:58:33 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument2017/08/25 20:58:33 kid1| Store logging disabled2017/08/25 20:58:33 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects2017/08/25 20:58:33 kid1| Target number of buckets: 10082017/08/25 20:58:33 kid1| Using 8192 Store buckets2017/08/25 20:58:33 kid1| Max Mem  size: 262144 KB2017/08/25 20:58:33 kid1| Max Swap size: 0 KB2017/08/25 20:58:33 kid1| Using Least Load store dir selection2017/08/25 20:58:33 kid1| Set Current Directory to /var/cache/squid2017/08/25 20:58:33 kid1| Finished loading MIME types and icons.2017/08/25 20:58:33 kid1| HTCP Disabled.2017/08/25 20:58:33 kid1| Squid plugin modules loaded: 02017/08/25 20:58:33 kid1| Adaptation support is off.2017/08/25 20:58:33 kid1| Accepting SSL bumped HTTP Socket connections at local=[::]:3128 remote=[::] FD 21 flags=92017/08/25 20:58:33 kid1| WARNING: ssl_crtd #Hlpr1 exited2017/08/25 20:58:33 kid1| Too few ssl_crtd processes are running (need 1/32)2017/08/25 20:58:33 kid1| Closing HTTP port [::]:31282017/08/25 20:58:33 kid1| storeDirWriteCleanLogs: Starting...2017/08/25 20:58:33 kid1|   Finished.  Wrote 0 entries.2017/08/25 20:58:33 kid1|   Took 0.00 seconds (  0.00 entries/sec).FATAL: The ssl_crtd helpers are crashing too rapidly, need help!