Proxy - pip through the proxy, test commands (lab)

category: solutionz/proxy · date: 2019-12-31 · updated: 2026-10-03 · author: LALA

Proxy Solution · Config document · referenced from SELinux and client trust

noteThe pip install -U pip in the middle of this set is the failing run, kept on purpose; the fix is the last line. The export lines set the variables for the current shell only; the notes do not make them permanent. The step title in the notes says "on centos" while the lab host is RHEL 7.5; the notes do not say on which machine the test ran.

The command set of the lab's client test: install pip from EPEL, point the shell at the lab proxy 10.90.114.114:3128 with the http_proxy and https_proxy variables, run pip through it and watch it fail with CERTIFICATE_VERIFY_FAILED, then make pip trust the proxy's generated certificates by pointing REQUESTS_CA_BUNDLE at the system bundle, into which the proxy's own certificate had been merged with the anchors step of the lab CA certificate commands.

Item	Value
Host	the notes do not say; the proxy is addressed as `10.90.114.114`, its internal lab address, and the lab schema shows one client at `10.90.114.1`
Run as	root
Proxy	`http://10.90.114.114:3128` for both `http_proxy` and `https_proxy`
Client	`pip` from the EPEL `python-pip` package on Python 2.7 (the error names `/usr/lib/python2.7/site-packages`); the notes do not print the version `pip -V` returned
CA bundle	`/etc/pki/tls/certs/ca-bundle.crt`, the bundle `update-ca-trust` writes, which holds the proxy's certificate after the anchors step
Fix	`export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt`

The commands

Install the EPEL repository, update, install pip from it, show its version and upgrade it. The notes title the first step "Install epel-release on centos". As root; the outputs of these commands are not in the notes.

bash

$ yum -y install epel-release
$ yum -y update
$ yum -y install python-pip
$ pip -V
$ pip install --upgrade pip

Reproduce the problem: point the shell at the proxy and run pip through it. The notes head this part "PROBLEM - 1" with the step "Reproduce". As root, same shell.

bash

$ export http_proxy=http://10.90.114.114:3128
$ export https_proxy=http://10.90.114.114:3128
$ pip install -U pip

output 2 lines

Could not fetch URL https://pypi.python.org/simple/pip/: There was a problem confirming the ssl certificate: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) - skipping
Requirement already up-to-date: pip in /usr/lib/python2.7/site-packages

The solution: tell the requests library inside pip which CA bundle to verify against. The notes hold no second pip install after it; that it worked is implied by the step title "Solution" and nothing else. Same shell, as root.

bash

$ export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt

What the lines mean

Line	Meaning
`yum -y install epel-release`	the EPEL repository package; as I understand it, `python-pip` is not in the RHEL 7 base repositories, which is why EPEL is needed; the note says only "from epel"
`export http_proxy=…` and `export https_proxy=…`	the proxy for plain and for TLS URLs; both go to the same proxy port, and the `https_proxy` value starts with `http://` because the connection to the proxy itself is plain, the proxy then bumps the TLS inside
`pip install -U pip`	the request to `https://pypi.python.org/simple/pip/` goes through the proxy, which answers with a certificate for `pypi.python.org` signed by the proxy's own certificate; `pip` does not know that signer and refuses: `CERTIFICATE_VERIFY_FAILED`
`Requirement already up-to-date`	`pip` could not reach the index and reports what is installed; as I read the order of the notes, the earlier `pip install --upgrade pip` had run before the proxy variables were set, so the installed `pip` was already current
`export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt`	`pip` verifies with the `requests` library it bundles, and `requests` ships its own CA list instead of using the system's; `REQUESTS_CA_BUNDLE` points it at the system bundle, which holds the proxy's certificate after `update-ca-trust`

Why requests ignores the system trust store, and that pip ships a copy of it, is my understanding of the two and of my links at the end of SELinux and client trust; the notes state the problem and the one-line solution only.

Checked against pip 26.2.1

As built	Today
`export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt`	Still documented: pip's "HTTPS Certificates" page names the `--cert` option and the `PIP_CERT` variable for a different bundle and adds "It is also possible to use REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE environment variables"
`pip` ignores the system trust store	Since pip 24.2 "system certificates are used in addition to certifi to verify HTTPS connections", through the `truststore` package; pip 22.2 to 24.1 needed `--use-feature=truststore`. On a current pip a CA added with `update-ca-trust` is honoured without any variable
`pip` from EPEL on Python 2.7	pip 21.0 (2021-01-23) dropped Python 2; 20.3.4, released the same day, is the last pip for Python 2.7. Current pip 26.2.1 requires Python 3.10 or later; RHEL 9 ships `python3-pip-21.3.1`, RHEL 10 `python3-pip-23.3.2`
`--trusted-host` as the alternative (not used)	Still exists, with `PIP_TRUSTED_HOST`; it marks a host as trusted "even though it does not have valid or any HTTPS", which is a different and weaker thing than trusting the proxy's CA
`/etc/pki/tls/certs/ca-bundle.crt`	Still a symlink to `/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem`, written by `update-ca-trust` from the anchors directory, in the current `ca-certificates` package

The fix is still valid, and on any pip from 24.2 on it is no longer needed: the anchors step alone makes the proxy's CA trusted. The client itself is gone, Python 2.7 pip having been unsupported since January 2021. Whether pip config set global.cert would do the same permanently I could not verify on the pip page checked.