Categories
network-traffic-debugging

Traffic Interception Tools

Want to inspect your traffic on your personal computer of mobile device?

A few tools you will find helpful for that task:

Questions

  1. Why can Proxyman see the headers or HTTPS traffic that it has not decrypted? As far as I know the whole packet is encrypted except for source and destination ip.

    According to security stackexchange – it is TCP that reveals the hostname. The path is hidden along with other HTTP headers.

  2. Why does Python keep a seperate certifcate store and not use the systems CAs on Mac?

  3. Why does firefox use its own root CA store?

To maintain a consistent experience no matter the platform or operating system and to keep the best interest of users at heart. More info on why Mozilla keeps its own root CA store.

After installing Proxyman’s Root CA then certain traffic can be decrypted on proxyman.

Firefox

If you do this for an entire app – like firefox – when you refresh the page firefox will warn you that it does not trust the certificate you are seeing. If you view the certificate it will be the one generated by proxyman.

If you accept the risk (and it is low risk as the proxyman cerificate was generated by you or the proxyman app) then you can now view the encrypted traffic.

The firefox warning:

Websites prove their identity via certificates, which are issued by certificate authorities.

Firefox is backed by the non-profit Mozilla, which administers a completely open certificate authority (CA) store. The CA store helps ensure that certificate authorities are following best practices for user security.

Firefox uses the Mozilla CA store to verify that a connection is secure, rather than certificates supplied by the user’s operating system. So, if an antivirus program or a network is intercepting a connection with a security certificate issued by a CA that is not in the Mozilla CA store, the connection is considered unsafe.

Error code: MOZILLA_PKIX_ERROR_MITM_DETECTED

Python

The proxyman certificate authority is stored in ~/.proxyman/proxyman-ca.pem

By default python won’t trust a cerfiticate signed by the proxyman-ca (self-signed).
I think python asks you to install the cerfiticates via the certifi packages which is a Python package for providing Mozilla’s CA Bundle.

If you enable TLS decryption on python from proxyman you will see your requests all fail with:

I am using httpie here

$ http https://number1.co.za

http: error: SSLError: HTTPSConnectionPool(host='number1.co.za', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))) while doing a GET request to URL: https://number1.co.za/

It is saying we cannot verify this certifcate because don’t know who issued it (a.k.a the certificate authority who verified it is unknown to us).

So we must tell it that it can be trusted by setting this env variable:

export REQUESTS_CA_BUNDLE=~/.proxyman/proxyman-ca.pem

Now it will work.

However when you disable TLS (SSL) proxying on proxyman – now the error will arise again.

export REQUESTS_CA_BUNDLE=

to unset it again and the requests will work.

To find the default place where certifi looks for the Ca bundle:

import certifi
certifi.where()
>>> .../site-packages/certifi/cacert.pem

Windows

On windows I has used fiddler.

Sources