Hi everyone, I’ve been trying to understand how MiTM setups like a transparent proxy work.

Obviously, the use-case here is in a personal scope: I’d like to inspect the traffic of some of my machines. I am aware that Squid can be a transparent proxy, and some might use the Burp Suite to analyse network traffic.

Could someone explain the basic networking and the concept of certificates in this scenario? I feel like I don’t understand how certificates are used well enough.


For example: I realise that if someone inserts a root certificate in the certificate store of an OS, the machine trusts said CA, thus allowing encrypted traffic from the machine to be decrypted. However, say the machine was trying to access Amazon; won’t Amazon have its own certificate? I don’t know how I’m confused about such a simple matter. Would really appreciate your help!

  • surewhynotlem@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    ·
    1 year ago

    If you are acting as the proxy for the user, then all web requests go through you. A normal proxy would just forward those on to the Amazon (e.g.) server.

    In your scenario, the proxy sends a cert to the client saying ‘I’m Amazon’, and the client believes it. The client talks to you like you’re Amazon, and you read all his messages. At the same time, you talk to Amazon pretending to be the user, so you can get all the correct Amazon responses to send back to the user.

    • MigratingtoLemmy@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      1 year ago

      the proxy sends a cert to the client saying ‘I’m Amazon’, and the client believes it

      Thanks, could you explain this a bit more? I didn’t understand what you mean by “sends a cert saying ‘I’m Amazon’”

      • Monkey With A Shell@lemmy.socdojo.com
        link
        fedilink
        English
        arrow-up
        7
        ·
        edit-2
        1 year ago

        A decrypting proxy acts like a local certificate authority and issues a certificate on behalf of the site to create one encrypted connection between the client and proxy while creating a second on with the original site on the other side.

        The client will get an error (or depending on the site and browser may refuse to connect at all in the case of pinned and preloaded certificates) indicating that they’re connecting to a site with a certificate from an untrusted issued unless they’ve been configured to trust the CA held on the proxy.

        The ‘break’ between the client side connection and the server side connection will be in plaintext and can be examined through internal tools or sent to external ones typically through ICAP.

        • MigratingtoLemmy@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          I see. Which would mean that in cases where the application/OS can be made to blindly trust the “fake” CA (by inserting a root certificate into the certificate store, like in Android), the proxy can simply send the certificate and the client will believe that the certificate comes from Amazon. The certificate list can be refreshed by flushing the cache, yes?

          Thanks for the explanation

          • stifle867@programming.dev
            link
            fedilink
            English
            arrow-up
            4
            ·
            1 year ago

            There are some cases where this would not work by the way. It’s called certificate pinning and it’s basically when an application comes with the trusted certificate for a host built-in. Even if you were to override it with a root certificate in the certificate store, the app simply wouldn’t use it.

              • stifle867@programming.dev
                link
                fedilink
                English
                arrow-up
                6
                ·
                1 year ago

                No, not at all. The request never hits the cache. The certificate is stored within the app and all internet communication is specifically pinned to said certificate. It doesn’t even ask your certificate store.

                  • stifle867@programming.dev
                    link
                    fedilink
                    English
                    arrow-up
                    2
                    ·
                    1 year ago

                    Yeah, unfortunately it’s a huge barrier if you’re wanting to see why your devices are phoning home and the data being sent. It makes it extremely difficult if not impossible for most people to bypass.

          • Monkey With A Shell@lemmy.socdojo.com
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            That’s the sum of it. Like others and I have noted some mobile apps (and Apple phones in particular have their entire OS configured to not trust any intercepted certs when attempting to speak to Apple home base) are prone to using certificate pinning and will reject the intercepted certs regardless of the trust store. It’s mostly beneficial for adjusting the browser.

            If I might ask, what’s the purpose of this proxy? Functionally there are a lot simpler and more efficient ways to block traffic from a phone. If it’s more for traffic inspection I’ve seen a couple VPN based pcap apps for Android that could get a lot more detail while a DNS filter could both control and give visibility to traffic from the device without all the cert hassles.

            • MigratingtoLemmy@lemmy.worldOP
              link
              fedilink
              English
              arrow-up
              2
              ·
              1 year ago

              Thanks, I didn’t realise that certificate pinning was this strict.

              This effort is to check if my mobile has a baseband processor that might be communicating with the internet. I want to know if my device has a backdoor in hardware. The idea with a VPN has me intrigued, could you tell me more about that?

              • Monkey With A Shell@lemmy.socdojo.com
                link
                fedilink
                English
                arrow-up
                3
                ·
                edit-2
                1 year ago

                https://play.google.com/store/apps/details?id=app.greyshirts.sslcapture

                Something like that should ship all traffic through a local VPN adapter and output a standard pcap file.

                Another option if you have a bit more fancy networking available is to set up a security onion instance, then mirror a port on the network and just capture everything at an on-wire level. That would also cover things beyond just web traffic to catch other things like ssh or whatever other remote connectivity could be in play. Seeing the content of the connection is different than just seeing the connections existence though. The endpoint generally has the best visibility before data gets pushed into a a connection but unless you start getting into kernel level debugging it can still be hard to see into the behavior of internal applications.

                https://play.google.com/store/apps/details?id=eu.faircode.netguard

                This is also a local VPN way of seeing all the outgoing traffic along with allowing control of it as a local firewall.

                • MigratingtoLemmy@lemmy.worldOP
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  1 year ago

                  Hi, thanks for the resources. However, I don’t think I’ll be able to decrypt the traffic from my mobile using this, yes? Using a VPN for this is a great idea though, this also happens to be how NoRoot firewalls work on Android

      • SheeEttin@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        That’s literally it. It sends a cert for amazon.com, that your client trusts, because the CA cert used to sign it is in your trusted store.

        • MigratingtoLemmy@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          I see, thanks for the explanation. Amazon sends the certificate to the proxy, and the proxy sends its own certificate, masquerading as amazon, to the user.

          Thanks

          • canni@lemmy.one
            link
            fedilink
            English
            arrow-up
            4
            ·
            1 year ago

            I think it’s important to understand how a typical SSL certificate is generated. Basically, there are a handful of companies that we have all agreeded to trust. When you download Chrome it comes with a set of trusted root certificates, so does your OS, etc. So when Amazon wants to create an SSL for amazon.com, the only way they can do that is by contacting one of those handful of trusted companies and getting them to issue a certificate that’s says Amazon.com. When you go to the site, you see a trusted party generated the cert and your browser is happy.

            When you create a new root certificate and install it on your computer, you become one of those companies. So now, you can intercept traffic, decrypt it, read it, reissue a certificate for amazon.com (the same way Amazon would have gotten one from the third parties), reencrypt it, and pass it along to the client. Because the client trusts you it’s still a valid certificate. But if you inspect the certificate on the client side the root signer will no longer be GoDaddy or whatever, it will be you.

            • MigratingtoLemmy@lemmy.worldOP
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              Basically, if I were to run a CA locally (on my network) and use that certificate in some OS, I should be able to set up a MiTM setup easily. Except that applications like web browsers don’t care and will utilise certificate pinning anyway, regardless of which certificates I keep in my certificate store in the OS.

              Also, let’s say I’m running HAProxy/Caddy/NGINX at home which acts as my SSL-termination point for my network. In this case, if a service wants to access the internet, do these reverse proxies encrypt the outbound traffic with a different certificate?

              • canni@lemmy.one
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                If I understand the question, the traffic in your local intranet will basically always be encrypted with your root cert. So client -> proxy with your cert, then normal internet encryption from proxy -> internet.

                For the apps, it depends on the app, but you can usually insert your cert into their store, it might just be different than the systems store. This could be hard to do on an non-rooted iPhone, idk. My experience is with Linux desktops. For example, in chromium based apps, there is a database in ~/.pki/nssdb that you can insert your cert into. Again, this is something I do at work where we have a very tightly controlled network and application stack. I would not recommend a MiTM proxy for your home environment, it will only cause headaches.

                • MigratingtoLemmy@lemmy.worldOP
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  1 year ago

                  Ah, I see. I suppose I got a bit confused, and wanted to confirm how encryption would work for outbound traffic. Just to be sure: when an application wants to send encrypted data to a website/service on the Internet, it will use the respective certificate for said website. However, if one runs a transparent proxy in the middle and inserts a root certificate in the certificate store of the application, said certificate becomes valid (and the de-facto pick) for all websites, and this is what the client begins to use for everything.

                  Is that how it works?

          • losttourist@kbin.social
            link
            fedilink
            arrow-up
            2
            ·
            1 year ago

            Yes, I think that ‘masquerading’ is the key bit to grasp. The MITM Proxy isn’t just intercepting the traffic, it alters the traffic as it passes through.