diff --git a/content/posts/announcing-clipper/chrome-devtools-live.webm b/content/posts/announcing-clipper/chrome-devtools-live.webm new file mode 100644 index 0000000..3ff6ac0 Binary files /dev/null and b/content/posts/announcing-clipper/chrome-devtools-live.webm differ diff --git a/content/posts/announcing-clipper.md b/content/posts/announcing-clipper/index.md similarity index 72% rename from content/posts/announcing-clipper.md rename to content/posts/announcing-clipper/index.md index 676b6ee..8020c24 100644 --- a/content/posts/announcing-clipper.md +++ b/content/posts/announcing-clipper/index.md @@ -6,18 +6,24 @@ tags = ["clipper", "rust"] title = "Announcing Clipper: TLS-transparent HTTP debugging for native apps" +++ - +[Repo link][repo] + +[repo]: https://github.com/lf-/clipper + +{% image(name="wire_blahaj.jpg", colocated=true) %} +little blahaj wrapped in ethernet cable +{% end %} Sometimes, I get jealous of web developers for having real network debugging tools for watching HTTP traffic. Other times, I decide I can have nice things too and spend a few weeks writing a tool. -A while ago I was debugging an issue with the Rust OpenTelemetry library and it -was immensely frustrated by not being able to get the actual requests out of -it. Eventually it yielded when I forked `h2` to add some logging that would be -exceptionally inadmissible in production. This is (probably) not the fault of -either rust-opentelemetry or the HTTP library that the observability system was -not observable, but it sure was frustrating. +A while ago I was debugging an issue with the Rust OpenTelemetry library on my +computer and it was immensely frustrated by not being able to get the actual +requests out of it. Eventually it yielded when I forked `h2` to add some +logging that would be exceptionally inadmissible in production. This is +(probably) not the fault of either rust-opentelemetry or the HTTP library that +the observability system was not observable, but it sure was frustrating. Perhaps HTTP libraries should have a standard interface for dumping the raw requests they perform, but this is going to be filtered through what the @@ -27,8 +33,14 @@ quite variable. What if there *was* one tool that was universal and could read any HTTP traffic with no configuration, regardless of the app? What if debugging a modern native -app's HTTP traffic (HTTP3 coming soon) could be (almost) as easy as opening dev -tools in a browser? +app's HTTP traffic could be (almost) as easy as opening dev tools in a browser? + +I picked up the hammer and built one: + +
+ +
If only modern computers had a [Clipper chip] and you happened to be the NSA so you could just decrypt everything. Thankfully that is not the case. I suppose @@ -49,21 +61,27 @@ supports: ## Wire shark doo doo doo doo doo doo - +{% image(name="ssl-added-and-removed-here.png", colocated=true) %} +Notorious NSA PRISM slide titled "Current Efforts - Google", showing Google +Front End stripping off TLS before sending cleartext into Google datacenters. +{% end %} But it's encrypted! Sadly, "TLS added and removed here" is an architecture of the past, and for good reason. However, what if we had the session keys and decrypted the TLS so we could read it? - -It turns out that Wireshark can actually do this, if you have the keys already. -However, getting the keys is the hard part. +It turns out that Wireshark [can actually do this][wireshark-tls], if you have +the keys already. However, getting the keys is the hard part. + +[wireshark-tls]: https://wiki.wireshark.org/TLS - Also, it turns out that we *did* collectively standardize getting the keys out of TLS, and practically every TLS implementation implements it. Specifically, -there is a standard format called `SSLKEYLOGFILE`, originally implemented in -Mozilla NSS, for logging the decryption keys from TLS libraries. +there is a [standard format called `SSLKEYLOGFILE`][sslkeylogfile-spec], +originally implemented in Mozilla NSS, for logging the decryption keys from TLS +libraries. + +[sslkeylogfile-spec]: https://www.ietf.org/archive/id/draft-thomson-tls-keylogfile-00.html Cool, so we're done? Well, as much as it is implemented in the TLS libraries, code changes to client code are required to actually use it. This is probably @@ -72,19 +90,20 @@ for good reason, since it does break TLS: - `curl` gates it behind a compile flag that is off by default. Not sure about libcurl. - NSS gates it behind a compile flag which is off by default. -- Firefox only enables the NSS flag on nightly or other dev builds. +- Firefox only enables that NSS flag on nightly or other dev builds. - rustls requires you set a special field on the ClientSettings and ServerSettings structures, which downstream users do not do by default. - Go `crypto/tls` requires a similar method to rustls. - OpenSSL requires you call `SSL_CTX_set_keylog_callback` when initializing it. - Chromium implements the `SSLKEYLOGFILE` environment variable by default, yay. - - - Alarmingly, there is a bug report of this feature being abused by antivirus + Alarmingly, [there is a bug report of this feature being + abused][no-good-deed-goes-unpunished] by antivirus vendors to do TLS decryption, leading the Chromium developers to remove the - "your traffic is being intercepted" banner. So perhaps everyone else removing - it was a good idea. + "dangerous environment variable" banner. So perhaps everyone else removing + support was a good idea. + +[no-good-deed-goes-unpunished]: https://crbug.com/991290 ## Fiddler2, OWASP ZAP, mitmproxy @@ -128,7 +147,6 @@ As for getting the keys, either you can patch the binary at compile time runtime (fun, indistinguishable from malware behaviour). Since I want a magical user experience, runtime patching it is. - Some absolute sickos at [mirrord] made a system which allows a local process on a developer computer to interact with the network as if it is on another machine in a staging environment. This was achieved by using the excellent @@ -136,6 +154,7 @@ machine in a staging environment. This was achieved by using the excellent along with `LD_PRELOAD` to get their code running inside processes. [Frida]: https://frida.re +[mirrord]: https://metalbear.co/blog/mirrord-internals-hooking-libc-functions-in-rust-and-fixing-bugs/ Clipper also uses Frida GUM. The actual patches to extract keys aren't much: - In OpenSSL, hook `SSL_new` to first call `SSL_CTX_set_keylog_callback` on the @@ -143,10 +162,17 @@ Clipper also uses Frida GUM. The actual patches to extract keys aren't much: - In rustls, they use vtable dispatch over a field set in the client/server settings structure. This seems like a pain in the ass, so we instead hook the no-op key log functions to not be no-ops. -- In Go `crypto/tls` they use a similar mechanism to rustls and we do the same - thing. +- In Go `crypto/tls` they use a similar mechanism to rustls and we can probably + do the same thing to rustls. Once we have the keys, we can add universal `SSLKEYLOGFILE` support, or send the keys over a socket to the capturing Clipper instance. Both of these are implemented. +# Thanks + +- [edef] for getting mad at this with me +- [puck] for helping me with several complex Linux and Rust issues + +[puck]: https://twitter.com/puckipedia +[edef]: https://twitter.com/edefic diff --git a/content/posts/announcing-clipper/ssl-added-and-removed-here.png b/content/posts/announcing-clipper/ssl-added-and-removed-here.png new file mode 100644 index 0000000..3471c4e Binary files /dev/null and b/content/posts/announcing-clipper/ssl-added-and-removed-here.png differ diff --git a/content/posts/announcing-clipper/wire_blahaj.jpg b/content/posts/announcing-clipper/wire_blahaj.jpg new file mode 100644 index 0000000..26dafee Binary files /dev/null and b/content/posts/announcing-clipper/wire_blahaj.jpg differ diff --git a/sass/main.scss b/sass/main.scss index 725e08b..534d671 100644 --- a/sass/main.scss +++ b/sass/main.scss @@ -207,7 +207,7 @@ details { margin-bottom: 2em; } -img { +img, video { // make the alt text extremely obnoxious and harder to miss if the image // is missing font-size: 3rem;