clipper post some more
This commit is contained in:
Jade Lovelace
parent
bea2d15b6b
commit
0cfb543828
5 changed files with 52 additions and 26 deletions
178
content/posts/announcing-clipper/index.md
Normal file
178
content/posts/announcing-clipper/index.md
Normal file
|
|
@ -0,0 +1,178 @@
|
|||
+++
|
||||
date = "2023-06-24"
|
||||
draft = true
|
||||
path = "/blog/announcing-clipper"
|
||||
tags = ["clipper", "rust"]
|
||||
title = "Announcing Clipper: TLS-transparent HTTP debugging for native apps"
|
||||
+++
|
||||
|
||||
[Repo link][repo]
|
||||
|
||||
[repo]: https://github.com/lf-/clipper
|
||||
|
||||
{% image(name="wire_blahaj.jpg", colocated=true) %}
|
||||
little blahaj wrapped in ethernet cable
|
||||
{% end %}
|
||||
|
||||
Sometimes, I get jealous of web developers for having real network debugging
|
||||
tools for watching HTTP traffic. Other times, I decide I can have nice things
|
||||
too and spend a few weeks writing a tool.
|
||||
|
||||
A while ago I was debugging an issue with the Rust OpenTelemetry library on my
|
||||
computer and it was immensely frustrated by not being able to get the actual
|
||||
requests out of it. Eventually it yielded when I forked `h2` to add some
|
||||
logging that would be exceptionally inadmissible in production. This is
|
||||
(probably) not the fault of either rust-opentelemetry or the HTTP library that
|
||||
the observability system was not observable, but it sure was frustrating.
|
||||
|
||||
Perhaps HTTP libraries should have a standard interface for dumping the raw
|
||||
requests they perform, but this is going to be filtered through what the
|
||||
library *thought* it did, rather than necessarily what it actually did. Plus,
|
||||
there's hundreds of HTTP libraries, so the experience would be undoubtedly
|
||||
quite variable.
|
||||
|
||||
What if there *was* one tool that was universal and could read any HTTP traffic
|
||||
with no configuration, regardless of the app? What if debugging a modern native
|
||||
app's HTTP traffic could be (almost) as easy as opening dev tools in a browser?
|
||||
|
||||
I picked up the hammer and built one:
|
||||
|
||||
<div class="image">
|
||||
<video controls alt="video of clipper showing curl requests in chrome devtools to the side" src="./chrome-devtools-live.webm">
|
||||
</video>
|
||||
</div>
|
||||
|
||||
If only modern computers had a [Clipper chip] and you happened to be the NSA so
|
||||
you could just decrypt everything. Thankfully that is not the case. I suppose
|
||||
the next best thing is to have code execution, a binary patching library, and
|
||||
insufficient fear of `LD_PRELOAD`.
|
||||
|
||||
[Clipper chip]: https://en.wikipedia.org/wiki/Clipper_chip
|
||||
|
||||
# What is this thing?
|
||||
|
||||
Clipper is a suite of tools for doing TLS-transparent HTTP interception. It
|
||||
supports:
|
||||
* Unprivileged packet capture that also catches keys, storing to PCAPNG files.
|
||||
* Attaching Chrome DevTools to unmodified native processes and viewing HTTP
|
||||
activity.
|
||||
* Viewing PCAPNG files in Chrome DevTools.
|
||||
* Extracting keys from unmodified applications.
|
||||
|
||||
## Wire shark doo doo doo doo doo doo
|
||||
|
||||
{% image(name="ssl-added-and-removed-here.png", colocated=true) %}
|
||||
Notorious NSA PRISM slide titled "Current Efforts - Google", showing Google
|
||||
Front End stripping off TLS before sending cleartext into Google datacenters.
|
||||
{% end %}
|
||||
|
||||
But it's encrypted! Sadly, "TLS added and removed here" is an architecture of
|
||||
the past, and for good reason. However, what if we had the session keys and
|
||||
decrypted the TLS so we could read it?
|
||||
|
||||
It turns out that Wireshark [can actually do this][wireshark-tls], if you have
|
||||
the keys already. However, getting the keys is the hard part.
|
||||
|
||||
[wireshark-tls]: https://wiki.wireshark.org/TLS
|
||||
|
||||
Also, it turns out that we *did* collectively standardize getting the keys out
|
||||
of TLS, and practically every TLS implementation implements it. Specifically,
|
||||
there is a [standard format called `SSLKEYLOGFILE`][sslkeylogfile-spec],
|
||||
originally implemented in Mozilla NSS, for logging the decryption keys from TLS
|
||||
libraries.
|
||||
|
||||
[sslkeylogfile-spec]: https://www.ietf.org/archive/id/draft-thomson-tls-keylogfile-00.html
|
||||
|
||||
Cool, so we're done? Well, as much as it is implemented in the TLS libraries,
|
||||
code changes to client code are required to actually use it. This is probably
|
||||
for good reason, since it does break TLS:
|
||||
|
||||
- `curl` gates it behind a compile flag that is off by default. Not sure about
|
||||
libcurl.
|
||||
- NSS gates it behind a compile flag which is off by default.
|
||||
- Firefox only enables that NSS flag on nightly or other dev builds.
|
||||
- rustls requires you set a special field on the ClientSettings and
|
||||
ServerSettings structures, which downstream users do not do by default.
|
||||
- Go `crypto/tls` requires a similar method to rustls.
|
||||
- OpenSSL requires you call `SSL_CTX_set_keylog_callback` when initializing it.
|
||||
- Chromium implements the `SSLKEYLOGFILE` environment variable by default, yay.
|
||||
|
||||
Alarmingly, [there is a bug report of this feature being
|
||||
abused][no-good-deed-goes-unpunished] by antivirus
|
||||
vendors to do TLS decryption, leading the Chromium developers to remove the
|
||||
"dangerous environment variable" banner. So perhaps everyone else removing
|
||||
support was a good idea.
|
||||
|
||||
[no-good-deed-goes-unpunished]: https://crbug.com/991290
|
||||
|
||||
## Fiddler2, OWASP ZAP, mitmproxy
|
||||
|
||||
These are all fine tools and perhaps better suited to this use case. However,
|
||||
in order to use any of these proxies, one needs to execute an actual
|
||||
woman-in-the-middle attack, which has Consequences:
|
||||
|
||||
- Proxy support is required
|
||||
- Support for adding trusted CAs is required
|
||||
- Key pinning must be disabled
|
||||
- It's *possible* that certain HTTP library compatibility bugs may be concealed
|
||||
by virtue of the proxy decoding and reencoding the data.
|
||||
|
||||
I would like to look at the *actual* traffic and not put anything relevant in
|
||||
the data plane to the greatest extent possible.
|
||||
|
||||
# What the hell crimes did you do to achieve *that*, jade
|
||||
|
||||
Glad you asked. Numerous!
|
||||
|
||||
Let's start at capturing packets. Normally this requires having high privilege
|
||||
on Linux in order to be able to bind a `AF_PACKET` socket to capture packets.
|
||||
However, `CAP_NET_RAW` privilege is in the eye of the beholder. Thanks to the
|
||||
technologies used by unprivileged containers, this is possible and fairly easy.
|
||||
|
||||
Specifically, by entering a new user namespace and network namespace, Clipper
|
||||
jails processes away from the host network onto a virtual network over which it
|
||||
has root due to the user namespace. Then it can open a `AF_PACKET` socket and
|
||||
send a handle out of the namespace to the host process over a `unix(7)` socket
|
||||
and capture everything going on in the namespace.
|
||||
|
||||
However, that's not sufficient, since the inner process cannot actually connect
|
||||
to anything from a new network namespace and *host* root would be required to
|
||||
attach virtual interfaces to the host network stack. Fortunately, again, the
|
||||
missing pieces were built for rootless containers: `slirp4netns` is a userspace
|
||||
NAT daemon that NATs a network namespace onto the host network, creating a
|
||||
`tap` interface with internet access inside the namespace.
|
||||
|
||||
As for getting the keys, either you can patch the binary at compile time
|
||||
(boring, may require patching dependencies), or you can patch the binary at
|
||||
runtime (fun, indistinguishable from malware behaviour). Since I want a magical
|
||||
user experience, runtime patching it is.
|
||||
|
||||
Some absolute sickos at [mirrord] made a system which allows a local process on
|
||||
a developer computer to interact with the network as if it is on another
|
||||
machine in a staging environment. This was achieved by using the excellent
|
||||
[Frida] GUM Rust bindings, which allow hooking arbitrary code in executables,
|
||||
along with `LD_PRELOAD` to get their code running inside processes.
|
||||
|
||||
[Frida]: https://frida.re
|
||||
[mirrord]: https://metalbear.co/blog/mirrord-internals-hooking-libc-functions-in-rust-and-fixing-bugs/
|
||||
|
||||
Clipper also uses Frida GUM. The actual patches to extract keys aren't much:
|
||||
- In OpenSSL, hook `SSL_new` to first call `SSL_CTX_set_keylog_callback` on the
|
||||
passed in context.
|
||||
- In rustls, they use vtable dispatch over a field set in the client/server
|
||||
settings structure. This seems like a pain in the ass, so we instead hook the
|
||||
no-op key log functions to not be no-ops.
|
||||
- In Go `crypto/tls` they use a similar mechanism to rustls and we can probably
|
||||
do the same thing to rustls.
|
||||
|
||||
Once we have the keys, we can add universal `SSLKEYLOGFILE` support, or send
|
||||
the keys over a socket to the capturing Clipper instance. Both of these are
|
||||
implemented.
|
||||
|
||||
# Thanks
|
||||
|
||||
- [edef] for getting mad at this with me
|
||||
- [puck] for helping me with several complex Linux and Rust issues
|
||||
|
||||
[puck]: https://twitter.com/puckipedia
|
||||
[edef]: https://twitter.com/edefic
|
||||
Loading…
Add table
Add a link
Reference in a new issue