# HTTP Toolkit

HTTP Toolkit

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Android 17 enables certificate transparency, and breaks custom CAs

DevFeed: [Android 17 enables certificate transparency, and breaks custom CAs](<https://devfeed.tech/articles/android-17-enables-certificate-transparency-and-breaks-custom-cas-58460.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/android-17-certificate-transparency/>)

Author: HTTP Toolkit; Tim Perry

Published: 2026-09-23T08:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Certificate Transparency](<https://devfeed.tech/topics/certificate-transparency.md>), [Android](<https://devfeed.tech/topics/android.md>), [TLS (Transport Layer Security)](<https://devfeed.tech/topics/tls.md>), [Security](<https://devfeed.tech/topics/security.md>), [Security & Privacy](<https://devfeed.tech/topics/security-privacy.md>), [Reverse Engineering](<https://devfeed.tech/topics/reverse-engineering.md>), [Mobile](<https://devfeed.tech/topics/mobile.md>)

Tags: [android](<https://devfeed.tech/tags/android.md>), [android-17](<https://devfeed.tech/tags/android-17.md>), [certificate-transparency](<https://devfeed.tech/tags/certificate-transparency.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [developers](<https://devfeed.tech/tags/developers.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [reverse-engineering](<https://devfeed.tech/tags/reverse-engineering.md>), [security](<https://devfeed.tech/tags/security.md>), [testing](<https://devfeed.tech/tags/testing.md>), [tls](<https://devfeed.tech/tags/tls.md>)

### AI overview

This article explains how Android 17's default certificate transparency affects custom certificate authorities and traffic interception. It covers the role of trusted CAs in TLS, security and privacy research, reverse engineering, and app debugging and testing, and introduces a working solution for intercepting Android traffic.

### Source excerpt

Do you want to know what your phone is sending & receiving? Nowadays, that means you need to control who it trusts. In modern connections everything sent & received is encrypted by TLS, and the only practical way to intercept traffic on Android is to generate a certificate, make the phone or app trust it as a certificate authority (CA), and then use this to act as a fully trusted proxy to the real server. Being able to view & modify this traffic is important. It's the basis of a huge amount of security & privacy research, allowing researchers to confirm what an app is actually sharing, and whether it requests and manages your data securely. It's also fundamental to reverse engineering so people can investigate how other apps work, for tools like AdGuard to filter ad traffic on mobile, and for plenty of developers' processes for debugging and testing their own apps too. In Android 17 (rolling out as we speak) certificate transparency has now been enabled by default for all apps, in ways that'll break all of this. I've been looking into this since it was first announced in the early preview releases though, and I now have a working solution! If you don't care about the details and you just want to intercept all Android traffic, without worrying about any of this, go download HTTP Toolkit now. On the other hand, if you want to know what Android's changed here, learn about some TLS minutiae, and work out how to fix all this (especially if you use your own CAs locally), then read on: Certificate configuration for me, not for thee Android has been steadily making CA certificate configuration more and more difficult over the years: In Android 7 (API level 24, in 2016), Google stopped all apps from trusting CA certificates configured by users by default. Users can install a CA themselves into the user-level store, but it's ignored by all apps that don't actively opt-in to trusting it. Since then, on standard Android devices a normal user effectively cannot control who their

## Dictionary compression for web data: how it works and how to use it with Node.js

DevFeed: [Dictionary compression for web data: how it works and how to use it with Node.js](<https://devfeed.tech/articles/dictionary-compression-is-finally-here-and-it-s-ridiculously-good-19056.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/dictionary-compression-performance-zstd-brotli/>)

Author: HTTP Toolkit; Tim Perry

Published: 2026-02-23T13:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Compression](<https://devfeed.tech/topics/compression.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [JavaScript](<https://devfeed.tech/topics/javascript.md>), [Node.js](<https://devfeed.tech/topics/node-js.md>), [API](<https://devfeed.tech/topics/api.md>), [WebAssembly](<https://devfeed.tech/topics/web-assembly.md>), [JSON](<https://devfeed.tech/topics/json.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [api](<https://devfeed.tech/tags/api.md>), [browser](<https://devfeed.tech/tags/browser.md>), [browsers](<https://devfeed.tech/tags/browsers.md>), [compression](<https://devfeed.tech/tags/compression.md>), [data](<https://devfeed.tech/tags/data.md>), [http](<https://devfeed.tech/tags/http.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [json](<https://devfeed.tech/tags/json.md>), [node-js](<https://devfeed.tech/tags/node-js.md>), [performance](<https://devfeed.tech/tags/performance.md>), [standards](<https://devfeed.tech/tags/standards.md>), [webassembly](<https://devfeed.tech/tags/webassembly.md>)

### AI overview

This tutorial explains dictionary compression, where the compressor and decompressor share known data so compressed output can reference it directly. It discusses using previous responses or custom dictionaries to reduce the size of JavaScript bundles, WebAssembly files, and structured API responses, and demonstrates the technique with Node.js.

### Source excerpt

Dictionary compression could completely change how applications send data over the web. It's recently gained broad support, and offers absurd real-world traffic reductions: initial testing shows YouTube JS download size for returning desktop users shrinking up to 90% (!!!) compared to existing best-practice compression, while the Google search results HTML (arguably the most optimized content on the internet) shrinks nearly 50%. This works by initializing the (de)compression algorithm with a dictionary of data known in advance to both compressor & decompressor, so that the compressed data can just be references to that directly ("insert bytes 1 - 10,000 from the dictionary") without having to include the original data at all. This is applicable in a surprising number of scenarios, because most data we send (especially on the web) isn't completely novel or unpredictable. Today's JavaScript bundle shares 99% of its content with yesterday's JavaScript bundle - if the browser already has the old one, using that as a dictionary means you can compress down to (approximately) just the differences. This can work either using a previous response as the dictionary for the next response, or using an explicit custom dictionary; for many kinds of dynamic response, you do know large chunks of the data in advance, like all the keys in your API's JSON response, and many common values that might be included, and you can generate & preload a dictionary defining exactly this to efficiently cover those. In either case, this can drastically shrink JS bundles, WebAssembly files, known-structure API responses, or many other kinds of incrementally updated & diffable content - a lot of the worst offenders for bandwidth usage that have become very common on the modern web. This is now widely usable, safe to deploy without compatibility concerns, and surprisingly easy to set up. Here's a quick low-level demo for Node.js (v24.6+ or v22.19+) so you can play with the raw compression directly for

## Funding the OSS Stack: HTTP Toolkit & Open Source in 2025

DevFeed: [Funding the OSS Stack: HTTP Toolkit & Open Source in 2025](<https://devfeed.tech/articles/funding-the-oss-stack-http-toolkit-open-source-in-2025-19090.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/open-source-funding-in-2025/>)

Author: HTTP Toolkit; Tim Perry

Published: 2026-02-21T18:00:00Z

Content type: opinion

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Open Source](<https://devfeed.tech/topics/open-source.md>), [Maintainers](<https://devfeed.tech/topics/maintainers.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [business](<https://devfeed.tech/tags/business.md>), [funding](<https://devfeed.tech/tags/funding.md>), [maintainers](<https://devfeed.tech/tags/maintainers.md>), [open-source](<https://devfeed.tech/tags/open-source.md>)

### AI overview

HTTP Toolkit describes its 2025 open-source funding, reporting $7,820 paid to maintainers and a cumulative total exceeding $20,000. The article lists recipients including Frida, Node.js, Electron, Mocha, Tauri, Servo, and Styled Components, and discusses broader support for open-source projects.

### Source excerpt

HTTP Toolkit, like effectively all software businesses, depends on a huge quantity of open-source code for much of its fundamental functionality & infrastructure. Most of this is tirelessly maintained by volunteers, completely for free! This honestly is a great deal for the businesses, but it would be even better if these maintainers were actually rewarded for their hard work. As part of HTTP Toolkit's commitment to giving back to open source under the Open Source Pledge, a substantial chunk of revenue goes back into these open-source projects, to keep them healthy, reward maintainers for their tireless efforts, and encourage development on projects I care about to keep them moving forwards. What does that mean in practice? In 2025, httptoolkit paid $7,820 to open-source maintainers, taking the project's total past $20,000! Not bad for one person. $1200 to Frida $1100 to Node.js $320 to Electron $320 to Mobx $240 to blakeembrey $240 to szmarczak $240 to lpinca $240 to apocas $240 to anonrig $240 to Phosphor Icons $240 to mmaietta $240 to Regolith $240 to endoflife.date $240 to react-window $240 to fast-xml-parser $240 to node-datachannel $240 to Ajv $240 to Mocha $220 to louislam $220 to openapi-directory $200 to johnnyreilly $180 to Node.js Mobile $180 to Open Web Docs $160 to Tauri $140 to Servo $140 to Styled Components $40 to roderickvd $40 to Waydroid Open source has done a lot to power my tech education & career, and literally powers the foundations of HTTP Toolkit, so I'm more than happy to be able to support the upstream projects those depend on. None of these are earth-shaking amounts individually but cumulatively they add up, especially as other people and organizations add their own contributions in turn: Open Source Pledge companies are now collectively donating nearly $3 million a year in total back to maintainers! If contributions like the above were expected behaviour from everybody building on open-source work, OSS would be a very different place. Of

## HTTP/3 adoption is widespread in browsers and CDNs but limited in open-source tools

DevFeed: [HTTP/3 adoption is widespread in browsers and CDNs but limited in open-source tools](<https://devfeed.tech/articles/http-3-is-everywhere-but-nowhere-19074.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/http3-quic-open-source-support-nowhere/>)

Author: HTTP Toolkit; Tim Perry

Published: 2025-03-12T16:00:00Z

Content type: opinion

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [HTTP](<https://devfeed.tech/topics/http.md>), [Network](<https://devfeed.tech/topics/network.md>), [Protocol (disambiguation)](<https://devfeed.tech/topics/protocol.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [browsers](<https://devfeed.tech/tags/browsers.md>), [http](<https://devfeed.tech/tags/http.md>), [http-3](<https://devfeed.tech/tags/http-3.md>), [http3](<https://devfeed.tech/tags/http3.md>), [networking](<https://devfeed.tech/tags/networking.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [quic](<https://devfeed.tech/tags/quic.md>), [standards](<https://devfeed.tech/tags/standards.md>)

### AI overview

This article examines the contrast between HTTP/3's widespread browser and CDN adoption and its limited support in major programming-language libraries, servers, Android networking, and Kubernetes tooling. It argues that HTTP/3 rollout beyond browsers and large CDNs remains at an early stage.

### Source excerpt

HTTP/3 has been in development since at least 2016, while QUIC (the protocol beneath it) was first introduced by Google way back in 2013. Both are now standardized, supported in 95% of users' browsers, already used in 32% of HTTP requests to Cloudflare, and support is advertised by 35% of websites (through alt-svc or DNS) in the HTTP Archive dataset. We've developed a totally new version of HTTP, and we're on track to migrate more than 1/3 of web traffic to it already! This is astonishing progress. At the same time, neither QUIC nor HTTP/3 are included in the standard libraries of any major languages including Node.js, Go, Rust, Python or Ruby. Curl recently gained support but it's experimental and disabled in most distributions. There are a rare few external libraries for some languages, but all are experimental and/or independent of other core networking APIs. Despite mobile networking being a key use case for HTTP/3, Android's most popular HTTP library has no support. Popular servers like Nginx have only experimental support, disabled by default, Apache has no support or published plan for support, and Ingress-Nginx (arguably the most popular Kubernetes reverse proxy) has dropped all plans for HTTP/3 support punting everything to a totally new (as yet unreleased) successor project instead. Really it's hard to point to any popular open-source tools that fully support HTTP/3: rollout has barely even started. This seems contradictory. What's going on? I'm going to assume a basic familiarity with the differences between HTTP/1.1 (et al), HTTP/2 and HTTP/3 here. If you're looking for more context, http2-explained and http3-explained from Daniel Stenberg (founder & lead developer of curl) is an excellent guide. Why do we need more than HTTP/1.1? Let's step back briefly. Why does this matter? Who cares about whether HTTP/3 is being rolled out successfully or not? If browser traffic and the big CDNs support HTTP/3, do we even need it in other client or server implementat

## HTTP Toolkit is joining the Open Source Pledge

DevFeed: [HTTP Toolkit is joining the Open Source Pledge](<https://devfeed.tech/articles/http-toolkit-is-joining-the-open-source-pledge-19091.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/open-source-funding-pledge/>)

Author: HTTP Toolkit; Tim Perry

Published: 2024-12-31T15:00:00Z

Content type: release

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Open Source](<https://devfeed.tech/topics/open-source.md>), [open source maintenance](<https://devfeed.tech/topics/open-source-maintenance.md>), [Maintainers](<https://devfeed.tech/topics/maintainers.md>)

Tags: [announcements](<https://devfeed.tech/tags/announcements.md>), [business](<https://devfeed.tech/tags/business.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [funding](<https://devfeed.tech/tags/funding.md>), [maintainers](<https://devfeed.tech/tags/maintainers.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [open-source-code](<https://devfeed.tech/tags/open-source-code.md>), [open-source-software](<https://devfeed.tech/tags/open-source-software.md>), [security](<https://devfeed.tech/tags/security.md>), [sustainability](<https://devfeed.tech/tags/sustainability.md>)

### AI overview

HTTP Toolkit is joining the Open Source Pledge, formally committing to fund open-source maintainers indefinitely at a minimum of $2,000 per full-time developer per year. The pledge also requires businesses to publicly report their payments and recipients. The article argues that relying on volunteers to maintain critical dependencies is unsustainable, and that funding maintainers is the underlying solution; for businesses, the pledge offers public recognition and aims to support healthier dependencies. The excerpt does not include the full explanation of the pledge's benefits or mechanism.

### Source excerpt

The Open Source Pledge is a new push to make companies commit to funding the maintainers of the open-source software they depend on, and to publicly recognize the ones that do. HTTP Toolkit has donated back to maintainers for a few years now, but joining the Open Source Pledge today means formally committing to that, and to doing so publicly with a sustainable minimum level ($2000 per full-time developer, or higher) indefinitely into the future. What is the Open Source Pledge? HTTP Toolkit (and effectively 100% of other software businesses) depends on a huge quantity of open-source code for much of its fundamental functionality & infrastructure. Most of this is tirelessly maintained by volunteers, completely for free. Astonishingly, that basically works, and we've built an entire software industry on top of it. But it's not a fair deal, and over the years it has become increasingly clear that businesses replying on people maintaining their critical dependencies for free is not a good or sustainable approach (for either the businesses or the maintainers' mental health). Plenty of important open-source projects have been abandoned as maintainers moved on with their lives, while some have been actively removed entirely or even replaced with malicious content. For businesses these are big problems, but for maintainers it's not reasonable to expect that they'll continue actively supporting their projects for free indefinitely just because your business chose to use it. There have been many attempts to fix this, from the organizational (internally forking key dependencies, avoiding dependencies entirely where possible) to the purely technical (dependency locking & mirroring, security scanning of new releases). At the end of the day though, the only real solution to open source sustainability is to fund the maintainers you depend on. The Open Source Pledge aims to commit businesses to doing this, and to build a wider culture where that becomes the norm. The actual mechanis

## ERR\_PROXY\_CONNECTION\_FAILED errors with HTTP proxies

DevFeed: [ERR\_PROXY\_CONNECTION\_FAILED errors with HTTP proxies](<https://devfeed.tech/articles/err-proxy-connection-failed-errors-with-http-proxies-19092.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/proxy_connection_failed/>)

Author: HTTP Toolkit; Tim Perry

Published: 2024-12-11T17:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [proxy](<https://devfeed.tech/topics/proxy.md>), [debugging](<https://devfeed.tech/topics/debugging.md>), [client](<https://devfeed.tech/topics/client.md>), [cURL](<https://devfeed.tech/topics/curl.md>)

Tags: [antivirus](<https://devfeed.tech/tags/antivirus.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [browser](<https://devfeed.tech/tags/browser.md>), [chrome](<https://devfeed.tech/tags/chrome.md>), [client](<https://devfeed.tech/tags/client.md>), [connect](<https://devfeed.tech/tags/connect.md>), [curl](<https://devfeed.tech/tags/curl.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [errors](<https://devfeed.tech/tags/errors.md>), [eset](<https://devfeed.tech/tags/eset.md>), [http](<https://devfeed.tech/tags/http.md>), [http-client](<https://devfeed.tech/tags/http-client.md>), [local](<https://devfeed.tech/tags/local.md>), [port](<https://devfeed.tech/tags/port.md>), [postman](<https://devfeed.tech/tags/postman.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [security](<https://devfeed.tech/tags/security.md>), [security-feature](<https://devfeed.tech/tags/security-feature.md>), [tcp](<https://devfeed.tech/tags/tcp.md>), [toolkit](<https://devfeed.tech/tags/toolkit.md>)

### AI overview

For ERR_PROXY_CONNECTION_FAILED with an HTTP proxy, first check the proxy address, port, and authentication, then try connecting directly with curl, Postman, or HTTP Toolkit Send. A TCP error suggests the proxy is unreachable or its details are wrong; an HTTP response indicates the server is reachable. For clients launched and configured automatically by HTTP Toolkit, the excerpt says basic settings are likely not the cause. It identifies antivirus connection interception, notably ESET, as a likely cause in that case: the antivirus sits between the browser and local proxy, then may reject the proxy's custom TLS certificate. The excerpt ends before giving a remedy.

### Source excerpt

If you're using a local debugging proxy tool like HTTP Toolkit, you might run into the dreaded ERR_PROXY_CONNECTION_FAILED error in Chrome and other similar apps. This can be a very frustrating and unhelpful error! There's only a few possible causes though, and it's usually easy to fix. The Simple Case The simplest explanation is exactly what it says: the browser can't connect to your proxy. In the simple case this may be caused by a basic connection issue: you have the address, port or some authentication details wrong, or the details are correct but the proxy is just not reachable on your connection, and so the browser can't talk to it. The easiest way to confirm this is to try connecting to the details directly using an HTTP client tool like curl, Postman, or the HTTP Toolkit Send page, to manually check if the server is listening on the port you expect. If you see a TCP error then the details are wrong, but if you see any kind of HTTP error or similar then you know that the server does exist and it's reachable with the given details. If the failure comes from a browser intercepted by a tool like HTTP Toolkit though, where intercepted clients are preconfigured & launched for you, this is not what's happening - the server is definitely reachable (it's running locally) and the config is correct (it's been done automatically). Instead, this might be caused by something more complicated... Beyond the Simple Case If you know the settings are correct, and you know the proxy is reachable, what could cause this ERR_PROXY_CONNECTION_FAILED error? Once you're ruled out basic connection issues, the most likely cause is antivirus software on your computer that is intercepting your connections. This is becoming a common feature of some antivirus & security software, most notably ESET. It's not directly related to viruses at all, but is a separate security feature that's often enabled automatically for additional protection. What happens in this case is that your browser is tryi

## Designing API Errors

DevFeed: [Designing API Errors](<https://devfeed.tech/articles/designing-api-errors-19054.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/designing-api-errors/>)

Author: HTTP Toolkit; Phil Sturgeon

Published: 2024-09-09T15:30:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [API](<https://devfeed.tech/topics/api.md>), [REST API](<https://devfeed.tech/topics/rest-api.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [client](<https://devfeed.tech/tags/client.md>), [code](<https://devfeed.tech/tags/code.md>), [ddos](<https://devfeed.tech/tags/ddos.md>), [designing](<https://devfeed.tech/tags/designing.md>), [error](<https://devfeed.tech/tags/error.md>), [errors](<https://devfeed.tech/tags/errors.md>), [http](<https://devfeed.tech/tags/http.md>), [retry](<https://devfeed.tech/tags/retry.md>), [standards](<https://devfeed.tech/tags/standards.md>), [status](<https://devfeed.tech/tags/status.md>), [string](<https://devfeed.tech/tags/string.md>), [token](<https://devfeed.tech/tags/token.md>)

### AI overview

HTTP status codes give API clients broad clues about failures and possible responses: for example, the excerpt says not to retry a 429 until the rate limit ends, while a 504 may merit an immediate retry. But a status code such as 400 can cover distinct problems. The carpooling examples use human-readable messages to explain issues like locations being too close or an impractical route; those messages can help users understand what to do, but the excerpt notes they do not let an application distinguish the errors programmatically. The text is truncated while beginning to discuss the search for more specific status codes, so it does not establish the proposed solution.

### Source excerpt

When everything goes smoothly with an API, life is pretty straightforward: you request a resource, and voilà, you get it. You trigger a procedure, and the API politely informs you it's all gone to plan. But what happens when something goes pear-shaped? Well, that's where things can get a bit tricky. HTTP Status Codes HTTP status codes are like a first aid kit: they're handy, but they won't fix everything. They give you a broad idea of what's gone wrong, which can help plenty of tools and developers make reasonable assumptions, like: 400 Bad Request: Report error to developers, something is broken. 401 Unauthorized: Might need to refresh a token, don't try again until you have. 404 Not Found: If accepting user input to lookup a resource then this isn't a problem, so don't worry about it. Just tell the user the thing they're looking for isn't there. 405 Method Not Allowed: Ahhhh panic, the API has changed or the client was built wrong. 429 Too Many Requests: Do not retry this request until after the rate limit is over or you'll DDoS the server and get banned. 501 Not Implemented: Oh heck you've gone live relying on an endpoint which isn't ready in production, alert everyone. 504 Gateway Timeout: Probably retry that one straight away as it's likely a network blip. HTTP status codes can convey a lot of assumptions, but they cannot possibly cover all situations, so it's important to add something for the human developers to see what's wrong. Written Description of the Problem Let's say you're building a carpooling app and you need to plan a trip between two places to find more riders. If the coordinates you provide are too close together, the API might respond with something like: HTTP/1.1 400 Bad Request { "error": "Too close for a carpool to be organized, suggest get out and walk." } This is a 400 Bad Request, but that's a pretty common error and a little more information needs to be conveyed, so a string has been added explaining the problem. Next a user tries to plan

## 22 years later, YAML now has a media type

DevFeed: [22 years later, YAML now has a media type](<https://devfeed.tech/articles/22-years-later-yaml-now-has-a-media-type-19105.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/yaml-media-type-rfc/>)

Author: HTTP Toolkit; Tim Perry

Published: 2024-02-20T15:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [YAML](<https://devfeed.tech/topics/yaml.md>), [OpenAPI Specification](<https://devfeed.tech/topics/openapi.md>), [Ruby on Rails](<https://devfeed.tech/topics/ruby-on-rails.md>)

Tags: [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [api](<https://devfeed.tech/tags/api.md>), [applications](<https://devfeed.tech/tags/applications.md>), [based](<https://devfeed.tech/tags/based.md>), [content](<https://devfeed.tech/tags/content.md>), [content-type](<https://devfeed.tech/tags/content-type.md>), [data](<https://devfeed.tech/tags/data.md>), [define](<https://devfeed.tech/tags/define.md>), [details](<https://devfeed.tech/tags/details.md>), [email](<https://devfeed.tech/tags/email.md>), [examples](<https://devfeed.tech/tags/examples.md>), [existing](<https://devfeed.tech/tags/existing.md>), [file](<https://devfeed.tech/tags/file.md>), [formal](<https://devfeed.tech/tags/formal.md>), [formalized](<https://devfeed.tech/tags/formalized.md>), [full](<https://devfeed.tech/tags/full.md>), [http](<https://devfeed.tech/tags/http.md>), [media](<https://devfeed.tech/tags/media.md>), [openapi](<https://devfeed.tech/tags/openapi.md>), [standards](<https://devfeed.tech/tags/standards.md>), [type](<https://devfeed.tech/tags/type.md>), [yaml](<https://devfeed.tech/tags/yaml.md>)

### AI overview

RFC 9512, registered on February 14, 2024, establishes application/yaml as the media type for YAML and +yaml as a structured suffix for more specific YAML-based media types. Listing the type with IANA gives applications a shared label for YAML in HTTP headers, file metadata, and processing. The suffix supports types such as application/openapi+yaml, which the source says is being formalized separately. Existing conventions vary: Rails is cited as using application/x-yaml, while text/yaml and text/x-yaml are also seen. The RFC offers a basis for more consistent naming, but older applications will take time to update; it also addresses interoperability, YAML's relationship with JSON, and security considerations.

### Source excerpt

As of February 14th 2024, RFC 9512 formally registers application/yaml as the media type for all YAML content, and adds +yaml as a standard structured suffix for all YAML-based more specific media types. With this registration, it's now included in the official media types list maintained by the IANA. Media types like this (also known as the MIME types, from their original invention for email attachment metadata) are heavily used particularly in HTTP Content-Type headers for both requests & responses, and in all sorts of file metadata and processing logic elsewhere. These names give applications a common vocabulary to describe data when passing it around. The additional +yaml suffix also defined here is particularly useful. Media type structured suffixes like this (+xml and +json are other common examples) are used to define specific types for content that's based on an existing generic content type (such as YAML). In this case, this notably opens the door to standardization of other YAML-based MIME types, such as application/openapi+yaml (for OpenAPI specifications that are written in YAML) which is currently being formalized in another standard, following closely behind this one. While a few applications have been using application/yaml and +yaml like this already, many haven't (e.g. Rails still uses application/x-yaml, and others like text/yaml and even text/x-yaml are frequently seen in the wild) and there's never been clear agreement on exactly how this should work. Hopefully with this RFC we'll be able to start picking a single media type consistently from now on, though updating older applications here will obviously take some time. The full RFC is well worth a read if you're interested in the finer details, and discusses all sorts of more detailed questions around interoperability for an evolving language like YAML, its relationship with JSON, and the (many) security considerations to be aware of when defining a formal API around YAML data. This RFC is just

## What is X-Forwarded-For and when can you trust it?

DevFeed: [What is X-Forwarded-For and when can you trust it?](<https://devfeed.tech/articles/what-is-x-forwarded-for-and-when-can-you-trust-it-19102.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/what-is-x-forwarded-for/>)

Author: HTTP Toolkit; Phil Sturgeon

Published: 2024-01-31T17:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [proxy](<https://devfeed.tech/topics/proxy.md>), [Server](<https://devfeed.tech/topics/server.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [apis](<https://devfeed.tech/tags/apis.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cdn](<https://devfeed.tech/tags/cdn.md>), [client](<https://devfeed.tech/tags/client.md>), [data-localization](<https://devfeed.tech/tags/data-localization.md>), [header](<https://devfeed.tech/tags/header.md>), [history](<https://devfeed.tech/tags/history.md>), [http](<https://devfeed.tech/tags/http.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [ipv4](<https://devfeed.tech/tags/ipv4.md>), [ipv6](<https://devfeed.tech/tags/ipv6.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [need](<https://devfeed.tech/tags/need.md>), [original](<https://devfeed.tech/tags/original.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [receiver](<https://devfeed.tech/tags/receiver.md>), [request](<https://devfeed.tech/tags/request.md>), [server](<https://devfeed.tech/tags/server.md>), [servers](<https://devfeed.tech/tags/servers.md>), [source](<https://devfeed.tech/tags/source.md>), [standards](<https://devfeed.tech/tags/standards.md>)

### AI overview

X-Forwarded-For (XFF) records client IP addresses as requests pass through proxies, CDNs, or load balancers, helping a backend see forwarding history that its direct connection address alone cannot show. The last proxy's address is available from the incoming connection, so the excerpt says it need not add itself to the header. It lists possible uses including login checks, load balancing, data localization, and geographic content delivery. The excerpt does not explain when XFF can be trusted.

### Source excerpt

The X-Forwarded-For (XFF) HTTP header provides crucial insight into the origin of web requests. The header works as a mechanism for conveying the original source IP addresses of clients, and not just across one hop, but through chains of multiple intermediaries. This list of IPv4 and IPv6 addresses is helpful to understand where requests have really come from in scenarios where they traverse several servers, proxies, or load balancers. A typical HTTP request goes on a bit of a journey, traversing multiple layers of infrastructure before reaching its destination. Without the X-Forwarded-For header, the receiving server would only see the IP address of the last intermediary in the chain (the direct source of the request) rather than the true client origin. {/* sequenceDiagram actor Client as Client 28.178.124.142 participant CDN as CDN 198.40.10.101 participant LB as Load Balancer 198.40.10.102 participant B as Backend App 10.0.0.1 Client->>CDN: CDN->>LB: LB->>B: */} In thie example, by the time the backend application is seeing an incoming request, the IP address of the original client is long forgotten. This is where the X-Forwarded-For header can help out. It looks like this: X-Forwarded-For: 28.178.124.142, 198.40.10.101 The goal here is to give a proxy the chance to say "Alright hang on, I'm forwarding you a request, and this is the history of where it came from, as far as I know". Note that the last proxy will not add its own IP address to the list, because that's already available: if the receiver of the request cares about who is calling it directly, they can combine the X-Forwarded-For with the request's source IP address from the incoming connection, e.g: req.connection.remoteAddress in NodeJS. In this example above, the load balancer has said "Hey backend app, I am forwarding you a request that came from this client, via the CDN", and it doesn't need to pop its own IP in there because the backend app can already tell if it's coming from the load balancer or

## Working with the new Idempotency Keys RFC

DevFeed: [Working with the new Idempotency Keys RFC](<https://devfeed.tech/articles/working-with-the-new-idempotency-keys-rfc-19075.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/idempotency-keys/>)

Author: HTTP Toolkit; Phil Sturgeon

Published: 2023-12-12T15:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [client library](<https://devfeed.tech/topics/client-library.md>)

Tags: [apis](<https://devfeed.tech/tags/apis.md>), [http](<https://devfeed.tech/tags/http.md>), [idempotency](<https://devfeed.tech/tags/idempotency.md>), [idempotent-operations](<https://devfeed.tech/tags/idempotent-operations.md>), [ietf](<https://devfeed.tech/tags/ietf.md>), [implementing](<https://devfeed.tech/tags/implementing.md>), [keys](<https://devfeed.tech/tags/keys.md>), [retries](<https://devfeed.tech/tags/retries.md>), [rfc](<https://devfeed.tech/tags/rfc.md>), [server](<https://devfeed.tech/tags/server.md>), [standards](<https://devfeed.tech/tags/standards.md>)

### AI overview

The IETF draft RFC standardizes a common way to make retries of state-changing HTTP requests safer. GET, HEAD, PUT, DELETE, OPTIONS, and TRACE are defined as idempotent, while POST and PATCH can intentionally cause repeated effects, such as creating users or sending payments. For those requests, clients and servers must explicitly handle retries; idempotency keys provide a mechanism for doing so. The excerpt illustrates the risk of retrying after a timeout: the client may not know whether the server already processed the first request. The RFC is still a draft, so its details could change.

### Source excerpt

Idempotency is when doing an operation multiple times is guaranteed to have the same effect as doing it just once. When working with APIs this is exceptionally helpful on slow or unreliable internet connections, or when dealing with particularly sensitive actions such as payments, because it makes retrying operations safe and reliable. This is why most payment gateways like Stripe and Adyen support 'idempotency keys' as a key feature of their APIs. Recently, the IETF have gone further, and created a draft RFC standard for this useful common pattern, as part of the 'Building Blocks for HTTP APIs' working group. This is technically still a draft and the details could change, but it's fairly mature now and increasingly widely used, so it's a good time to take a closer look, and start using & implementing it for yourself. Idempotency in HTTP APIs Many HTTP methods are defined as idempotent in all cases. In theory, any GET, HEAD, PUT, DELETE, OPTIONS, or TRACE operation can be executed multiple times without any unintended side effects (though for badly behaved APIs your mileage may vary). The idea is that an HTTP request like DELETE /users/123 clearly wants to delete that user, and if that accidentally happens twice then that's just fine. User 123 ends up deleted just the same. It's a lot more complicated for POST and PATCH requests, which do not provide that same level of confidence out of the box. These are designed to allow non-idempotent operations, like adding a new user, sending a payment, or appending to existing data. Those are important use cases too - sometimes you really do want to send the same thing twice, and have it happen twice - but this can cause problems when things go wrong. When sending POST and PATCH requests to modify server state, if you want to support retries then both the client and server need to explicitly handle this - which is exactly what idempotency keys are designed to allow you to do. How can non-idempotency go wrong? Without idempoten

## A brief introduction to OpenAPI

DevFeed: [A brief introduction to OpenAPI](<https://devfeed.tech/articles/a-brief-introduction-to-openapi-19081.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/introduction-to-openapi/>)

Author: HTTP Toolkit; Phil Sturgeon

Published: 2023-11-28T09:30:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [OpenAPI Specification](<https://devfeed.tech/topics/openapi.md>), [Documentation](<https://devfeed.tech/topics/documentation.md>), [REST API](<https://devfeed.tech/topics/rest-api.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [mock api](<https://devfeed.tech/topics/mock-api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [documentation](<https://devfeed.tech/tags/documentation.md>), [format](<https://devfeed.tech/tags/format.md>), [generate](<https://devfeed.tech/tags/generate.md>), [http](<https://devfeed.tech/tags/http.md>), [introduction](<https://devfeed.tech/tags/introduction.md>), [json](<https://devfeed.tech/tags/json.md>), [json-schema](<https://devfeed.tech/tags/json-schema.md>), [metadata](<https://devfeed.tech/tags/metadata.md>), [openapi](<https://devfeed.tech/tags/openapi.md>), [reference](<https://devfeed.tech/tags/reference.md>), [request](<https://devfeed.tech/tags/request.md>), [specifications](<https://devfeed.tech/tags/specifications.md>), [standards](<https://devfeed.tech/tags/standards.md>), [swagger](<https://devfeed.tech/tags/swagger.md>), [tools](<https://devfeed.tech/tags/tools.md>), [type](<https://devfeed.tech/tags/type.md>), [type-system](<https://devfeed.tech/tags/type-system.md>), [type-systems](<https://devfeed.tech/tags/type-systems.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

OpenAPI is a machine-readable description format for HTTP APIs, specifying metadata, endpoints, request and response formats, authentication, and validation details. Based on JSON Schema, it gives REST-style APIs a type system. Teams commonly use OpenAPI specifications to generate reference documentation; the excerpt also describes validation, contract testing, client- and server-side validation, SDKs, server stubs, and mock servers as uses across the API lifecycle. Redoc can generate documentation sites from a spec, while HTTP Toolkit uses published specs to validate request parameters and show metadata with requests and responses. The supplied text is an excerpt and ends during the HTTP Toolkit example.

### Source excerpt

It's hard to work on APIs without hearing about OpenAPI. OpenAPI is an API description format, which is essentially metadata that describes an HTTP API: where it lives, how it works, what data is available, and how it's authenticated. Additional keywords can be used to provide all sorts of validation information, adding a type system to what would otherwise just be arbitrary JSON flying around the internet. OpenAPI has been around for donkeys years, previously known as Swagger but renamed to OpenAPI in 2016. It's powered by JSON Schema, which is also pretty popular in certain circles, but it's only in the last few years that OpenAPI has solidified its place as the description format for HTTP APIs, pushing aside others like RAML and API Blueprint. Elder developers will remember working with WSDLs and XML Schema, and gRPC and GraphQL folks might be thinking "hang on this sounds a bit familiar", and absolutely. Type systems for APIs are pretty common, but here's an excellent one you can use for your REST/RESTish API. Here's an example to give you an idea: openapi: 3.1.3 info: title: Your Awesome API version: '1.0.3' description: More information and introduction. paths: /things: post: summary: Create a thing requestBody: content: application/json: schema: type: object properties: name: type: string examples: - Tim responses: '201': description: "Created" content: application/json: schema: type: object properties: id: type: string format: uuid name: type: string examples: - Tim created_at: type: string format: date-time example: 2020-01-01T00:00:00Z This describes an API in a machine-readable format, including overall metadata, endpoint paths, request formats, and the details of possible responses you might receive. What can OpenAPI do? OpenAPI specifications provide a machine-readable base on top of which lots of neat API tools can be used and even generated. One of the most common uses by many API teams to to generate API reference documentation, which helps end-users

## 6 Ways to Debug a Failing Docker Container

DevFeed: [6 Ways to Debug a Failing Docker Container](<https://devfeed.tech/articles/6-ways-to-debug-an-exploding-docker-container-19050.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/debug-failing-docker-container/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-10-31T14:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Docker](<https://devfeed.tech/topics/docker.md>), [debugging](<https://devfeed.tech/topics/debugging.md>), [container](<https://devfeed.tech/topics/container.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>)

Tags: [command-line](<https://devfeed.tech/tags/command-line.md>), [container](<https://devfeed.tech/tags/container.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [docker](<https://devfeed.tech/tags/docker.md>), [errors](<https://devfeed.tech/tags/errors.md>), [logs](<https://devfeed.tech/tags/logs.md>), [metrics](<https://devfeed.tech/tags/metrics.md>)

### AI overview

A tutorial on diagnosing Docker containers that fail to start or behave unexpectedly. It covers commands for viewing logs and resource usage, copying files, opening a shell, and inspecting a stopped container by creating an image with a different command.

### Source excerpt

Everything crashes. Sometimes things crash when they're running inside a Docker container though, and then all of a sudden it can get much more difficult to work out why, or what the hell to do next. Docker's great, but it's an extra layer of complexity that means you can't always easily poke your app up close any more, and that can really hinder debugging when your container fails to start or breaks in unusual ways. If you're stuck in that situation, here are my goto debugging commands to help you get a bit more information on exactly what's up: docker logs <container_id> Hopefully you've already tried this, but if not: start here. This'll give you the full STDOUT and STDERR command-line output from the command that was run initially in your container. You can also use docker attach <container id> to stream the live logs from an active container, if you want to keep an eye on the output as it runs. docker stats <container_id> If you just need to keep an eye on the metrics of your container to work out what's gone wrong, docker stats can help: it'll give you a live stream of resource usage, so you can see just how much memory you've leaked so far and easily spot if your CPU usage is way out of control. docker cp <container_id>:/path/to/useful/file /local-path Often just getting hold of more log files is enough to sort you out. If you already know what you want, docker cp has your back: copy any file from any container back out onto your local machine, so you can examine it in depth (especially useful analysing heap dumps). docker exec -it <container_id> /bin/bash Next up, if you can run the container (if it's crashed, you can restart it with docker start <container_id>) then you can use this command to oppen a command line shell inside the container directly, and start digging around for further details by hand. docker commit <container_id> my-broken-container && docker run -it my-broken-container /bin/bash Can't start your container at all? If your container starts

## New ways to inject system CA certificates in Android 14

DevFeed: [New ways to inject system CA certificates in Android 14](<https://devfeed.tech/articles/new-ways-to-inject-system-ca-certificates-in-android-14-19038.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/android-14-install-system-ca-certificate/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-09-21T12:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Android](<https://devfeed.tech/topics/android.md>), [Cryptography](<https://devfeed.tech/topics/cryptography.md>)

Tags: [android](<https://devfeed.tech/tags/android.md>), [android-14](<https://devfeed.tech/tags/android-14.md>), [ca](<https://devfeed.tech/tags/ca.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [interception](<https://devfeed.tech/tags/interception.md>), [root](<https://devfeed.tech/tags/root.md>), [system](<https://devfeed.tech/tags/system.md>), [tls](<https://devfeed.tech/tags/tls.md>)

### AI overview

Android 14 blocks the old direct method of writing system CA certificates, even with root, but the excerpt says root-based injection remains practical through other mechanisms. For immediate traffic interception, it points to HTTP Toolkit's automatic setup: connect the device to ADB and choose "Android Device via ADB." The change concerns system-level CA injection on rooted devices or emulators; the excerpt says enterprise-provisioned CA installation and user-installed certificates are unaffected, and system CAs can still be soft-removed through Settings. It also says AOSP is affected, likely including alternative distributions unless they disable the change. The excerpt does not include the promised manual injection steps.

### Source excerpt

A couple of weeks ago I published a post about changes in Android 14 that fundamentally break existing approaches to installing system-level CA certificates, even with root access. This has triggered some fascinating discussion! I highly recommend a skim through the debate on Mastodon and Hacker News. Since that was posted, quite a few people have talked to me about possible solutions, going beyond the previous approaches with new mechanisms that make it practical to do this in Android 14+, and there are some good options here. While direct root access to change these certificates by simply writing to a directory is indeed no longer possible, root is root, and so with a bit of work there are still some practical & effective ways to dig down into the internals of Android and seize control of these certs once more. Choose your own adventure: If you just want to intercept an Android 14+ device right now, stop reading this, download the latest HTTP Toolkit, connect your device to ADB, click the 'Android Device via ADB' interception option for automatic setup, and dive into your traffic. If you just want to know the steps to manually do system certificate injection on Android 14 for yourself, jump down to How to install system CA certificates in Android 14. If you want the full background, so you can understand how & why this all works, read on: Clearing up confusion Before digging into this, I do want to explicitly clear up a few misunderstandings that I've seen repeatedly pop up from the previous article: These changes don't affect installation of CAs in other scenarios. As far as I'm aware, CA installation for fully managed enterprise-provisioned devices and the limited user-installed (as opposed to system-level) CA certificates will continue functioning as before. If you're not using root access to inject system-level CA certificates into a rooted device or emulator, you don't need to worry about this. Similarly, it is still possible to soft-remove system CA certific

## Android 14 blocks modification of system certificates, even as root

DevFeed: [Android 14 blocks modification of system certificates, even as root](<https://devfeed.tech/articles/android-14-blocks-modification-of-system-certificates-even-as-root-19037.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/android-14-breaks-system-certificate-installation/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-09-05T14:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Android](<https://devfeed.tech/topics/android.md>), [enterprise certificate authority certificates](<https://devfeed.tech/topics/enterprise-certificate-authority-certificates.md>)

Tags: [2016](<https://devfeed.tech/tags/2016.md>), [android](<https://devfeed.tech/tags/android.md>), [android-14](<https://devfeed.tech/tags/android-14.md>), [api](<https://devfeed.tech/tags/api.md>), [ca](<https://devfeed.tech/tags/ca.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [certificate-authorities](<https://devfeed.tech/tags/certificate-authorities.md>), [certificate-authority](<https://devfeed.tech/tags/certificate-authority.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [developer](<https://devfeed.tech/tags/developer.md>), [google](<https://devfeed.tech/tags/google.md>), [interception](<https://devfeed.tech/tags/interception.md>), [nougat](<https://devfeed.tech/tags/nougat.md>), [root](<https://devfeed.tech/tags/root.md>), [software](<https://devfeed.tech/tags/software.md>), [tester](<https://devfeed.tech/tags/tester.md>), [tls](<https://devfeed.tech/tags/tls.md>), [user-control](<https://devfeed.tech/tags/user-control.md>)

### AI overview

Android 14 tightens control over trusted certificate authorities, and the post initially reports that changing the system trust set appears impossible even on rooted devices--creating challenges for developers, testers, and reverse engineers who need to control device trust. The excerpt traces this shift to Android 7, which split certificates into a vendor-provided set used by default and user-added certificates available only to apps that opt in. It notes that trusted CAs affect who can guarantee the security of encrypted network traffic. An update says multiple working certificate-injection solutions for Android 14 later emerged; this excerpt does not describe how they work.

### Source excerpt

Update: This post sparked a lot of excellent discussion and debate on workarounds, and there are now multiple working solutions to allow certificate injection on Android 14, despite the restrictions discussed here. See the update post for more details. When Android was initially announced in 2007 by the Open Handset Alliance (headed by Google) their flagship project was billed as an "open platform", "providing developers a new level of openness", and giving them "complete access to handset capabilities and tools". We've come a long way since then, steadily retreating from openness & user control of devices, and shifting towards a far more locked-down vendor-controlled world. The next step of Android's evolution is Android 14 (API v34, codename Upside-Down Cake) and it takes more steps down that path. In this new release, the restrictions around certificate authority (CA) certificates become significantly tighter, and appear to make it impossible to modify the set of trusted certificates at all, even on fully rooted devices. If you're an Android developer, tester, reverse engineer, or anybody else interested in directly controlling who your device trusts, this is going to create some new challenges. Before we get into the finer details, first I want to talk a little about the context around Android CA management and how we got here, but if you want to jump to the latest details you can go straight to the Enter Android 14 section below. "Open Software, Open Device, Open Ecosystem" While the initial principles of Android were very much focused on open software, controllable by users and developers, over more recent years Android has increasingly limited the control of users, developers & researchers over their own devices. The key turning point in this process was Android 7 (Nougat, released in 2016) in which the certificate authorities (CAs) on the device that were previously fully modifiable by the owner of the phone were split in two: one fixed list of CAs provided

## Apple already shipped attestation on the web, and we barely noticed

DevFeed: [Apple already shipped attestation on the web, and we barely noticed](<https://devfeed.tech/articles/apple-already-shipped-attestation-on-the-web-and-we-barely-noticed-19041.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/apple-private-access-tokens-attestation/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-07-25T14:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [App attestation](<https://devfeed.tech/topics/app-attestation.md>), [frontend security](<https://devfeed.tech/topics/frontend-security.md>), [Web](<https://devfeed.tech/topics/web.md>), [Chromium](<https://devfeed.tech/topics/chromium.md>), [iOS](<https://devfeed.tech/topics/ios.md>)

Tags: [access-tokens](<https://devfeed.tech/tags/access-tokens.md>), [apple](<https://devfeed.tech/tags/apple.md>), [attestation](<https://devfeed.tech/tags/attestation.md>), [browser-apis](<https://devfeed.tech/tags/browser-apis.md>), [browsers](<https://devfeed.tech/tags/browsers.md>), [cloudflare](<https://devfeed.tech/tags/cloudflare.md>), [fastly](<https://devfeed.tech/tags/fastly.md>), [http](<https://devfeed.tech/tags/http.md>), [ios](<https://devfeed.tech/tags/ios.md>), [macos](<https://devfeed.tech/tags/macos.md>), [safari](<https://devfeed.tech/tags/safari.md>), [the-web](<https://devfeed.tech/tags/the-web.md>)

### AI overview

Apple's Private Access Tokens use device attestation to help websites recognize verified clients without disclosing identity, with captcha reduction described as the main use. The excerpt says the system is integrated into macOS 13, iOS 16, Safari, Cloudflare, and Fastly. In the described HTTP flow, a server responds with a 401 challenge; the browser sends challenge data and OS-verified device details to an attester such as Apple, which works with a trusted token issuer to return a signed token. The browser retries with that token, allowing the server to verify the client. The excerpt notes that the attester determines what counts as legitimate, and does not include the remainder of the flow or article.

### Source excerpt

There's been a lot of concern recently about the Web Environment Integrity proposal, developed by a selection of authors from Google, and apparently being prototyped in Chromium. There's good reason for anger here (though I'm not sure yelling at people on GitHub is necessarily the best outlet). This proposal amounts to attestation on the web, limiting access to features or entire sites based on whether the client is approved by a trusted issuer. In practice, that will mean Apple, Microsoft & Google. Of course, Google isn't the first to think of this, but in fact they're not even the first to ship it. Apple already developed & deployed an extremely similar system last year, now integrated into MacOS 13, iOS 16 & Safari, called "Private Access Tokens": Private Access Tokens are powerful tools that prove when HTTP requests are coming from legitimate devices without disclosing someone's identity. The focus here is primarily on removing captchas, and as such it's been integrated into Cloudflare (discussed here) and Fastly (here) as a mechanism for recognizing 'real' clients without needing other captcha mechanisms. Fundamentally though, it's exactly the same concept: a way that web servers can demand your device prove it is a sufficiently 'legitimate' device before browsing the web. How do Private Access Tokens work? The mechanism is a fairly simple exchange over HTTP, handled by built-in browser APIs, which in turn integrate with operating system components to confirm that the browser & OS are 'legitimate' (the exact definition of that is left to the attester - i.e. Apple). The flow looks like this: A browser makes an HTTP request from a web server. The web server refuses the request, and returns an HTTP 401 response with a PrivateToken challenge: HTTP/1.1 401 Unauthorized WWW-Authenticate: PrivateToken challenge=<base64 challenge data>, token-key=<base64 public-key> (Newlines added for readability) The browser recognizes this, and sends parts of the challenge, in addit

## Leaking secrets through caching with Bunny CDN

DevFeed: [Leaking secrets through caching with Bunny CDN](<https://devfeed.tech/articles/leaking-secrets-through-caching-with-bunny-cdn-19044.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/bunny-cdn-caching-vulnerability/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-06-20T11:30:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [cache-control](<https://devfeed.tech/topics/cache-control.md>), [frontend security](<https://devfeed.tech/topics/frontend-security.md>), [Secrets Management](<https://devfeed.tech/topics/secrets-management.md>), [Web app](<https://devfeed.tech/topics/webapp.md>)

Tags: [auth](<https://devfeed.tech/tags/auth.md>), [caching](<https://devfeed.tech/tags/caching.md>), [cdn](<https://devfeed.tech/tags/cdn.md>), [http](<https://devfeed.tech/tags/http.md>), [private](<https://devfeed.tech/tags/private.md>), [secrets](<https://devfeed.tech/tags/secrets.md>), [security](<https://devfeed.tech/tags/security.md>)

### AI overview

The author reports that Bunny.net's CDN once served private HTTP responses intended for one authenticated user to other users while they were testing deployment options; the issue is now fixed. The excerpt frames the incident against a broader tension: CDNs may cache more aggressively than HTTP standards anticipate to protect upstream services from traffic spikes and support performance, but this can create privacy and application risks. It notes that API authentication commonly uses an Authorization header, but the excerpt ends before explaining how that interacted with the cache or how the issue was resolved.

### Source excerpt

Caching is hard. Unfortunately though, caching is quite important. Hosted caching & CDNs offer incredible powers that can provide amazing performance boosts, cost savings & downtime protection, essential for most modern sites with any serious volume of users. Unfortunately, while there are strict standards for how caching is supposed to work with HTTP on the web, many cache providers do not quite follow these, instead giving their customers free reign over all kinds of invalid caching behaviour, and providing their own default configurations that often don't closely follow these standards to start with either. There are many good reasons for this, but the main one is that CDNs are now doing dual service: providing performance improvements, and actively protecting upstream sites from DoS attacks and traffic spikes (similar problems - the key difference between a DoS attack and hitting #1 on Hacker News etc is intent, not impact). This conflicts with many of the standards, which prioritize correctness and predictability over this use case and, for example, expect clients to be able to unilaterally request that the cache be ignored. Bunny.net provides one of these CDNs, and like most they aggressively cache content beyond the limits of the standards, both to help protect upstream servers and to support advanced user use cases. This has upsides, but in some edge cases can result in awkward bugs and break developer expectations. In some more dramatic cases though, it can expose private user data, break applications & even leak auth credentials, and that's where this story gets serious. A few months ago, I ran into exactly that issue while testing out deployment options with Bunny.net, where I discovered that private HTTP responses intended for one authenticated user could be served to other users instead. Spoiler: this is now fixed! That said, it's worth exploring where this went wrong, the many ways this can work right, and how CDNs solve issues like this in practice. C

## Automatic npm publishing, with GitHub Actions & npm granular tokens

DevFeed: [Automatic npm publishing, with GitHub Actions & npm granular tokens](<https://devfeed.tech/articles/automatic-npm-publishing-with-github-actions-npm-granular-tokens-19042.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/automatic-npm-publish-gha/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-03-22T10:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [npm](<https://devfeed.tech/topics/npm.md>), [GitHub Actions](<https://devfeed.tech/topics/github-actions.md>), [GitHub](<https://devfeed.tech/topics/github.md>), [Security](<https://devfeed.tech/topics/security.md>), [Continuous integration](<https://devfeed.tech/topics/continuous-integration.md>)

Tags: [ci](<https://devfeed.tech/tags/ci.md>), [github](<https://devfeed.tech/tags/github.md>), [github-actions](<https://devfeed.tech/tags/github-actions.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [node-js](<https://devfeed.tech/tags/node-js.md>), [npm](<https://devfeed.tech/tags/npm.md>), [releases](<https://devfeed.tech/tags/releases.md>), [secrets](<https://devfeed.tech/tags/secrets.md>), [security](<https://devfeed.tech/tags/security.md>)

### AI overview

This tutorial explains how to automate npm package publishing with GitHub Actions using npm granular access tokens. It describes restricting tokens to specific packages, storing them as GitHub Actions secrets, allowing publishing without 2FA for granular or automation tokens, and configuring a publishing workflow.

### Source excerpt

This week, at long last, GitHub announced granular access tokens for npm. This is a big deal! It's great for security generally, but also particularly useful if you maintain any npm packages, as it removes the main downside of automating npm publishing, by allowing you to give CI jobs only a very limited token instead of full 2FA-free access to your account. In the past, I've wished for this, because I maintain a fair few npm packages including some very widely used ones. The previous solution of "just disable 2FA on your account, create an all-powerful access token with global access to every package, and give that token to your CI job" was not a comfortable one. Regardless of your situation, isolating any risk of issues in security-sensitive situations like this is a good move, and ensures that any leak of (or legitimate access to) your CI secrets for one project doesn't imply a complete takeover of everything on your npm account. As soon as I saw this was now available, I jumped on automating npm publishing for a few of the packages that I've been manually publishing until now. The process is pretty quick and easy, let's walk through the steps: Get an access token for your package Log into npmjs.com Click your profile picture in the top right, then 'Access Tokens', 'Generate New Token', and 'Granular Access token' (or jump to npmjs.com/settings/$YOUR_USERNAME/tokens/granular-access-tokens/new) Set a useful name, a long expiry (up to you), 'Read and write' permissions, and pick the specific package that you're publishing Add your token as a secret for your project's GitHub Actions Jump to https://github.com/$YOU/$REPO/settings/secrets/actions/new Set NPM_PUBLISH_TOKEN as the secret name Copy the npm_... token from the previous step as the secret value In your npm package's settings (i.e. https://www.npmjs.com/package/$PACKAGE_NAME/access), allow publish without 2FA for granular/automation tokens only, so that tokens can be used for publishing: Add a publish step t

## Dodge the next Dockerpocalypse: how to own your own Docker Registry address

DevFeed: [Dodge the next Dockerpocalypse: how to own your own Docker Registry address](<https://devfeed.tech/articles/dodge-the-next-dockerpocalypse-how-to-own-your-own-docker-registry-address-19057.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/docker-image-registry-facade/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-03-17T11:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Docker Hub](<https://devfeed.tech/topics/docker-hub.md>), [container images](<https://devfeed.tech/topics/container-images.md>), [Self-hosted](<https://devfeed.tech/topics/self-hosted.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [docker](<https://devfeed.tech/tags/docker.md>), [docker-hub](<https://devfeed.tech/tags/docker-hub.md>), [docker-registry](<https://devfeed.tech/tags/docker-registry.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [policy](<https://devfeed.tech/tags/policy.md>), [publishing](<https://devfeed.tech/tags/publishing.md>), [registry](<https://devfeed.tech/tags/registry.md>), [self-hosting](<https://devfeed.tech/tags/self-hosting.md>)

### AI overview

The excerpt says Docker Hub's revised policy appears to preserve existing images but prevents unpaid projects from publishing new ones, leaving them without control of their publishing namespace. It argues that community and hobbyist publishers should consider leaving: self-hosting a registry brings ongoing monitoring, security update, load, and disk-space responsibilities, while moving directly to another hosted registry risks facing another policy change and having to update image references again. The excerpt ends before presenting the proposed alternative, so it does not establish how to own a registry address.

### Source excerpt

As you may have seen, Docker Hub made a dramatic shift in policy this week, and effectively gave a 30 day eviction notice to almost all community-run images. They've now made an apology to 'clarify' a few details, and helpfully take some of the hard edges off, but this still highlights a big problem. Fortunately, there are solutions. As initially described, this would've been catastrophic: Docker Hub has been used as the default host in tooling, tutorials, demos, blog posts, scripts, deployment definitions, CI builds and more for many years, and all those references were going to break - a self-inflicted left-pad for the Docker ecosystem. In their updated policy, it appears they now won't remove any existing images, but projects who don't pay up will not be able to publish any new images, so they've effectively lost control of the namespace they used to deploy to their communities unless they purchase a full team subscription. Many will not do so. This is an interesting challenge. Even if the existing images aren't removed, the direction of travel for Docker Hub is now clear: they don't want to host the core of the Docker community any more, no more freebies, pay up or go elsewhere (not unreasonable, but something of a rugpull after a full decade of the opposite). As a small/open-source/community/hobbyist image publisher, or if you depend on Docker Hub for free image publishing in any capacity, you now have a problem. They don't want you there. You're explicitly not their audience, and the rules will likely tighten further. This isn't unreasonable - it's their service and hosting isn't free - but it's worth considering explicitly and reacting accordingly. If you're not a paying Docker Hub customer, it's time to leave Docker Hub. The hard part is what to do instead. Self-hosting a registry is not free, and it's more work than it sounds: it's a proper piece of infrastructure, and comes with all the obligations that implies, from monitoring to promptly applying securit

## EU Funding for Mobile App Traffic Interception

DevFeed: [EU Funding for Mobile App Traffic Interception](<https://devfeed.tech/articles/eu-funding-for-mobile-app-traffic-interception-19061.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/frida-mobile-interception-funding/>)

Author: HTTP Toolkit; Tim Perry

Published: 2023-02-27T12:00:00Z

Content type: release

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [interception](<https://devfeed.tech/topics/interception.md>), [Android](<https://devfeed.tech/topics/android.md>), [iOS](<https://devfeed.tech/topics/ios.md>)

Tags: [accessible](<https://devfeed.tech/tags/accessible.md>), [android](<https://devfeed.tech/tags/android.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [apis](<https://devfeed.tech/tags/apis.md>), [app](<https://devfeed.tech/tags/app.md>), [apps](<https://devfeed.tech/tags/apps.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [data](<https://devfeed.tech/tags/data.md>), [developers](<https://devfeed.tech/tags/developers.md>), [device](<https://devfeed.tech/tags/device.md>), [eu](<https://devfeed.tech/tags/eu.md>), [eu-funding](<https://devfeed.tech/tags/eu-funding.md>), [frida](<https://devfeed.tech/tags/frida.md>), [funding](<https://devfeed.tech/tags/funding.md>), [interception](<https://devfeed.tech/tags/interception.md>), [ios](<https://devfeed.tech/tags/ios.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [mobile-app](<https://devfeed.tech/tags/mobile-app.md>), [traffic](<https://devfeed.tech/tags/traffic.md>), [traffic-interception](<https://devfeed.tech/tags/traffic-interception.md>)

### AI overview

NLNet Foundation funding through NGI Zero Entrust will support a year of HTTP Toolkit development to make HTTPS interception more precise and usable for individual apps on iOS and Android. The planned workflow aims to let users select an installed app and inspect its traffic without device-wide background noise, with automatic certificate unpinning and less manual setup, using new Frida integrations and hooks. Currently, Android interception is automatic but device-wide, lacks certificate unpinning, and injects system certificates; iOS has no automatic setup support. The existing Android device-wide option is intended to remain supported. The text presents this as open-source R&D funding tied to transparency and privacy goals.

### Source excerpt

HTTP Toolkit has been selected to receive another round of open-source funding from the EU! This aims to improve interception of HTTPS traffic from mobile apps, making it easier for both security/privacy researchers and normal technical users to inspect & manipulate the data that any app they use sends & receives. This funding will directly support work to improve the precision & usability of mobile app interception, on both iOS and Android. In a couple of clicks, you'll be able to target any app installed on a connected device (on either platform) and inspect all its traffic. That means no background noise from full system interception, automatic certificate unpinning, and no fiddly manual setup required. HTTP Toolkit can already automatically intercept Android devices, but only globally for the whole device (using Android's VPN APIs), with system certificates injected but without certificate unpinning, and with no automatic setup support for iOS at all. All those caveats are going away (of course, I'm intending the existing device-wide interception to remain a fully supported option indefinitely too). This is going be powered by a set of new integrations & hooks for Frida, a popular open-source instrumentation toolkit. Many advanced HTTP Toolkit users are already using Frida independently (see the Frida certificate unpinning guide for more info) but doing so often requires quite a bit of setup and specialist knowledge. This project is going to take that away, making network interception of mobile apps easily accessible to anybody who knows what "HTTP" is. All this is being funded by the fantastic NLNet Foundation as part of NGI Zero Entrust, a program funding open-source EU projects that support transparency around data use & privacy in modern technology. This is part of the EU's Next Generation Internet (NGI) initiative, aiming to directly fund researchers & open-source developers to encourage the future engineering of the Internet towards European values: "openn

## Debugging WebRTC, IPFS & Ethereum with HTTP Toolkit

DevFeed: [Debugging WebRTC, IPFS & Ethereum with HTTP Toolkit](<https://devfeed.tech/articles/debugging-webrtc-ipfs-ethereum-with-http-toolkit-19053.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/decentralized-web-webrtc-debugging/>)

Author: HTTP Toolkit; Tim Perry

Published: 2022-10-28T13:30:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [WebRTC](<https://devfeed.tech/topics/webrtc.md>), [debugging](<https://devfeed.tech/topics/debugging.md>), [Ethereum](<https://devfeed.tech/topics/ethereum.md>), [IPFS](<https://devfeed.tech/topics/ipfs.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [Web app](<https://devfeed.tech/topics/webapp.md>), [browser](<https://devfeed.tech/topics/browser.md>), [WebSocket](<https://devfeed.tech/topics/websocket.md>), [QUIC](<https://devfeed.tech/topics/quic.md>)

Tags: [debugging](<https://devfeed.tech/tags/debugging.md>), [decentralized-web](<https://devfeed.tech/tags/decentralized-web.md>), [ethereum](<https://devfeed.tech/tags/ethereum.md>), [http](<https://devfeed.tech/tags/http.md>), [interception](<https://devfeed.tech/tags/interception.md>), [ipfs](<https://devfeed.tech/tags/ipfs.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [toolkit](<https://devfeed.tech/tags/toolkit.md>), [webrtc](<https://devfeed.tech/tags/webrtc.md>)

### AI overview

HTTP Toolkit now displays intercepted browser activity over WebRTC, IPFS, and Ethereum inline with collected HTTP traffic. Users can create rules to rewrite requests, mock responses, or inject errors, extending the tool's debugging and testing coverage to these protocols in multi-protocol web applications. The excerpt frames this as a response to applications using more communication protocols alongside HTTP, but ends before explaining how the support works internally.

### Source excerpt

HTTP is important on the web, but as other alternative protocols grow popular in networked applications, it's often important to be able to capture, debug and mock those too. I've been working on expanding HTTP Toolkit's support for this over the past year (as one part of a project funded by EU Horizon's Next Generation Internet initiative), to extend HTTP Toolkit to cover three additional rising protocols that are often used alongside simple HTTP in decentralized web applications: WebRTC, IPFS & Ethereum. This is now live! If you're using HTTP Toolkit to intercept browsers, and a web application connects to another peer over WebRTC, interacts with the Ethereum blockchain, or pulls content from the IPFS network, then you'll now see this inline amongst your collected HTTP traffic, and you can create rules to rewrite these requests, define mock responses, or inject errors. Let's talk about why this matters, what you can do with this, and how it actually works internally: Why? HTTP remains the key protocol on the web, by a wide margin, and that's not going to change any time soon. That said, there are an increasing number of new protocols being explored both alongside and on top of HTTP, covering alternate use cases and supporting different communication models, and as software evolves this will only increase. HTTP Toolkit needs to be able to support these, to ensure that you can easily understand and test all communications from the increasing number of multi-protocol applications. There's a lot of examples of HTTP-adjacent protocols where this applies: WebSockets, which grew to cover use cases needing persistent connections that weren't well supported by the request/response model, which use HTTP just for initial connection setup, and are now widely supported and used in HTTP-based apps. WebTransport, a new QUIC-based connection protocol that's aiming to supplant WebSockets with a modern approach that adds a swathe of other benefits on top. GraphQL, a general-purpose

## Testing libraries for the Decentralized Web

DevFeed: [Testing libraries for the Decentralized Web](<https://devfeed.tech/articles/testing-libraries-for-the-decentralized-web-19052.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/decentralized-web-testing-libraries/>)

Author: HTTP Toolkit; Tim Perry

Published: 2022-10-24T17:00:00Z

Content type: tutorial

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [IPFS](<https://devfeed.tech/topics/ipfs.md>), [Ethereum](<https://devfeed.tech/topics/ethereum.md>), [Integration testing](<https://devfeed.tech/topics/integration-testing.md>), [Continuous integration](<https://devfeed.tech/topics/continuous-integration.md>), [WebRTC](<https://devfeed.tech/topics/webrtc.md>), [JavaScript](<https://devfeed.tech/topics/javascript.md>), [P2P](<https://devfeed.tech/topics/p2p.md>), [Web app](<https://devfeed.tech/topics/webapp.md>)

Tags: [ci](<https://devfeed.tech/tags/ci.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [decentralized-web](<https://devfeed.tech/tags/decentralized-web.md>), [ethereum](<https://devfeed.tech/tags/ethereum.md>), [hosting](<https://devfeed.tech/tags/hosting.md>), [integration-testing](<https://devfeed.tech/tags/integration-testing.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [testing](<https://devfeed.tech/tags/testing.md>), [web-applications](<https://devfeed.tech/tags/web-applications.md>), [webrtc](<https://devfeed.tech/tags/webrtc.md>)

### AI overview

This article introduces MockIPFS and Mockthereum, network interception libraries for IPFS and Ethereum. It explains how they support automated integration testing and CI workflows while also serving as foundations for network proxy tools for decentralized web applications.

### Source excerpt

The world of decentralized web applications is an exciting place that has exploded in recent years, with technologies such as IPFS and Ethereum opening up possibilities for a peer-to-peer web - creating applications that live outside the traditional client/server model, where users to interact and control their own data directly. At the same time, it's still immature, and for software developers it lacks a lot of the affordances & ecosystem of the traditional HTTP-based web app world. There's far fewer tools and libraries for developers working in this space. I've been working on improving this over the last year (as one part of a project funded by EU Horizon's Next Generation Internet initiative), by building network interception libraries for both IPFS & Ethereum: MockIPFS & Mockthereum. These each act as both an immediately useful automated testing library, to support modern integration testing & CI workflows, and a base for building more general network proxy tools for web applications using either (or both) technologies. If that sounds cool and you just want to jump straight in and try these for yourself, you can get started at github.com/httptoolkit/mockipfs/ and github.com/httptoolkit/mockthereum/. On the other hand, if you want to hear what this can do in practice, and learn a little about how it works under the hood, read on: A new way to build web apps Decentralized web apps often use a mix of many different technologies, at various layers of the stack, such as: IPFS - for decentralized static content hosting & data storage Ethereum - for decentralized consistent global state, computation on that state, and financial transactions Filecoin/Storj - for paid decentralized long-term content storage WebRTC - for peer-to-peer raw data transfer, and video/audio connections Service workers - a JavaScript API allowing fully offline web apps Handshake (HNS)/Ethereum Name System (ENS) - to map domain names to web applications GunDB - a decentralized database for the

## How to intercept, observe & mock WebRTC traffic

DevFeed: [How to intercept, observe & mock WebRTC traffic](<https://devfeed.tech/articles/how-to-intercept-observe-mock-webrtc-traffic-19079.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/intercepting-webrtc-traffic/>)

Author: HTTP Toolkit; Tim Perry

Published: 2022-10-13T10:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [WebRTC](<https://devfeed.tech/topics/webrtc.md>), [Developer Tools](<https://devfeed.tech/topics/developer-tools.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [all](<https://devfeed.tech/tags/all.md>), [allows](<https://devfeed.tech/tags/allows.md>), [also](<https://devfeed.tech/tags/also.md>), [apis](<https://devfeed.tech/tags/apis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [apps](<https://devfeed.tech/tags/apps.md>), [audio](<https://devfeed.tech/tags/audio.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [automated](<https://devfeed.tech/tags/automated.md>), [automated-testing](<https://devfeed.tech/tags/automated-testing.md>), [avoid](<https://devfeed.tech/tags/avoid.md>), [browser](<https://devfeed.tech/tags/browser.md>), [browser-tools](<https://devfeed.tech/tags/browser-tools.md>), [build](<https://devfeed.tech/tags/build.md>), [building](<https://devfeed.tech/tags/building.md>), [built-in](<https://devfeed.tech/tags/built-in.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [interception](<https://devfeed.tech/tags/interception.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [mock](<https://devfeed.tech/tags/mock.md>), [traffic](<https://devfeed.tech/tags/traffic.md>), [webrtc](<https://devfeed.tech/tags/webrtc.md>)

### AI overview

WebRTC's peer-to-peer negotiated encryption, lack of PKI-based proxying, varied network negotiation, and limited debugging APIs make traffic inspection and mocking difficult. The excerpt says interception is possible when you control one peer: with one line of code, a connection can be intercepted to observe raw unencrypted traffic or inject and transform WebRTC data and media. The author says they are building MockRTC as a framework for this work; the excerpt does not describe its further features or implementation.

### Source excerpt

WebRTC allows two users on the web to communicate directly, sending real-time streams of video, audio & data peer-to-peer, from within a browser environment. It's exciting tech that's rapidly maturing, already forming the backbone of a huge range of video chat, screen sharing and live collaboration tools, but also as a key technology for decentralization of web apps - providing a P2P data transport layer used by everything from WebTorrent to IPFS to Yjs. Unfortunately though, it doesn't have the tooling ecosystem that developers used to networking with HTTP often expect. There's few supporting tools or libraries, inspecting raw traffic is hard or impossible, and mocking WebRTC traffic for automated testing is even harder. Even built-in low-level browser tools like chrome://webrtc-internals don't allow seeing messages sent on WebRTC data channels. It's hard to build modern secure web applications on top of protocols that you can't directly see or interact with. This doesn't just affect developers: it also seriously impacts security & privacy researchers and reverse engineers, each trying to investigate the traffic sent & received by the apps we all use. If you want to know what data a webapp you use is sending over WebRTC, right now it's very hard to find out. Intercepting WebRTC traffic to build these tools and libraries is difficult, because unlike protocols like HTTP that were designed to allow active proxying and user-configureable PKI (i.e. CA certificates) early on, WebRTC encrypts all traffic using peer-to-peer negotiated certificates for authentication without PKI, communicates in a wide variety of different negotiated ways at the network level to avoid NAT issues, and offers no convenient APIs to configure this for debugging. All of this provides some great features to the protocol as a user, but some serious challenges when building developer tools. As it turns out though, despite this, there are just enough places where we can hook into that it is possible

## Android Chrome 99 expands Certificate Transparency, breaking all MitM dev tools

DevFeed: [Android Chrome 99 expands Certificate Transparency, breaking all MitM dev tools](<https://devfeed.tech/articles/android-chrome-99-expands-certificate-transparency-breaking-all-mitm-dev-tools-19047.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/chrome-android-certificate-transparency/>)

Author: HTTP Toolkit; Tim Perry

Published: 2022-05-11T16:00:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [Certificate Transparency](<https://devfeed.tech/topics/certificate-transparency.md>), [Android](<https://devfeed.tech/topics/android.md>), [Google Chrome](<https://devfeed.tech/topics/google-chrome.md>), [Developer Tools](<https://devfeed.tech/topics/developer-tools.md>), [certificates](<https://devfeed.tech/topics/certificates.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [android](<https://devfeed.tech/tags/android.md>), [browsers](<https://devfeed.tech/tags/browsers.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [certificate-transparency](<https://devfeed.tech/tags/certificate-transparency.md>), [chrome](<https://devfeed.tech/tags/chrome.md>), [mitm](<https://devfeed.tech/tags/mitm.md>), [tls](<https://devfeed.tech/tags/tls.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

Chrome 99 expands Certificate Transparency enforcement to all Android Chrome users, according to the excerpt. This can prevent Chrome traffic interception by debugging tools such as HTTP Toolkit when they rely on a user-installed, trusted system CA, including on rooted devices; Chrome may report ERR_CERTIFICATE_TRANSPARENCY_REQUIRED. Certificate Transparency is intended to make misissued HTTPS certificates visible to domain owners, addressing the risk that a fraudulent certificate could enable undetected traffic interception. The excerpt introduces this security mechanism but does not provide the promised debugging workarounds.

### Source excerpt

Certificate transparency is superb improvement to HTTPS certificate security on the web that's great for users and businesses, but on Android it creates a huge problem for the many developer tools like HTTP Toolkit which install trusted system certificates into Android to intercept & debug app traffic. This doesn't appear in the main announcements anywhere, but buried deep in the enterprise release notes for Chrome v99 there's a small note that says: Certificate transparency is already enforced on desktop platforms, and for some Android users. Chrome 99 expands certificate transparency to all Android Chrome users. And with that small note, Chrome on Android become uninterceptable for all HTTP Toolkit users using rooted devices, and anybody else who actively installs and trusts their own system CA certificates. If you're running into an ERR_CERTIFICATE_TRANSPARENCY_REQUIRED error in Chrome while trying to debug your HTTPS traffic with some MitM debugging proxy, then this is affecting you too. Let's talk about how certificate transparency works, why this breaks, and how you can work around it to keep debugging HTTPS from Chrome on your Android device regardless. Certificate Transparency (CT) HTTPS certificates are issued and signed by Certificate Authorities (CAs) who are trusted by your browser & OS. That's great when it works, but sometimes it doesn't. CAs can make mistakes when issuing certificates, when verifying a client's identity beforehand, or through malice somewhere, and issue fraudulent certificates to people who shouldn't have them. For example, let's say a trusted CA issues a certificate for google.com to the wrong person (this actually happened, repeatedly). That issued certificate is incredibly powerful - whoever has it can freely intercept all traffic sent by anybody to Google.com and both see & modify that traffic, whilst browsers will show all users a padlock and tell them everything is totally fine & secure. Even worse though: attacks like this were

## Fighting TLS fingerprinting with Node.js

DevFeed: [Fighting TLS fingerprinting with Node.js](<https://devfeed.tech/articles/fighting-tls-fingerprinting-with-node-js-19098.md>)

Original publisher: [Read original article](<https://httptoolkit.com/blog/tls-fingerprinting-node-js/>)

Author: HTTP Toolkit; Tim Perry

Published: 2021-12-07T13:40:00Z

Content type: article

Language: en

Sources: [HTTP Toolkit](<https://devfeed.tech/sources/http-toolkit.md>)

Topics: [TLS (Transport Layer Security)](<https://devfeed.tech/topics/tls.md>), [Node.js](<https://devfeed.tech/topics/node-js.md>)

Tags: [browser](<https://devfeed.tech/tags/browser.md>), [fingerprint](<https://devfeed.tech/tags/fingerprint.md>), [http](<https://devfeed.tech/tags/http.md>), [https](<https://devfeed.tech/tags/https.md>), [interception](<https://devfeed.tech/tags/interception.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [mitm](<https://devfeed.tech/tags/mitm.md>), [network](<https://devfeed.tech/tags/network.md>), [node](<https://devfeed.tech/tags/node.md>), [node-js](<https://devfeed.tech/tags/node-js.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [python](<https://devfeed.tech/tags/python.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [scraping](<https://devfeed.tech/tags/scraping.md>), [scripts](<https://devfeed.tech/tags/scripts.md>), [server](<https://devfeed.tech/tags/server.md>), [servers](<https://devfeed.tech/tags/servers.md>), [services](<https://devfeed.tech/tags/services.md>), [software](<https://devfeed.tech/tags/software.md>), [tls](<https://devfeed.tech/tags/tls.md>)

### AI overview

TLS fingerprinting can identify client software before it sends an HTTP request by analyzing the unencrypted TLS ClientHello, including parameters such as the cipher list. Servers can use this to recognize or block Node.js clients, automated scripts, or HTTP Toolkit's MitM proxy; the excerpt describes a real case where non-browser traffic was blocked based on its TLS fingerprint. The article says it will explain how to work around this with Node.js, but the supplied excerpt does not include the workaround steps.

### Source excerpt

The modern internet is full of services that want to know who you are. Fingerprinting is the latest way to do this: capturing many small details about your client, and using it to create an id that's sufficiently unique to recognize you and infer details about your network client and device. This is a privacy problem, which I'm not going to focus on here, but collecting and analysing interaction metadata is also a powerful tool to recognize certain types of clients, even when they attempt to disguise themselves. TLS provides a particularly good surface for this kind of fingerprinting, which allows a server or proxy to recognize the kind of software (a specific browser version, Python, Ruby, Node.js, etc) that's opening any TLS connection, before the client has even sent any data (such as an HTTP request) within the connection, and purely using unencrypted public data from the connection. In many cases, this is a problem. HTTP Toolkit acts as a MitM proxy for HTTP(S) traffic inspection & mocking, and this potentially allows servers to recognize and block it, along with any other similar debugging proxies. Many other automated scripts and tools can also be recognized, blocking web scraping and other requests from anything but a real browser. Until recently, I thought this was fairly theoretical, but last week an HTTP Toolkit user showed me a real-world example, where non-browser traffic is blocked completely, based just on its TLS fingerprint, causing big problems for HTTP Toolkit usage. Fortunately, we can work around this. In this article, I want to explain how TLS fingerprinting works, look at a real-world example, and then see exactly how you can defeat this blocking using Node.js (with techniques that you can easily apply elsewhere too). How does TLS fingerprinting work? TLS provides a huge amount of data for fingerprinting. Every connection secured by TLS (for example, all HTTPS requests) starts with a 'client hello' message from the client, sent unencrypted, wi

[Next page](<https://devfeed.tech/sources/http-toolkit.md?cursor=WyIyMDIxLTEyLTA3VDEzOjQwOjAwKzAwOjAwIiwgIjRjYjQzMGM3LTUwZjAtNDMzNi04ZDFhLTNmMmMxNWIwZjdlZCJd>)