# Reading JSON at 277 GB/s in Python

DevFeed: [Reading JSON at 277 GB/s in Python](<https://devfeed.tech/articles/reading-json-at-277-gb-s-in-python-134077.md>)

Original publisher: [Read original article](<https://lemire.me/blog/2026/11/14/reading-json-at-277-gb-s-in-python/>)

Author: Daniel Lemire

Published: 2026-11-14T14:53:50Z

Content type: article

Language: en

Sources: [Daniel Lemire](<https://devfeed.tech/sources/daniel-lemire.md>)

Topics: [json lines format](<https://devfeed.tech/topics/json-lines-format.md>), [Python](<https://devfeed.tech/topics/python.md>), [JSON](<https://devfeed.tech/topics/json.md>)

Tags: [json](<https://devfeed.tech/tags/json.md>), [python](<https://devfeed.tech/tags/python.md>), [reading](<https://devfeed.tech/tags/reading.md>)

## AI overview

In the reported benchmarks, fastsimdjson combines selective field extraction with threaded parsing to reach 138 GB/s when counting GitHub stars across 7.6 GB of NDJSON on a 192-core AWS instance; extracting fields from a Twitter benchmark file reaches 277 GB/s. On the counting task, it outpaces the best msgspec process pool (116 GB/s at 96 processes), with the advantage narrowing from 3.5x on one core to about 20% at scale. Across six benchmark files, four exceed 100 GB/s, while one is near 20 GB/s and another ties msgspec. The article notes counting is especially favorable to processes because little data must be returned.

## Source excerpt

Many systems log their data as JSON, one record per line (NDJSON). GitHub, for example, publishes every public event on its site that way through GH Archive: about a gigabyte per hour. Suppose you want to know which repositories received the most stars over eight hours. That is 7.6 GB of JSON, 2.2 million events. ... Continue reading Reading JSON at 277 GB/s in Python