# json error logging

Published articles for json error logging.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Scalable JSON Streaming with HTTP and Go - Ep.5

DevFeed: [Scalable JSON Streaming with HTTP and Go - Ep.5](<https://devfeed.tech/articles/scalable-json-streaming-with-http-and-go-ep-5-22271.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/11/scalable-json-streaming-with--http-and-go.html>)

Published: 2024-11-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [JSON](<https://devfeed.tech/topics/json.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [Error Handling](<https://devfeed.tech/topics/error-handling.md>), [datasets](<https://devfeed.tech/topics/datasets.md>)

Tags: [chunked-transfer-json](<https://devfeed.tech/tags/chunked-transfer-json.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [data-transmission](<https://devfeed.tech/tags/data-transmission.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [decoding-json-streams](<https://devfeed.tech/tags/decoding-json-streams.md>), [efficient-json-handling](<https://devfeed.tech/tags/efficient-json-handling.md>), [encoding-json](<https://devfeed.tech/tags/encoding-json.md>), [encoding-json-in-go](<https://devfeed.tech/tags/encoding-json-in-go.md>), [error-handling](<https://devfeed.tech/tags/error-handling.md>), [examples](<https://devfeed.tech/tags/examples.md>), [go](<https://devfeed.tech/tags/go.md>), [go-json-encoding](<https://devfeed.tech/tags/go-json-encoding.md>), [go-json-memory-management](<https://devfeed.tech/tags/go-json-memory-management.md>), [handling-large-json-files](<https://devfeed.tech/tags/handling-large-json-files.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [http](<https://devfeed.tech/tags/http.md>), [http-chunked-encoding](<https://devfeed.tech/tags/http-chunked-encoding.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [json](<https://devfeed.tech/tags/json.md>), [json-chunked-transfer](<https://devfeed.tech/tags/json-chunked-transfer.md>), [json-error-logging](<https://devfeed.tech/tags/json-error-logging.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-lines-format](<https://devfeed.tech/tags/json-lines-format.md>), [json-lines-ndjson](<https://devfeed.tech/tags/json-lines-ndjson.md>), [json-streaming](<https://devfeed.tech/tags/json-streaming.md>), [json-streaming-in-go](<https://devfeed.tech/tags/json-streaming-in-go.md>), [memory-efficient-data-transmission](<https://devfeed.tech/tags/memory-efficient-data-transmission.md>), [memory-efficient-json](<https://devfeed.tech/tags/memory-efficient-json.md>), [real-time-data-streaming](<https://devfeed.tech/tags/real-time-data-streaming.md>), [real-time-json-streaming](<https://devfeed.tech/tags/real-time-json-streaming.md>), [scalable-json-processing](<https://devfeed.tech/tags/scalable-json-processing.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-json-over-http](<https://devfeed.tech/tags/streaming-json-over-http.md>), [streaming-large-json-datasets](<https://devfeed.tech/tags/streaming-large-json-datasets.md>)

### AI overview

The final episode of the JSON for Engineers series explains how to stream large JSON datasets efficiently using JSON Lines, Go's encoding/json package, and HTTP/1.1 chunked transfer encoding. It covers incremental encoding and decoding, memory usage, response flushing, and logging errors after streaming begins.

### Source excerpt

Introduction: Welcome to the final episode of the JSON for Engineers series! In this concluding session, we tackle the challenges of working with large JSON datasets, exploring efficient strategies for streaming data while minimizing memory usage. These techniques enable developers to handle massive payloads without overburdening system resources, ensuring scalable and cost-effective applications. JSON Streaming: Using JSON Lines for memory-efficient data transmission. HTTP Chunked Encoding: Leveraging HTTP/1.1 chunked transfer encoding for streaming large datasets. Practical Error Handling: Logging and managing errors in streaming JSON responses. This episode starts by addressing the inefficiencies of constructing large JSON objects in memory when working with massive datasets, such as database query results. Instead of consuming significant memory to create one monolithic JSON object, the recommended approach involves using JSON Lines (NDJSON), a format where each line represents a separate JSON object. This method reduces memory requirements by transmitting data incrementally. Using Go's encoding/json package, developers can easily encode and stream multiple JSON objects, as it automatically appends newlines between objects. On the receiving end, decoding JSON streams requires careful looping to handle incoming data dynamically while avoiding memory reuse issues, which could lead to errors or stale data.