# JSON for Engineers

Published articles for JSON for Engineers.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Scalable JSON Streaming with HTTP and Go - Ep.5

DevFeed: [Scalable JSON Streaming with HTTP and Go - Ep.5](<https://devfeed.tech/articles/scalable-json-streaming-with-http-and-go-ep-5-22271.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/11/scalable-json-streaming-with--http-and-go.html>)

Published: 2024-11-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [JSON](<https://devfeed.tech/topics/json.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [Error Handling](<https://devfeed.tech/topics/error-handling.md>), [datasets](<https://devfeed.tech/topics/datasets.md>)

Tags: [chunked-transfer-json](<https://devfeed.tech/tags/chunked-transfer-json.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [data-transmission](<https://devfeed.tech/tags/data-transmission.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [decoding-json-streams](<https://devfeed.tech/tags/decoding-json-streams.md>), [efficient-json-handling](<https://devfeed.tech/tags/efficient-json-handling.md>), [encoding-json](<https://devfeed.tech/tags/encoding-json.md>), [encoding-json-in-go](<https://devfeed.tech/tags/encoding-json-in-go.md>), [error-handling](<https://devfeed.tech/tags/error-handling.md>), [examples](<https://devfeed.tech/tags/examples.md>), [go](<https://devfeed.tech/tags/go.md>), [go-json-encoding](<https://devfeed.tech/tags/go-json-encoding.md>), [go-json-memory-management](<https://devfeed.tech/tags/go-json-memory-management.md>), [handling-large-json-files](<https://devfeed.tech/tags/handling-large-json-files.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [http](<https://devfeed.tech/tags/http.md>), [http-chunked-encoding](<https://devfeed.tech/tags/http-chunked-encoding.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [json](<https://devfeed.tech/tags/json.md>), [json-chunked-transfer](<https://devfeed.tech/tags/json-chunked-transfer.md>), [json-error-logging](<https://devfeed.tech/tags/json-error-logging.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-lines-format](<https://devfeed.tech/tags/json-lines-format.md>), [json-lines-ndjson](<https://devfeed.tech/tags/json-lines-ndjson.md>), [json-streaming](<https://devfeed.tech/tags/json-streaming.md>), [json-streaming-in-go](<https://devfeed.tech/tags/json-streaming-in-go.md>), [memory-efficient-data-transmission](<https://devfeed.tech/tags/memory-efficient-data-transmission.md>), [memory-efficient-json](<https://devfeed.tech/tags/memory-efficient-json.md>), [real-time-data-streaming](<https://devfeed.tech/tags/real-time-data-streaming.md>), [real-time-json-streaming](<https://devfeed.tech/tags/real-time-json-streaming.md>), [scalable-json-processing](<https://devfeed.tech/tags/scalable-json-processing.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-json-over-http](<https://devfeed.tech/tags/streaming-json-over-http.md>), [streaming-large-json-datasets](<https://devfeed.tech/tags/streaming-large-json-datasets.md>)

### AI overview

The final episode of the JSON for Engineers series explains how to stream large JSON datasets efficiently using JSON Lines, Go's encoding/json package, and HTTP/1.1 chunked transfer encoding. It covers incremental encoding and decoding, memory usage, response flushing, and logging errors after streaming begins.

### Source excerpt

Introduction: Welcome to the final episode of the JSON for Engineers series! In this concluding session, we tackle the challenges of working with large JSON datasets, exploring efficient strategies for streaming data while minimizing memory usage. These techniques enable developers to handle massive payloads without overburdening system resources, ensuring scalable and cost-effective applications. JSON Streaming: Using JSON Lines for memory-efficient data transmission. HTTP Chunked Encoding: Leveraging HTTP/1.1 chunked transfer encoding for streaming large datasets. Practical Error Handling: Logging and managing errors in streaming JSON responses. This episode starts by addressing the inefficiencies of constructing large JSON objects in memory when working with massive datasets, such as database query results. Instead of consuming significant memory to create one monolithic JSON object, the recommended approach involves using JSON Lines (NDJSON), a format where each line represents a separate JSON object. This method reduces memory requirements by transmitting data incrementally. Using Go's encoding/json package, developers can easily encode and stream multiple JSON objects, as it automatically appends newlines between objects. On the receiving end, decoding JSON streams requires careful looping to handle incoming data dynamically while avoiding memory reuse issues, which could lead to errors or stale data.

## JSON Handling and Type Management with Large Datasets in Go - Ep.4

DevFeed: [JSON Handling and Type Management with Large Datasets in Go - Ep.4](<https://devfeed.tech/articles/json-handling-and-type-management-with-large-datasets-in-go-ep-4-22262.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/10/json-handling-and-type-management-with-large-datasets-in-go.html>)

Published: 2024-11-14T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [JSON](<https://devfeed.tech/topics/json.md>), [Code](<https://devfeed.tech/topics/code.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [advanced-json-management-go](<https://devfeed.tech/tags/advanced-json-management-go.md>), [code](<https://devfeed.tech/tags/code.md>), [custom-go-struct-json-serialization](<https://devfeed.tech/tags/custom-go-struct-json-serialization.md>), [custom-json-serialization-go](<https://devfeed.tech/tags/custom-json-serialization-go.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [efficient-json-processing-in-go](<https://devfeed.tech/tags/efficient-json-processing-in-go.md>), [encoding-json](<https://devfeed.tech/tags/encoding-json.md>), [examples](<https://devfeed.tech/tags/examples.md>), [go](<https://devfeed.tech/tags/go.md>), [go-encoding-json-package](<https://devfeed.tech/tags/go-encoding-json-package.md>), [go-json-handling-tutorial](<https://devfeed.tech/tags/go-json-handling-tutorial.md>), [go-selective-decoding](<https://devfeed.tech/tags/go-selective-decoding.md>), [go-type-management-json](<https://devfeed.tech/tags/go-type-management-json.md>), [handling-mixed-type-arrays-json-go](<https://devfeed.tech/tags/handling-mixed-type-arrays-json-go.md>), [interfaces](<https://devfeed.tech/tags/interfaces.md>), [introduction](<https://devfeed.tech/tags/introduction.md>), [json](<https://devfeed.tech/tags/json.md>), [json-anonymous-structures-go](<https://devfeed.tech/tags/json-anonymous-structures-go.md>), [json-best-practices](<https://devfeed.tech/tags/json-best-practices.md>), [json-data-optimization-techniques](<https://devfeed.tech/tags/json-data-optimization-techniques.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-handling-techniques-for-engineers](<https://devfeed.tech/tags/json-handling-techniques-for-engineers.md>), [json-marshaler-and-unmarshaler-interfaces-in-go](<https://devfeed.tech/tags/json-marshaler-and-unmarshaler-interfaces-in-go.md>), [json-nested-fields-go](<https://devfeed.tech/tags/json-nested-fields-go.md>), [json-strategies-large-datasets](<https://devfeed.tech/tags/json-strategies-large-datasets.md>), [json-type-pollution-avoidance](<https://devfeed.tech/tags/json-type-pollution-avoidance.md>), [managing-large-json-datasets-in-go](<https://devfeed.tech/tags/managing-large-json-datasets-in-go.md>), [pitfalls](<https://devfeed.tech/tags/pitfalls.md>), [reducing-type-declarations-go](<https://devfeed.tech/tags/reducing-type-declarations-go.md>), [serialization](<https://devfeed.tech/tags/serialization.md>)

### AI overview

Episode 4 of the JSON for Engineers series explains how to process large JSON datasets in Go. It covers selective decoding with anonymous structures, custom JSON serialization through Marshaler and Unmarshaler interfaces, and handling mixed-type arrays.

### Source excerpt

Introduction: Welcome to Episode 4 of the JSON for Engineers series! In this episode, we tackle the complexities of working with JSON data, especially when dealing with extensive datasets and optimizing type management. Here, Miki introduces key strategies for managing large JSON documents, avoiding unnecessary type definitions, and using Go's encoding/json package to handle custom serialization needs effectively. Type Pollution Avoidance: Minimizing type declarations by using selective decoding and anonymous structures. Custom JSON Serialization: Implementing Go's Marshaler and Unmarshaler interfaces for precise JSON formatting. Working with Mixed-Type Arrays: Handling JSON arrays containing diverse types in Go Miki begins by addressing the challenge of "type pollution" when working with large JSON responses. When dealing with massive JSON datasets, like data from social trading platforms, defining a separate type for every possible field can quickly lead to an unwieldy codebase. To streamline this, Miki demonstrates how to leverage anonymous structures and selective decoding in Go. By defining only the relevant fields within a function, we can avoid creating additional struct types, making the code more compact and efficient. This technique is especially useful when working with JSON data containing nested fields that may not all be necessary for your application. Through this approach, Go's encoding/json package will simply ignore any unreferenced fields, allowing you to handle only the data that matters to your project.

## Managing JSON Field Tags and Value Handling in Go - Ep.3

DevFeed: [Managing JSON Field Tags and Value Handling in Go - Ep.3](<https://devfeed.tech/articles/managing-json-field-tags-and-value-handling-in-go-ep-3-22265.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/10/json-the-fine-print-ep-3-managing-json-field-tags-and-value-handling-in-go.html>)

Published: 2024-10-31T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [JSON](<https://devfeed.tech/topics/json.md>), [Parsing](<https://devfeed.tech/topics/parsing.md>)

Tags: [default-values-in-go-structs](<https://devfeed.tech/tags/default-values-in-go-structs.md>), [go](<https://devfeed.tech/tags/go.md>), [go-default-values](<https://devfeed.tech/tags/go-default-values.md>), [go-field-tagging-techniques](<https://devfeed.tech/tags/go-field-tagging-techniques.md>), [go-struct-mapping](<https://devfeed.tech/tags/go-struct-mapping.md>), [handling-flexible-json](<https://devfeed.tech/tags/handling-flexible-json.md>), [handling-missing-json-values](<https://devfeed.tech/tags/handling-missing-json-values.md>), [json](<https://devfeed.tech/tags/json.md>), [json-code-reliability](<https://devfeed.tech/tags/json-code-reliability.md>), [json-data-consistency](<https://devfeed.tech/tags/json-data-consistency.md>), [json-data-structures-go](<https://devfeed.tech/tags/json-data-structures-go.md>), [json-field-consistency-go](<https://devfeed.tech/tags/json-field-consistency-go.md>), [json-field-integration-in-go](<https://devfeed.tech/tags/json-field-integration-in-go.md>), [json-field-mapping](<https://devfeed.tech/tags/json-field-mapping.md>), [json-field-tagging-in-go](<https://devfeed.tech/tags/json-field-tagging-in-go.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-in-go](<https://devfeed.tech/tags/json-in-go.md>), [json-in-strongly-typed-languages](<https://devfeed.tech/tags/json-in-strongly-typed-languages.md>), [json-keys-to-go-fields](<https://devfeed.tech/tags/json-keys-to-go-fields.md>), [json-parsing-errors](<https://devfeed.tech/tags/json-parsing-errors.md>), [json-parsing-strategies-go](<https://devfeed.tech/tags/json-parsing-strategies-go.md>), [json-pointers-in-go](<https://devfeed.tech/tags/json-pointers-in-go.md>), [json-schema-handling](<https://devfeed.tech/tags/json-schema-handling.md>), [json-structs-in-go](<https://devfeed.tech/tags/json-structs-in-go.md>), [json-to-go-data-integration](<https://devfeed.tech/tags/json-to-go-data-integration.md>), [json-to-go-parsing](<https://devfeed.tech/tags/json-to-go-parsing.md>), [json-unmarshalling-in-go](<https://devfeed.tech/tags/json-unmarshalling-in-go.md>), [json-value-handling](<https://devfeed.tech/tags/json-value-handling.md>), [managing-missing-values-in-json](<https://devfeed.tech/tags/managing-missing-values-in-json.md>), [managing-zero-values-in-json](<https://devfeed.tech/tags/managing-zero-values-in-json.md>), [map-string-interface-json](<https://devfeed.tech/tags/map-string-interface-json.md>), [mapping-json-to-go-structs](<https://devfeed.tech/tags/mapping-json-to-go-structs.md>), [unmarshalling](<https://devfeed.tech/tags/unmarshalling.md>)

### AI overview

This episode explains how to map JSON fields to Go struct fields with tags, distinguish missing values from zero values during unmarshalling, and use pointers, maps, or default struct values to handle flexible JSON data.

### Source excerpt

Introduction: Welcome to Episode 3 of JSON for Engineers! In this episode, Miki Tebeka explores the intricacies of JSON field mapping and value handling, especially when working in strongly typed languages like Go. This session covers the practical challenges of matching JSON fields to Go struct fields, distinguishing between missing and zero values, and using default values to ensure robust JSON parsing. These strategies are crucial for engineers managing flexible JSON data structures in a language with strict type requirements.

## Exploring JSON Encoding and Data Handling in Go - Ep.2

DevFeed: [Exploring JSON Encoding and Data Handling in Go - Ep.2](<https://devfeed.tech/articles/exploring-json-encoding-and-data-handling-in-go-ep-2-22264.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/10/json-the-fine-print-ep-2-exploring-json-encoding-and-data-handling-in-go.html>)

Published: 2024-10-17T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [JSON](<https://devfeed.tech/topics/json.md>), [Go Language](<https://devfeed.tech/topics/go-language.md>), [Structured-data](<https://devfeed.tech/topics/structured-data.md>), [web applications](<https://devfeed.tech/topics/web-applications.md>)

Tags: [debug](<https://devfeed.tech/tags/debug.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [efficient-data-encoding-in-go](<https://devfeed.tech/tags/efficient-data-encoding-in-go.md>), [encoding-json-in-go-with-type-assertions](<https://devfeed.tech/tags/encoding-json-in-go-with-type-assertions.md>), [format](<https://devfeed.tech/tags/format.md>), [go](<https://devfeed.tech/tags/go.md>), [go-encoding-json-package-tutorial](<https://devfeed.tech/tags/go-encoding-json-package-tutorial.md>), [go-interface-with-json](<https://devfeed.tech/tags/go-interface-with-json.md>), [go-json-handling](<https://devfeed.tech/tags/go-json-handling.md>), [go-programming-json-tips](<https://devfeed.tech/tags/go-programming-json-tips.md>), [go-strongly-typed-system-and-json](<https://devfeed.tech/tags/go-strongly-typed-system-and-json.md>), [handling-json-null-values-in-go](<https://devfeed.tech/tags/handling-json-null-values-in-go.md>), [json](<https://devfeed.tech/tags/json.md>), [json-decoding-in-go-examples](<https://devfeed.tech/tags/json-decoding-in-go-examples.md>), [json-encoding-in-go](<https://devfeed.tech/tags/json-encoding-in-go.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-marshalling-and-unmarshalling-in-go](<https://devfeed.tech/tags/json-marshalling-and-unmarshalling-in-go.md>), [json-schema-less-format-benefits-and-challenges](<https://devfeed.tech/tags/json-schema-less-format-benefits-and-challenges.md>), [json-vs-binary-formats-in-go](<https://devfeed.tech/tags/json-vs-binary-formats-in-go.md>), [mapping-json-to-go-types](<https://devfeed.tech/tags/mapping-json-to-go-types.md>), [pitfalls](<https://devfeed.tech/tags/pitfalls.md>), [working-with-json-in-go](<https://devfeed.tech/tags/working-with-json-in-go.md>)

### AI overview

Episode 2 explains JSON as a schema-less, human-readable format and examines its benefits and pitfalls. It focuses on mapping JSON data to Go's strongly typed system, handling nullable and numeric values, comparing JSON with binary formats, and using Go's encoding and decoding tools.

### Source excerpt

Introduction: Welcome to Episode 2 of JSON for Engineers! In this episode, we explore the unique characteristics of JSON as a schema-less format, discussing both its benefits and challenges. You'll learn how JSON's flexibility, while enabling rapid development, can create issues when working with strongly-typed languages like Go. Additionally, we'll cover the differences between JSON and more compact binary formats, and how to handle JSON's broader use in web applications.

## Data Integrity with JSON Serialization - Ep.1

DevFeed: [Data Integrity with JSON Serialization - Ep.1](<https://devfeed.tech/articles/data-integrity-with-json-serialization-ep-1-22263.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/10/json-the-fine-print-ep-1-data-integrity-with-json-serialization.html>)

Published: 2024-10-03T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [JSON](<https://devfeed.tech/topics/json.md>), [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>)

Tags: [common-serialization-mistakes](<https://devfeed.tech/tags/common-serialization-mistakes.md>), [custom-json-validation](<https://devfeed.tech/tags/custom-json-validation.md>), [data-integrity-in-software-engineering](<https://devfeed.tech/tags/data-integrity-in-software-engineering.md>), [data-serialization-in-api-design](<https://devfeed.tech/tags/data-serialization-in-api-design.md>), [data-validation](<https://devfeed.tech/tags/data-validation.md>), [efficient-data-serialization](<https://devfeed.tech/tags/efficient-data-serialization.md>), [how-to-validate-json-data](<https://devfeed.tech/tags/how-to-validate-json-data.md>), [introduction-to-json-for-engineers](<https://devfeed.tech/tags/introduction-to-json-for-engineers.md>), [json](<https://devfeed.tech/tags/json.md>), [json-best-practices](<https://devfeed.tech/tags/json-best-practices.md>), [json-best-practices-for-api](<https://devfeed.tech/tags/json-best-practices-for-api.md>), [json-data-validation-techniques](<https://devfeed.tech/tags/json-data-validation-techniques.md>), [json-for-engineers](<https://devfeed.tech/tags/json-for-engineers.md>), [json-in-software-development](<https://devfeed.tech/tags/json-in-software-development.md>), [json-optimization-for-storage](<https://devfeed.tech/tags/json-optimization-for-storage.md>), [json-serialization-tutorial](<https://devfeed.tech/tags/json-serialization-tutorial.md>), [miki-tebeka-json-guide](<https://devfeed.tech/tags/miki-tebeka-json-guide.md>), [serialization](<https://devfeed.tech/tags/serialization.md>), [serialization-in-software-architecture](<https://devfeed.tech/tags/serialization-in-software-architecture.md>), [software-architecture](<https://devfeed.tech/tags/software-architecture.md>), [software-layer-separation-and-data-handling](<https://devfeed.tech/tags/software-layer-separation-and-data-handling.md>)

### AI overview

Episode 1 of JSON for Engineers explains serialization fundamentals and how JSON converts data structures to byte sequences. It recommends serializing at system boundaries such as APIs and storage, keeping native types internally, separating models across layers, and validating data beyond JSON syntax.

### Source excerpt

Introduction: Welcome to Episode 1 of JSON for Engineers! In this first episode, Miki Tebeka dives into the fundamentals of serialization, with a special focus on JSON, one of the most widely-used data formats in software engineering. Miki draws from his extensive development experience to explain how JSON fits into the bigger picture of data interchange and serialization, laying the groundwork for engineers to use it effectively in real-world applications.