# Serialization Published articles for Serialization. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Dragonfly 2.0 adds Redis and Valkey compatibility and reduces processing overhead DevFeed: [Dragonfly 2.0 adds Redis and Valkey compatibility and reduces processing overhead]() Original publisher: [Read original article]() Author: Michael Larabel Published: 2026-09-17T10:02:19Z Content type: release Language: en Sources: [Phoronix]() Topics: [in-memory database](), [Memcached](), [Redis](), [data]() Tags: [compatibility](), [desktop-linux](), [in-memory-database](), [linux-benchmarking](), [linux-hardware-benchmarks](), [linux-hardware-reviews](), [linux-how-to](), [linux-performance](), [linux-server-benchmarks](), [memcached](), [open-source-graphics](), [overhead](), [phoronix](), [phoronix-test-suite](), [redis](), [release](), [serialization](), [ubuntu-benchmarks](), [ubuntu-hardware](), [valkey](), [version]() ### AI overview Dragonfly 2.0 is a new release of the in-memory database that expands Redis and Valkey compatibility, adds Valkey 9 RDB loading support, and reduces connection processing and memory accounting overhead. ### Source excerpt Released on Wednesday was a new version of the Dragonfly database that continues to promote itself as "the world's most efficient in-memory data store" and a modern replacement to the likes of Redis and Memcached while being fully compatible with their APIs... ## v1 Transactions and the ALT Trade-off DevFeed: [v1 Transactions and the ALT Trade-off]() Original publisher: [Read original article]() Author: Umberto Natale Published: 2026-08-17T00:00:00Z Content type: article Language: en Sources: [Solana News Feed]() Topics: [Solana](), [Transactions](), [benchmarking](), [SDKs]() Tags: [blockchain](), [blockchain-technology](), [crypto-news](), [cryptocurrency](), [defi](), [nfts](), [podcasts](), [reference](), [reports](), [rpc](), [sdks](), [serialization](), [solana](), [solana-ecosystem](), [transactions](), [upgrades](), [web3]() ### AI overview Solana is introducing the v1 transaction format with a maximum serialized size of 4,096 bytes, replacing Account Lookup Table resolution with larger inline transaction capacity. The analysis finds that current workloads are broadly compatible with the new envelope, while the unchanged 64-account limit may remain a constraint for some applications. ### Source excerpt Solana is introducing a new transaction format (v1) alongside an increase of the maximum transaction size from 1232 bytes to 4096 bytes. The v1 format aims to unlock new possibilities on-chain. This is done trading-off Account Lookup Tables (ALT) usage. In this research we explore in details the current usage of ALTs, alongside implications of removing them. ## Zero-flicker Firestore SSR with React DevFeed: [Zero-flicker Firestore SSR with React]() Original publisher: [Read original article]() Author: Jeff Huleatt Published: 2026-06-24T00:00:00Z Content type: tutorial Language: en Sources: [Firebase Blog]() Topics: [Firestore](), [Server-side rendering](), [React](), [Next.js](), [web applications]() Tags: [firebase](), [firestore](), [next-js](), [react](), [real-time](), [serialization](), [server-side-rendering](), [ssr](), [web](), [web-app]() ### AI overview This tutorial explains how to use Firebase JS SDK SSR-specific APIs to serialize Firestore query results on the server, resume them on the client, and support realtime updates in a Next.js and React application. It also discusses payload size and when serialization is unnecessary. ### Source excerpt News, tutorials, and updates from the Firebase team. ## Cloud Topics: the Metastore DevFeed: [Cloud Topics: the Metastore]() Original publisher: [Read original article]() Author: Andrew Wong Published: 2026-06-11T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Kafka](), [Cloud](), [rocksdb](), [data]() Tags: [architecture](), [cloud](), [consumer](), [engineering](), [kafka](), [learn](), [mapping](), [memory](), [object-storage](), [retention](), [rocksdb](), [scale](), [serialization](), [storage]() ### AI overview This article explains Redpanda's metastore for Cloud Topics. The metastore maps Apache Kafka offsets to byte ranges in L1 objects stored in object storage, while tracking metadata such as leader-term boundaries and compaction state. Redpanda built a general-purpose key-value store, inspired by LevelDB and RocksDB and implemented as an LSM tree, to scale metadata independently of memory, local disk, and metadata formats. ### Source excerpt Learn how Redpanda's metastore powers Cloud Topics, from offset lookups and whole cluster restore to cross-region read replicas, and why it's built to be a foundational primitive for the future. ## Custom Class Serialization in Workflow SDK DevFeed: [Custom Class Serialization in Workflow SDK]() Original publisher: [Read original article]() Author: John Lindquist Published: 2026-04-02T23:23:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [SDKs](), [JavaScript](), [Vercel]() Tags: [documentation](), [sdk](), [serialization]() ### AI overview Workflow SDK now supports custom class serialization, allowing developers to pass class instances between workflow and step functions. The @workflow/serde package enables custom serialization and deserialization through static methods, with automatic handling for arguments and return values. ### Source excerpt Workflow SDK now supports custom class serialization, letting you pass your own class instances between workflow and step functions. Workflow SDK serializes standard JavaScript types like primitives, objects, arrays, Date, Map, Set, and more. Custom class instances were previously not supported because the serialization system didn't know how to reconstruct them. With the new @workflow/serde package, you can define how your classes are serialized and deserialized by implementing two static methods using WORKFLOW_SERIALIZE and WORKFLOW_DESERIALIZE. Here's an example of how we used custom serialization in @vercel/sandbox to greatly improve DX: Once implemented, instances of your class can be passed as arguments and return values between workflow and step functions, with the serialization system handling conversion automatically. See the full example application utilizing @vercel/sandbox and Workflow SDK. Read the serialization documentation to learn more. Read more ## The maturity gap in ML pipeline infrastructure DevFeed: [The maturity gap in ML pipeline infrastructure]() Original publisher: [Read original article]() Published: 2026-01-26T00:00:00Z Content type: opinion Language: en Sources: [Chainguard: Unchained]() Topics: [Security](), [Architecture & Design](), [Tooling](), [Machine learning](), [Software Engineering](), [PyTorch]() Tags: [ai-coding](), [attacks](), [chainguard-pytorch-image](), [infrastructure](), [machine-learning](), [machine-learning-pipelines](), [ml](), [ml-ops](), [pipeline](), [python](), [pytorch](), [secure-by-default](), [security](), [security-best-practices](), [serialization](), [serialization-format](), [tooling]() ### AI overview This article argues that ML pipeline infrastructure in 2026 has a security maturity gap: common tooling does not yet provide the secure-by-default protections expected in software engineering. It examines risks including data poisoning, model laundering, and insecure model serialization, and discusses short-term mitigations and longer-term industry improvements. ### Source excerpt ML pipelines in 2026 still lack secure-by-default tooling. Learn the key security gaps in ML Ops and how teams can reduce risk today. ## Embedded records - extracting data from classes DevFeed: [Embedded records - extracting data from classes]() Original publisher: [Read original article]() Author: Stephen Colebourne (noreply@blogger.com) Published: 2025-11-05T06:30:00Z Content type: article Language: en Sources: [Stephen Colebourne]() Topics: [Java](), [data]() Tags: [classes](), [data](), [java](), [javaideas](), [serialization]() ### AI overview The article discusses embedded records, a proposal to extend the benefits of Java records to classes that represent data. It addresses the gap between records and classes and mentions possible relevance to Serialization 2.0. ### Source excerpt At Devoxx Belgium 2025 I discussed the idea of embedded records with a few people. The idea is to take what is great about records, and extend that to classes that represent data. This responds to a pain point in Java, where there is a bit of a cliff-edge between records and classes. While millions of classes could and should be converted to records, millions more cannot. Yet they still represent data, and it would be a Good Thing to be able to capture that. Especially with Serialization 2.0 on the horizon. Please see the proposal document for more details. ## Making complex JSON 58x faster, use 3,300x less memory, in ClickHouse DevFeed: [Making complex JSON 58x faster, use 3,300x less memory, in ClickHouse]() Original publisher: [Read original article]() Author: Pavel Kruglov Published: 2025-10-09T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [JSON](), [data](), [Amazon S3]() Tags: [analytics](), [clickhouse](), [data](), [json](), [memory](), [performance](), [s3](), [serialization](), [storage]() ### AI overview The article explains how ClickHouse v25.8 improves performance for workloads containing thousands or tens of thousands of unique JSON paths. New shared-data serializations, including bucketed shared data, allow ClickHouse to read only the relevant data instead of scanning an entire shared column, reducing data scanned and memory usage while avoiding excessive file-system overhead. ### Source excerpt The latest changes to the JSON data type in v25.8 make ClickHouse the leader in analytics over JSON data. ## Supporting faster file load times with memory optimizations in Rust DevFeed: [Supporting faster file load times with memory optimizations in Rust]() Original publisher: [Read original article]() Author: Raghav Anand Published: 2025-06-18T00:00:00Z Content type: article Language: en Sources: [Figma Blog]() Topics: [Figma](), [Rust](), [Data structures](), [User Experience]() Tags: [data](), [efficiency](), [figma](), [memory](), [profiling](), [rust](), [serialization](), [user-experience](), [vector]() ### AI overview Figma describes how it reduced memory usage and improved file load performance by replacing a default Rust map with a flat, sorted vector. The optimization was based on profiling Figma file property maps, which contained relatively few keys within a constrained keyspace. ### Source excerpt Memory efficiency is essential for a great user experience. To keep files fast and performant, the Figma team is always hunting for optimizations--here are a few. ## JSON - The Fine Print: Part 3 - Zero vs Missing Values DevFeed: [JSON - The Fine Print: Part 3 - Zero vs Missing Values]() Original publisher: [Read original article]() Published: 2025-05-19T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [JSON](), [Go Language](), [Code]() Tags: [api-layers](), [code](), [common-serialization-errors](), [data-serialization](), [data-structure-separation](), [data-transmission](), [data-validation](), [encoding-json](), [go](), [go-json-types](), [go-language-serialization](), [go-language-validation](), [go-type-mapping](), [json](), [json-best-practices](), [json-go-mapping](), [json-marshaling](), [json-unmarshaling](), [json-validation](), [passing-serialized-data](), [rest-apis](), [schema-less-data-format](), [serialization](), [serialization-mistakes](), [serialization-protocols](), [unmarshalling]() ### AI overview This tutorial examines how to distinguish zero values from missing or NULL fields when consuming JSON in Go. It explains basic unmarshalling into structs, notes that absent fields do not cause errors, and introduces DisallowUnknownFields for validating exact JSON input. ### Source excerpt Introduction In part 1 we took a higher level view on serialization in general and JSON in specific. In part 2 we looked at emitting JSON. In part 3, we'll look at an issue you might encounter when consuming JSON, Zero vs NULL field values. To clarify the definition of NULL, this means the absence of value. So here is the question: Given a field in a Go struct set to its zero value, how do you know that zero value was set by the user or it's zero because it was never provided? ## Election 2029: Storage DevFeed: [Election 2029: Storage]() Original publisher: [Read original article]() Author: jonskeet Published: 2025-03-27T11:46:31Z Content type: tutorial Language: en Sources: [Jon Skeet]() Topics: [Firestore](), [Databases](), [JSON](), [Code](), [Development]() Tags: [code](), [database](), [development](), [election-2029](), [firestore](), [json](), [serialization]() ### AI overview The article describes storage for the Election 2029 project, using separate Firestore databases for test, staging, and production alongside JSON files for local development. It introduces an abstraction interface and discusses patterns for serializing data models to JSON or XML, with particular attention to the Firestore implementation. ### Source excerpt Storage Since my last post about the data models, I've simplified things very slightly - basically the improvements that I thought about while writing the post have now been implemented. I won't go into the details of the changes, as they're not really important, but that's just to explain why some examples might look like ... Continue reading Election 2029: Storage -> ## JSON - The Fine Print: Part 2 - Emitting JSON DevFeed: [JSON - The Fine Print: Part 2 - Emitting JSON]() Original publisher: [Read original article]() Published: 2025-02-04T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [JSON](), [API](), [Code](), [Structured-data]() Tags: [api](), [api-layers](), [code](), [common-serialization-errors](), [data](), [data-serialization](), [data-structure-separation](), [data-transmission](), [data-validation](), [encoding](), [encoding-json](), [function](), [go-json-types](), [go-language-serialization](), [go-language-validation](), [go-type-mapping](), [http](), [io](), [io-writer](), [json](), [json-best-practices](), [json-go-mapping](), [json-marshaling](), [json-unmarshaling](), [json-validation](), [map](), [passing-serialized-data](), [properties](), [rest-apis](), [schema](), [schema-less-data-format](), [serialization](), [serialization-mistakes](), [serialization-protocols](), [standard](), [structure](), [writing]() ### AI overview This tutorial explains how to emit JSON in Go using the encoding/json package. It compares json.Marshal, which returns bytes, with json.Encoder, which writes to an io.Writer, and discusses error handling in HTTP handlers. It also demonstrates marshaling maps and structs and using field tags to control JSON property names. ### Source excerpt Introduction In part 1 we took a high-level view on serialization and JSON. In this part, we'll roll our sleeves and start working with JSON, focused on emitting JSON. You might think this is a basic topic, but there is much more to it than just calling json.Marshal. json.Marshal vs json.Encoder The encoding/json package has two main APIs: Marshal and NewEncoder. The Marshal function returns a []byte while the NewEncoder function will write to an io.Writer. The question is: When should you use one API over the other? ## Learning Path: Important Standard Library Modules DevFeed: [Learning Path: Important Standard Library Modules]() Original publisher: [Read original article]() Author: Real Python Published: 2024-11-14T12:00:00Z Content type: tutorial Language: en Sources: [Real Python]() Topics: [Python](), [Programming](), [CSV](), [JSON](), [math](), [data]() Tags: [data](), [json](), [learning](), [math](), [python](), [serialization](), [tutorials]() ### AI overview A Python learning path covering essential standard library modules for mathematics, dates and times, random data, serialization, JSON, CSV, XML, regular expressions, and system tools. ### Source excerpt Explore Python standard library modules including math, datetime, JSON, CSV, regex, subprocess, and argparse through hands-on tutorials and courses. ## JSON Handling and Type Management with Large Datasets in Go - Ep.4 DevFeed: [JSON Handling and Type Management with Large Datasets in Go - Ep.4]() Original publisher: [Read original article]() Published: 2024-11-14T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [Go Language](), [JSON](), [Code](), [data]() Tags: [advanced-json-management-go](), [code](), [custom-go-struct-json-serialization](), [custom-json-serialization-go](), [decoding](), [efficient-json-processing-in-go](), [encoding-json](), [examples](), [go](), [go-encoding-json-package](), [go-json-handling-tutorial](), [go-selective-decoding](), [go-type-management-json](), [handling-mixed-type-arrays-json-go](), [interfaces](), [introduction](), [json](), [json-anonymous-structures-go](), [json-best-practices](), [json-data-optimization-techniques](), [json-for-engineers](), [json-handling-techniques-for-engineers](), [json-marshaler-and-unmarshaler-interfaces-in-go](), [json-nested-fields-go](), [json-strategies-large-datasets](), [json-type-pollution-avoidance](), [managing-large-json-datasets-in-go](), [pitfalls](), [reducing-type-declarations-go](), [serialization]() ### AI overview Episode 4 of the JSON for Engineers series explains how to process large JSON datasets in Go. It covers selective decoding with anonymous structures, custom JSON serialization through Marshaler and Unmarshaler interfaces, and handling mixed-type arrays. ### Source excerpt Introduction: Welcome to Episode 4 of the JSON for Engineers series! In this episode, we tackle the complexities of working with JSON data, especially when dealing with extensive datasets and optimizing type management. Here, Miki introduces key strategies for managing large JSON documents, avoiding unnecessary type definitions, and using Go's encoding/json package to handle custom serialization needs effectively. Type Pollution Avoidance: Minimizing type declarations by using selective decoding and anonymous structures. Custom JSON Serialization: Implementing Go's Marshaler and Unmarshaler interfaces for precise JSON formatting. Working with Mixed-Type Arrays: Handling JSON arrays containing diverse types in Go Miki begins by addressing the challenge of "type pollution" when working with large JSON responses. When dealing with massive JSON datasets, like data from social trading platforms, defining a separate type for every possible field can quickly lead to an unwieldy codebase. To streamline this, Miki demonstrates how to leverage anonymous structures and selective decoding in Go. By defining only the relevant fields within a function, we can avoid creating additional struct types, making the code more compact and efficient. This technique is especially useful when working with JSON data containing nested fields that may not all be necessary for your application. Through this approach, Go's encoding/json package will simply ignore any unreferenced fields, allowing you to handle only the data that matters to your project. ## JSON - The Fine Print: Part 1 DevFeed: [JSON - The Fine Print: Part 1]() Original publisher: [Read original article]() Published: 2024-10-23T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [JSON](), [API]() Tags: [api-layers](), [apis](), [common-serialization-errors](), [data-serialization](), [data-structure-separation](), [data-transmission](), [data-validation](), [database](), [elasticsearch](), [encoding-json](), [go-json-types](), [go-language-serialization](), [go-language-validation](), [go-type-mapping](), [json](), [json-best-practices](), [json-go-mapping](), [json-marshaling](), [json-unmarshaling](), [json-validation](), [passing-serialized-data](), [rest-apis](), [schema-less-data-format](), [sensitive-data](), [serialization](), [serialization-mistakes](), [serialization-protocols]() ### AI overview This article explains JSON and serialization fundamentals, then examines common mistakes. It recommends serializing data at program boundaries and using separate data structures for API, business, and storage layers to avoid inefficiency, coupling, and accidental exposure of sensitive data. ### Source excerpt Introduction Everybody knows JSON, it's a simple serialization format and the default format for REST APIs. Like many other topics, there are fine points you should know in order to work with JSON more effectively and avoid common mistakes. In this article we'll explore some big picture aspects and some low level details of using JSON. Serialization Before diving into JSON, I'd like to take a look at serialization in general and discuss common mistakes I've seen my customers make. ## Data Integrity with JSON Serialization - Ep.1 DevFeed: [Data Integrity with JSON Serialization - Ep.1]() Original publisher: [Read original article]() Published: 2024-10-03T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [JSON](), [Software Engineering](), [software-architecture]() Tags: [common-serialization-mistakes](), [custom-json-validation](), [data-integrity-in-software-engineering](), [data-serialization-in-api-design](), [data-validation](), [efficient-data-serialization](), [how-to-validate-json-data](), [introduction-to-json-for-engineers](), [json](), [json-best-practices](), [json-best-practices-for-api](), [json-data-validation-techniques](), [json-for-engineers](), [json-in-software-development](), [json-optimization-for-storage](), [json-serialization-tutorial](), [miki-tebeka-json-guide](), [serialization](), [serialization-in-software-architecture](), [software-architecture](), [software-layer-separation-and-data-handling]() ### AI overview Episode 1 of JSON for Engineers explains serialization fundamentals and how JSON converts data structures to byte sequences. It recommends serializing at system boundaries such as APIs and storage, keeping native types internally, separating models across layers, and validating data beyond JSON syntax. ### Source excerpt Introduction: Welcome to Episode 1 of JSON for Engineers! In this first episode, Miki Tebeka dives into the fundamentals of serialization, with a special focus on JSON, one of the most widely-used data formats in software engineering. Miki draws from his extensive development experience to explain how JSON fits into the bigger picture of data interchange and serialization, laying the groundwork for engineers to use it effectively in real-world applications. ## Swift deserialization security primer DevFeed: [Swift deserialization security primer]() Original publisher: [Read original article]() Author: Sam Sanoop Published: 2023-07-18T16:00:00Z Content type: article Language: en Sources: [Blog RSS Feed | Snyk]() Topics: [Swift](), [Security](), [Vulnerabilities](), [Code](), [Protocol (disambiguation)]() Tags: [apis](), [attacks](), [awareness](), [blog](), [code](), [code-security](), [developer](), [distributed](), [engineering](), [exploit](), [remote-code-execution](), [security](), [security-vulnerabilities](), [serialization](), [swift](), [vulnerabilities]() ### AI overview This Swift security primer explains how deserialization vulnerabilities can arise when using NSCoding and NSSecureCoding. It describes object substitution attacks, potential remote code execution, command injection, and stricter protections provided by NSSecureCoding. ### Source excerpt This blog will detail deserialization vulnerabilities in Swift that can occur when using the popular APIs, NScoding and NSSecureCoding, and how to prevent it properly. ## Data replication across backend services with Kafka and Protobuf DevFeed: [Data replication across backend services with Kafka and Protobuf]() Original publisher: [Read original article]() Author: Emmanuel Joubaud Published: 2023-04-14T10:01:57Z Content type: tutorial Language: en Sources: [JobTeaser]() Topics: [Back end](), [Kafka](), [Replication](), [data-architecture](), [API](), [Structured-data]() Tags: [api](), [architecture](), [backend](), [data](), [data-replication](), [distributed-systems](), [kafka](), [microservices](), [replication](), [serialization](), [serialization-format](), [software-architecture](), [software-engineering]() ### AI overview This tutorial explains Jobteaser's Silos architecture for replicating data across backend services. It describes using Kafka topics and Google Protobuf messages so consumer services can maintain local database copies while avoiding synchronous service-to-service API calls. ### Source excerpt The Jobteaser application contains a lot of different relatively independent modules to help universities provide career guidance to students: a job board, a career event management system, a career advice appointment management system... When we decided to migrate our application's backend from a monolith to a service-oriented architecture, we strived to keep each module as isolated as possible from the others in the event of an incident. If the career appointment system was down, students should still be able to browse and apply to job ads. That isolation is achieved through what we've called our Silos architecture. The gist is we avoid synchronous API calls between backend services, and prefer asynchronous data integration between services. There are a lot of different ways to implement asynchronous communication between services, and few companies share the details of theirs so we had to figure out a lot of stuff on our own. Now that we've refined our system, we thought we'd share the details of our approach, based on simple data replication using Kafka and Protobuf. How it worksWhen a write operation happens, data is changed first on the Source of Truth service, who then publishes it to a Kafka topic that consumers can subscribe to, in order to replicate their own local copy in their databaseSource of Truth and consumers Every table in our data model has an owner service, also called the Source of Truth (SoT) for that data, or the producer. For instance, the job board service may be the SoT for the job ad data. The owner service has 3 main responsibilities: it receives and validates all write requests for the data it owns (Create, Update, Delete) it stores the data to its own database, i.e. the Source of Truth, the authoritative state of the data if it's data that other services might want to access, it publishes its latest state as a message into a Kafka topic Consumer services that are interested in accessing a given table can then subscribe to the Kafka topic ## Caching Without Marshal Part 2: The Path to MessagePack DevFeed: [Caching Without Marshal Part 2: The Path to MessagePack]() Original publisher: [Read original article]() Author: Chris Salzberg Published: 2022-11-15T15:00:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Caching](), [Cache](), [Rails](), [Ruby](), [data]() Tags: [cache](), [caching](), [code](), [rails](), [ruby](), [serialization](), [serialization-format]() ### AI overview Shopify's second article on replacing Ruby's Marshal cache format explains how MessagePack provides a more robust binary serialization format for Rails caching. Unlike Marshal, MessagePack is language-independent and avoids encoding Ruby-specific types that can make cached data fail after code changes. ### Source excerpt Shopify wanted a cache format that would not blow up when we shipped code changes. Part two of Caching Without Marshal describes the MessagePack format, and how we migrated to MessagePack. ## Kotlin JSON Benchmark on Android (2022): Moshi vs Kotlin Serialization DevFeed: [Kotlin JSON Benchmark on Android (2022): Moshi vs Kotlin Serialization]() Original publisher: [Read original article]() Author: Christophe Beyls Published: 2022-10-14T17:31:03Z Content type: comparison Language: en Sources: [Stories by Christophe Beyls on Medium]() Topics: [JSON](), [Kotlin](), [Benchmark](), [Gson](), [build times](), [Code generation](), [legacy]() Tags: [android-app-development](), [annotation-processor](), [benchmark](), [gson](), [json](), [kotlin](), [kotlin-serialization](), [legacy](), [moshi](), [performance](), [serialization](), [vs]() ### AI overview This article sets up a 2022 Android benchmark comparing Moshi and Kotlin Serialization for JSON serialization and deserialization of Kotlin classes. It discusses Kotlin metadata compatibility, generated adapters, reflection, streaming support, Okio integration, runtime dependencies, and build-time tradeoffs. The supplied text does not include the benchmark results or identify which library is fastest. ### Source excerpt When it comes to automatic serialization and deserialization of Kotlin classes using the JSON format, the two main libraries compatible with Kotlin metadata are currently Moshi and Kotlin Serialization. This compatibility is especially important for non-null types and default values during deserialization, where lack of proper Kotlin support could result in unexpected values occurring at runtime, such as null values in non-null fields. If you're still using legacy libraries like Gson to parse Kotlin classes, it's time to reconsider. Moshi has been supporting Kotlin classes since version 1.5.0, released in 2017. One year later, the next major release 1.6.0 added an annotation processor to generate adapters for Kotlin classes at compile time. I was quite interested in the performance gains allowed by this solution and wrote an article detailing what the generated code does. JetBrains released version 1.0.0 of Kotlin Serialization in 2020 with built-in support for JSON. This library generates adapters at compile time similarly to Moshi's annotation processor while being compatible with more platforms and formats. The lack of initial support for streaming was disappointing, so I didn't even consider using it in production until streaming support was eventually added in version 1.3.0 in 2021. The JSON engine had also been rewritten in the meantime in order to improve performance. Recently, version 1.4.0 added integration with the Okio library, the same library used by Moshi under the hood. Now that Kotlin Serialization looks full-featured and well-optimized, I thought it would be a good time to compare its performance against Moshi on Android devices with some benchmarks. Which one is the fastest? Take your bets. The contendersMoshi-Kotlin Reflection The runtime Kotlin plugin of the Moshi library. The JSON adapters are generated at runtime using reflection. The main downside of this library is that it adds a runtime dependency to the big kotlin-reflect library (currently ## KStore: A Kotlin Multiplatform Library for Saving and Restoring Objects to Disk DevFeed: [KStore: A Kotlin Multiplatform Library for Saving and Restoring Objects to Disk]() Original publisher: [Read original article]() Author: Isuru Rajapakse Published: 2022-10-02T16:01:17Z Content type: tutorial Language: en Sources: [Stories by Isuru Rajapakse on Medium]() Topics: [Kotlin Multiplatform](), [Library](), [Kotlin](), [Coroutines](), [JSON](), [Testing](), [CI/CD](), [Filesystems](), [GitHub Actions](), [Open Source](), [test-coverage]() Tags: [ci-cd](), [code-coverage](), [coroutines](), [dependencies](), [github-actions](), [kotlin](), [kotlin-multiplatform](), [library](), [open-source](), [serialization](), [testing]() ### AI overview KStore is a small Kotlin Multiplatform library for saving and restoring serialized objects to disk. The article describes its coroutine-based API, storage operations, mutex-protected updates, and project tooling for testing, CI/CD, code coverage, and binary compatibility validation. ### Source excerpt A tiny Kotlin multiplatform library to save and restore objects to and from the disk So I put out a new library, my first Multiplatform library. KStore is a tiny Kotlin multiplatform library that assists in saving and restoring objects to and from disk using kotlinx.coroutines, kotlinx.serializationand okio Why? A newer version of kotlinx.serialization was released and one thing caught my eye Integration with Okio's BufferedSource and BufferedSink Release 1.4.0-RC - Kotlin/kotlinx.serialization However, there was little to no documentation on how to use it. I scanned through the repository looking for tests but couldn't find any. I asked on slack Got nothing.. But I pulled it down and tried it out anyways. dependencies { implementation("org.jetbrains.kotlinx:kotlinx-serialization-json-okio:1.4.0") } Tried out the most obvious way and wrote up a test case Json.encodeToBufferedSink(value, FILE_SYSTEM.sink(path).buffer()) No luck. Asked around on slack once again Not a minute went by, and our lord and saviour himself descended from heaven to help me out After using use, it worked 🙌. His divinely presence inspired me to write out a bunch more test cases and put this out there in under ~48hrs GitHub - xxfast/KStore: A tiny Kotlin multiplatform library that assists in saving and restoring objects to and from disk using kotlinx.coroutines, kotlinx.serialisation and okio API KStore is inspired by RxStore, and it provides a simple API. Most of these interface methods are suspended, and can only be invoked from a suspended context. SuspendedCreate a storeval store: KStore = storeOf("path/to/file")Get valueval mylo: Pet? = store.get() Or observe for changes val pets: Flow = store.updatesSet valuestore.set(mylo)Update a valuestore.update { pet: Pet? -> pet?.copy(age = pet.age + 1) } Note: this maintains a single mutex lock transaction, unlike get() and a subsequent set() Delete/Reset valuestore.delete() You can also reset a value back to its default. store.reset()Sel ## Making Open Source Safer for Everyone with Shopify's Bug Bounty Program DevFeed: [Making Open Source Safer for Everyone with Shopify's Bug Bounty Program]() Original publisher: [Read original article]() Author: Zack Deveau Published: 2022-06-15T18:00:02Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Application Security](), [Rails](), [vulnerability](), [Bug Bounty](), [Security](), [Open Source](), [Ruby]() Tags: [application-security](), [bug-bounty](), [open-source](), [rails](), [remote-code-execution](), [ruby](), [serialization](), [shopify](), [vulnerability]() ### AI overview Shopify describes how a bug bounty report led to contributions that improve the security of the Rails library. The report identified a deserialization vulnerability in ActiveSupport's MessageEncryptor, where leaked development secrets could enable seemingly trusted data and potentially remote code execution. Shopify addressed the issue in its system, helped Rails adopt safer default serializers, and developed tools to help Ruby application developers identify similar risks. ### Source excerpt Zack Deveau, Senior Application Security Engineer at Shopify, shares the details behind a recent contribution to the Rails library, inspired by a bug bounty report we received. He'll go over the report and its root cause, how we fixed it in our system, and how we took it a step further to make Rails more secure by updating the default serializer for a few classes to use safe defaults. ## AttributedString's Codable format and what it has to do with Unicode DevFeed: [AttributedString's Codable format and what it has to do with Unicode]() Original publisher: [Read original article]() Author: Ole Begemann Published: 2022-04-27T13:28:03Z Content type: article Language: en Sources: [Ole Begemann]() Topics: [Swift](), [JSON](), [Code]() Tags: [code](), [json](), [serialization](), [swift]() ### AI overview The article examines how to encode Swift AttributedString values and explains why character-based or UTF-8 byte-based formatting ranges can become invalid across Unicode versions, Swift versions, operating systems, or normalization changes. It discusses storing UTF-8 bytes as a safer alternative. ### Source excerpt Here's a simple AttributedString with some formatting: import Foundation let str = try! AttributedString( markdown: "Café **Sol**", options: .init(interpretedSyntax: .inlineOnly) ) AttributedString is Codable. If your task was to design the encoding format for an attributed string, what would you come up with? Something like this seems reasonable (in JSON with comments): { "text": "Café Sol", "runs": [ { // start..) Original publisher: [Read original article]() Author: Dominik Liebler Published: 2022-01-24T08:04:19Z Content type: tutorial Language: en Sources: [YAZIO Engineering - Medium]() Topics: [Kafka](), [Apache Spark](), [Streaming](), [data lake](), [parquet](), [Kotlin](), [Kubernetes](), [ceph](), [JSON Schema]() Tags: [apache-spark](), [backpressure](), [ceph](), [confluent](), [data-engineering](), [data-lake](), [data-pipeline](), [json](), [kafka](), [kotlin](), [kubernetes](), [parquet](), [payload](), [pipelines](), [schema](), [schemaregistry](), [serialization](), [spark](), [streaming]() ### AI overview This engineering article explains YAZIO's data pipeline from mobile and web applications through Kafka and Spark Structured Streaming into a Ceph-based data lake. It discusses why schemas matter and describes replacing JSON with Apache Avro and Confluent Schema Registry to reduce message size while keeping schema information externally stored and cached. ### Source excerpt At YAZIO, we believe in making decisions backed by data to help people live healthier lives through better nutrition. For each new and existing feature we want to evaluate how well it performs and how our users interact with it. In order to do so, we need a lot of data and we need to handle backpressure in our systems. To cope with that we use a Kafka cluster managed by Strimzi operators running in Kubernetes. The data itself is being ingested from our mobile and web apps via HTTP or TCP endpoints serialized into JSON and stored in Kafka by a small application written in Kotlin/JVM. Overview of our data pipeline architecture At the other end of the pipeline, different Spark Structured Streaming applications (also written in Kotlin) dump this information into our data lake residing in a Ceph bucket. They read data from Kafka, deserialize it, transform some of the fields and write Parquet files into the data lake using a new schema. Why schemas? Schemas play an important role in data pipelines because they give meaning and context to data. In a world without schemas we would still do random interpretations about the context and meaning of data every now and then when using it. As you might have guessed already this would lead to a lot of bugs and misunderstandings. Photo by EJ Strat https://unsplash.com/photos/VjWi56AWQ9k Similar to a legal contract that binds you to certain limits, a schema binds the data to certain limits and meaning which narrow down the need of interpretation. Choice of serialization formats At the time of writing, Confluent Schema Registry supports these three serialization formats: Apache Avro Protocol Buffers (protobuf) JSON Schema From those choices, only two really provide more than just validation of the data that is ingested and transmitted through our data pipelines. Avro and Protobuf also allow us to shrink the sizes of our topics because only the payload is contained in a message, while the repeating schema will not be stored. In the cas [Next page]()