# Xet is on the Hub

DevFeed: [Xet is on the Hub](<https://devfeed.tech/articles/xet-is-on-the-hub-7570.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/xet-on-the-hub>)

Author: Assaf Vayner; Brian Ronan; Di Xiao; Joseph Godlewski; Sam Horradarn; Jared Sulzdorf

Published: 2025-03-18T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [xet](<https://devfeed.tech/topics/xet.md>), [content defined chunking](<https://devfeed.tech/topics/content-defined-chunking.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [migration](<https://devfeed.tech/topics/migration.md>), [parquet](<https://devfeed.tech/topics/parquet.md>), [SQLite](<https://devfeed.tech/topics/sqlite.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [content-addressed-store](<https://devfeed.tech/tags/content-addressed-store.md>), [content-defined-chunking](<https://devfeed.tech/tags/content-defined-chunking.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [migration](<https://devfeed.tech/tags/migration.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [performance](<https://devfeed.tech/tags/performance.md>), [sqlite](<https://devfeed.tech/tags/sqlite.md>), [storage](<https://devfeed.tech/tags/storage.md>), [xet](<https://devfeed.tech/tags/xet.md>)

## AI overview

Hugging Face describes migrating the first Model and Dataset repositories from LFS to Xet storage. The article explains how content-defined chunking enables byte-level deduplication, reducing the data transferred for small edits to large files and improving upload performance.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.