# Build awesome datasets for video generation

DevFeed: [Build awesome datasets for video generation](<https://devfeed.tech/articles/build-awesome-datasets-for-video-generation-7553.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/vid_ds_scripts>)

Author: Sayak Paul

Published: 2025-02-12T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [datasets](<https://devfeed.tech/topics/datasets.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Tooling](<https://devfeed.tech/topics/tooling.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [generation](<https://devfeed.tech/tags/generation.md>), [guide](<https://devfeed.tech/tags/guide.md>), [ocr](<https://devfeed.tech/tags/ocr.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [tool](<https://devfeed.tech/tags/tool.md>), [training](<https://devfeed.tech/tags/training.md>), [video](<https://devfeed.tech/tags/video.md>)

## AI overview

This article presents open tooling for building datasets used to fine-tune video generation models. It describes a three-stage pipeline for downloading videos, splitting them into clips, filtering for watermarks, aesthetics, NSFW content, and motion, and applying captioning, object recognition, and OCR to extracted frames. The tooling is intended to support both small-scale community projects and larger video dataset efforts.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.