# Categorizing Data with Large Language Models in Rust

DevFeed: [Categorizing Data with Large Language Models in Rust](<https://devfeed.tech/articles/categorizing-data-with-large-language-models-in-rust-22248.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2024/07/categorize_data_large_language_model_rust.html>)

Published: 2024-08-06T00:00:00Z

Content type: tutorial

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [data](<https://devfeed.tech/topics/data.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Networks](<https://devfeed.tech/topics/networks.md>), [CSV](<https://devfeed.tech/topics/csv.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [networks](<https://devfeed.tech/tags/networks.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [rust](<https://devfeed.tech/tags/rust.md>)

## AI overview

This tutorial describes processing ASN and domain data for LibreQoS, an open-source project that monitors Internet traffic and connection quality. It shows how to obtain ASN data as CSV files and use Rust to load and de-duplicate domain values efficiently.

## Source excerpt

Introduction LibreQoS is an open source project for monitoring and providing quality-of-experience for Internet Service Providers (ISPs) and large networks. It runs as a "middle-box", monitoring traffic that passes through it. It recently gained the ability to track individual data flows - connections between two endpoints. It's also my favorite Open Source project, to which I contribute regularly. Public Internet IP addresses belong to an ASN - an Autonomous System Number. An ASN is a unique number that identifies a network on the Internet. Tracking flows allows you to see which ASNs your users are connecting to, how much data is flowing, and monitor the quality of the connection.