# Detecting Trustworthy Domains

DevFeed: [Detecting Trustworthy Domains](<https://devfeed.tech/articles/detecting-trustworthy-domains-31895.md>)

Original publisher: [Read original article](<http://engineering.flipboard.com//2017/04/domainranking>)

Author: https://www.linkedin.com/in/mikecora/ (Mike Vlad Cora)

Published: 2017-04-12T00:00:00Z

Content type: article

Language: en

Sources: [Flipboard](<https://devfeed.tech/sources/flipboard.md>)

Topics: [Machine Learning & Artificial Intelligence](<https://devfeed.tech/topics/machine-learning-artificial-intelligence.md>), [Python](<https://devfeed.tech/topics/python.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>)

Tags: [article](<https://devfeed.tech/tags/article.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [ml](<https://devfeed.tech/tags/ml.md>), [python](<https://devfeed.tech/tags/python.md>), [quality](<https://devfeed.tech/tags/quality.md>)

## AI overview

Flipboard describes its Domain Ranker, a machine learning system that predicts how its editorial team would label domains. The article focuses on the engineering work behind the system, including live data pipelines, feature engineering, and classifier experiments using scikit-learn in Python.

## Source excerpt

High quality, truthful, diverse and informative content is Flipboard's #1 priority. Hand-picking trusted sources guarantees quality, but is very time consuming, and can potentially miss out on the multitude of excellent but smaller publishers. To address this problem, we've developed a machine learning (ML) system called the Domain Ranker. Its goal is to automatically distinguish authoritative domains from plagiarists, spammers and other low quality sources. It learns to predict how our editorial team would label a domain by analyzing the content and the signals generated by our users. The Domain Ranker scales our editorial thinking to a much larger amount of content than we could handle manually, ensuring high quality across all topics. Overview Flipboard has indexed hundreds of millions of articles in the last year alone. In addition to the constant stream of articles from our trusted partners, any user can add any web article into their personal magazines, further expanding Flipboard's pool to non-partner publishers. Our community support and editorial teams are constantly battling the endless churn of spam sites. Through their efforts, we have identified thousands of labeled spam domains, alongside thousands of partner and whitelisted publishers. The Domain Ranker is a machine learning system that uses this labeled data to learn and generalize our editorial thinking to unlabeled sources. In this blog post I do not delve into the theory behind any of the machine learning classifiers used. They are all well known, off-the-shelf implementations in the scikit-learn Python library. Instead I will focus on the engineering journey: managing the live data pipeline, exploring and engineering reasonable features, and experimenting with a multitude of classifiers to maximize accuracy. I compare an ML project to an open-ended "Choose Your Own Adventure" book: every path leads to an almost unlimited number of forks, fraught with perils and rewards. There are many paths that