# training-data Published articles for training-data. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Interpreting Pangram DevFeed: [Interpreting Pangram]() Original publisher: [Read original article]() Author: Armin Ronacher Published: 2026-09-14T00:00:00Z Content type: opinion Language: en Sources: [Armin Ronacher]() Topics: [Artificial Intelligence](), [Large Language Model](), [datasets](), [Training AI Models]() Tags: [ai](), [llm](), [model](), [openai](), [training](), [training-data]() ### AI overview The article discusses Pangram, an AI-text detector that classifies passages as human-written, AI-generated, or mixed. It explains that Pangram manufactures training data from human-authored text and LLM-generated rewrites and edits, then describes an experiment using Opus 5 to generate text intended to read as entirely AI-generated. ### Source excerpt Yesterday David Sacks wrote a tweet and within a few minutes people did, what they usually do, and they asked Pangram if it was AI. And Pangram said it's entirely AI generated. To which David replied that these AI detectors are bogus. Now Pangram has a pretty low false positive rate, but if you have ever used an LLM as a writing assitant, you will have probably noticed that it claims your posts 100% AI, even though you don't feel like they are. Pangram itself is a trained model, that attempts to detect segments of text as being definitely human, definitely AI and a mixture of the two. If you want to know how it works, they published a paper. The short summary is that they are manufacturing its own training data by starting from collections of known human authored text. An LLM is then tasked to understand the text and write a fresh new text on the same topic. They also let the LLM perform partial edits on that original human text and through that they can pick up on these co-authored details. Pangram claims their model to have rates of 0.0041% false AI accusations and 0.34% missed AI text. So now that we know this I figured it might be fun to have an LLM re-create David's tweet. I first came up with a prompt. And when I say I came up with that prompt I in fact used an LLM to propose to me from that tweet what I might want to say for the structure. I'm sure if you ask Pangram about if the above text is AI, it will probably say so, but that's not really the point. The point is that I then used Opus 5 to generate a text which reads entirely AI generated. If you are curious, this is the prompt I used: Original prompt to generate a David Sacks like tweet Please write an 8 paragraph tweet about "Pacing the Frontier", not longer than 350 words. Context of this tweet is this post by Dario: https://darioamodei.com/post/we-must-pace-the-frontier And this tweet by Sam Altman: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we' ## Some thoughts on the Navier-Stokes Millennium Prize Problem DevFeed: [Some thoughts on the Navier-Stokes Millennium Prize Problem]() Original publisher: [Read original article]() Author: Simon Willison Published: 2026-09-08T23:55:12Z Content type: opinion Language: en Sources: [Simon Willison](), [Simon Willison's Weblog]() Topics: [OpenAI](), [Mathematics](), [Claude](), [codex](), [Math and Logic](), [Lean]() Tags: [agents](), [ai](), [ai-2-235](), [ai-ethics](), [ai-ethics-342](), [claude](), [codex](), [generative-ai](), [generative-ai-1-981](), [llms](), [llms-1-947](), [mathematics](), [mathematics-22](), [openai](), [openai-463](), [paper](), [training-data](), [training-data-68]() ### AI overview This commentary examines OpenAI's reported resolution of the Navier-Stokes existence and smoothness problem with an unreleased model, alongside accusations that the effort may have drawn on information from related work by mathematicians using Claude and Codex. It also describes questions about timing, data access, authorship, and OpenAI's subsequent use of agents and Lean formalization. ### Source excerpt On the Navier-Stokes Millennium Prize Problem introduces an impressive result from OpenAI, who used an unreleased model to produce a resolution to the Navier-Stokes existence and smoothness problem, one of the seven Millennium Prize Problems that have been subject to a $1,000,000 prize since May 24th, 2000. The discovery is somewhat overshadowed by accusations of skulduggery from Tristan Buckmaster, an NYU mathematics professor who was collaborating on related problems with Levent Alpöge, an accomplished mathematician who currently works for Anthropic. Tristan's complaint accompanied a hastily published version of their own results. Here's the PDF describing what happened. The very short version is that Tristan and Levent worked on the problem for almost a year, making extensive use of Claude and Codex (mainly GPT-5.6 Sol), then had a breakthrough on August 15th. The mathematical rumour mill kicked into gear and Tristan and Levent heard that OpenAI had heard that Anthropic had resolved "a major open problem", so they reached out and learned that OpenAI had a team working on a related problem, with a similar approach. Quoting Tristan: I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI. I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. It gets more complicated from there. The OpenAI team offered to wait for Tristan to publish, or to have him author a paper about their result, but were clear that Levent would not be invited as a co-author due to OpenAI's competitive relationship with his employer. Here's how OpenAI described their work: On Tuesday, September 1, we h ## How Object Storage Performance Affects GPU Training Workloads DevFeed: [How Object Storage Performance Affects GPU Training Workloads]() Original publisher: [Read original article]() Author: Sridhar Rajarao Published: 2026-08-30T00:00:00Z Content type: article Language: en Sources: [Sridhar Rajarao]() Topics: [GPU](), [cloud-infrastructure](), [Caching](), [Disk image]() Tags: [ai](), [aws](), [cache](), [compute](), [gpu](), [nvme](), [object-storage](), [performance](), [storage](), [training-data]() ### AI overview This article explains how object-storage latency, network transfers, many small files, and concurrent requests can leave GPUs idle during AI training. It recommends sharding small samples into larger files, reusing connections, and caching data near the GPU; it also notes AWS's suggested shard range of roughly 100 MB to 1 GB for many training workloads. ### Source excerpt The GPU does not care that object storage is durable and scalable. It cares whether the next batch of data arrives before it goes idle. ## How LLMs are Actually Trained DevFeed: [How LLMs are Actually Trained]() Original publisher: [Read original article]() Author: Ashish Pratap Singh Published: 2026-06-11T04:50:19Z Content type: tutorial Language: en Sources: [AlgoMaster Newsletter]() Topics: [Training AI Models](), [LLM Techniques](), [datasets](), [Language models]() Tags: [dataset](), [llm](), [llm-training](), [llms](), [models](), [training](), [training-data]() ### AI overview This tutorial explains how large language models are trained in stages: pretraining, mid-training, and post-training. It describes pretraining datasets, next-token prediction, cross-entropy loss, and the creation and filtering of datasets such as Dolma 3 Mix. ### Source excerpt In the last lesson, we learned how the Transformer architecture powers an LLM. ## Fine-tuning a DistilBERT classifier with numerical and text inputs DevFeed: [Fine-tuning a DistilBERT classifier with numerical and text inputs]() Original publisher: [Read original article]() Author: Paloma Jol Published: 2026-04-10T10:04:33Z Content type: tutorial Language: en Sources: [FreeAgent]() Topics: [Fine-tuning](), [Transformer](), [foundation-models](), [Training AI Models](), [datasets](), [Kaggle](), [scikit-learn]() Tags: [ai](), [aws](), [data](), [data-ml](), [data-science](), [dataset](), [encoding](), [fine-tuning](), [hugging-face](), [kaggle](), [llms](), [machine-learning](), [model](), [models](), [nlp](), [performance](), [text-classification](), [training-data](), [validation]() ### AI overview This tutorial explains how to fine-tune a DistilBERT text classifier while incorporating a numerical feature into the same network. It uses the Kaggle wine reviews dataset, combining review descriptions with price and preprocessing the data for training, validation, and testing. ### Source excerpt Text classification is often done through fine-tuning of a pretrained foundation model with domain-specific data. In FreeAgent we use transformer based models to automatically classify incoming bank transactions. Specifically we use a DistilBERT model that is fine-tuned on hundreds of millions of bank transactions with customer-labelled accounting categories. The model inputs are currently text-based, built from a combination of bank transaction descriptions and amounts. In this post we describe an approach to fine-tuning the DistilBERT model and training the classifier including the numerical amount feature as a single network. ## PostmarketOS demonstrates Gboard integration for Linux mobile keyboards DevFeed: [PostmarketOS demonstrates Gboard integration for Linux mobile keyboards]() Original publisher: [Read original article]() Published: 2025-04-01T00:00:00Z Content type: tutorial Language: en Sources: [postmarketOS Blog]() Topics: [keyboard](), [Mobile](), [Linux](), [Android APK](), [Software](), [Android]() Tags: [android](), [android-apk](), [apk](), [crash](), [environment](), [focus](), [keyboard](), [linux](), [mobile](), [open-source](), [other](), [postmarketos](), [tensorflow](), [training-data]() ### AI overview PostmarketOS demonstrates an experimental keyboard setup in Phosh using Gboard through ATL. The prototype offers suggestions, swiping, autocorrection, and smart input detection, but it crashes often and will not ship because it relies on proprietary software. ### Source excerpt In our previous blog post 2025 Priorities: Reliability, Audio, Cameras and More, we asked "is there anything that you find terribly important that we have missed here?". And indeed, there was one thing that people would have liked us to focus more on, which is a better keyboard experience. This was unbelievably great timing, because we are actually integrating a new keyboard with the following features: Word complete suggestions Swiping Autocorrection of misspelled words Smart detection of what you meant to type vs. which buttons you have actually hit with your fingers We have hooked it up to Phosh, see it for yourself (this is a video, you might need to click it in order to have it start playing): Tech stack Please note that this is in early development. Currently it is still crashing often and you also need to be comfortable running proprietary software since this is actually Gboard running through ATL. So uh... on second thought we will not be shipping this in postmarketOS. However the underlying technology is very much worth showcasing since it allows running components and apps from that other operating system on proper Linux (Mobile) without running a full container! The video is real, and it shows that it is possible to build features like autocorrection and swiping with the APIs we already have in Linux Mobile today. In fact, phosh-osk-stub and swipeGuess can do most of this already! If somebody out there has the interest, skills and time to improve the Linux mobile keyboards to the point that they work as intuitively as proprietary ones: it is possible, get involved! Try it for yourself If you really want to, you can now run this proprietary piece of TensorFlow based software to tap into the results of training data from millions of Android user's keystrokes while tapping with your fingers on your postmarketOS phone. You can also use these instructions with the open source Hacker's Keyboard. Build ATL from sources: sudo apk add -t .atl-builddeps \ libopensles ## AI Hallucinations Are Often Reasonable Suggestions DevFeed: [AI Hallucinations Are Often Reasonable Suggestions]() Original publisher: [Read original article]() Author: Graham King Published: 2024-05-26T19:00:00Z Content type: opinion Language: en Sources: [Graham King]() Topics: [Artificial Intelligence](), [Large Language Model]() Tags: [ai](), [ai-hallucination](), [hallucinations](), [llms](), [society](), [software](), [training-data]() ### AI overview The article argues that AI hallucinations are often plausible suggestions produced when language models fill gaps in incomplete or ambiguous input. Using Air Canada's bereavement-fare chatbot incident and the Age of the Captain problem, it suggests these responses should be taken seriously as ideas that may reveal how policies or assumptions could change. ### Source excerpt Don't dismiss the statistically probable ## Cappy: Outperforming and boosting large multi-task language models with a small scorer DevFeed: [Cappy: Outperforming and boosting large multi-task language models with a small scorer]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-03-14T19:38:00Z Content type: article Language: en Sources: [Google Research]() Topics: [Large Language Model](), [Natural language processing](), [data](), [Google]() Tags: [google](), [language](), [language-models](), [large-language-models](), [machine-learning](), [model](), [models](), [natural-language-processing](), [neurips](), [software](), [training-data]() ### AI overview This Google Research article introduces the motivation for Cappy, a small scorer intended to help adapt and improve large multi-task language models. It describes how instruction-following LLMs generalize across tasks, while noting that their size creates substantial computational, memory, storage, and adaptation challenges. ### Source excerpt Posted by Yun Zhu and Lijuan Liu, Software Engineers, Google Research Large language model (LLM) advancements have led to a new paradigm that unifies various natural language processing (NLP) tasks within an instruction-following framework. This paradigm is exemplified by recent multi-task LLMs, such as T0, FLAN, and OPT-IML. First, multi-task data is gathered with each task following a task-specific template, where each labeled example is converted into an instruction (e.g., "Put the concepts together to form a sentence: ski, mountain, skier") paired with a corresponding response (e.g., "Skier skis down the mountain"). These instruction-response pairs are used to train the LLM, resulting in a conditional generation model that takes an instruction as input and generates a response. Moreover, multi-task LLMs have exhibited remarkable task-wise generalization capabilities as they can address unseen tasks by understanding and solving brand-new instructions. The demonstration of the instruction-following pre-training of multi-task LLMs, e.g., FLAN. Pre-training tasks under this paradigm improves the performance for unseen tasks. Due to the complexity of understanding and solving various tasks solely using instructions, the size of multi-task LLMs typically spans from several billion parameters to hundreds of billions (e.g., FLAN-11B, T0-11B and OPT-IML-175B). As a result, operating such sizable models poses significant challenges because they demand considerable computational power and impose substantial requirements on the memory capacities of GPUs and TPUs, making their training and inference expensive and inefficient. Extensive storage is required to maintain a unique LLM copy for each downstream task. Moreover, the most powerful multi-task LLMs (e.g., FLAN-PaLM-540B) are closed-sourced, making them impossible to be adapted. However, in practical applications, harnessing a single multi-task LLM to manage all conceivable tasks in a zero-shot manner remains difficult, ## Learning the importance of training data under concept drift DevFeed: [Learning the importance of training data under concept drift]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-02-14T18:32:00Z Content type: article Language: en Sources: [Google Research]() Topics: [Artificial Intelligence](), [Training AI Models](), [Benchmark](), [datasets](), [data](), [Google]() Tags: [ai](), [ai-models](), [benchmark](), [continual-learning](), [datasets](), [deep-learning](), [performance](), [research](), [supervised-learning](), [training](), [training-data]() ### AI overview Google Research describes a method for handling slow concept drift in nonstationary learning by assigning training instances importance scores based on their content and age. An auxiliary model jointly learned with the primary model produces the scores, and the authors report up to 15% relative accuracy gains on a large benchmark dataset. ### Source excerpt Posted by Nishant Jain, Pre-doctoral Researcher, and Pradeep Shenoy, Research Scientist, Google Research The constantly changing nature of the world around us poses a significant challenge for the development of AI models. Often, models are trained on longitudinal data with the hope that the training data used will accurately represent inputs the model may receive in the future. More generally, the default assumption that all training data are equally relevant often breaks in practice. For example, the figure below shows images from the CLEAR nonstationary learning benchmark, and it illustrates how visual features of objects evolve significantly over a 10 year span (a phenomenon we refer to as slow concept drift), posing a challenge for object categorization models. Sample images from the CLEAR benchmark. (Adapted from Lin et al.) Alternative approaches, such as online and continual learning, repeatedly update a model with small amounts of recent data in order to keep it current. This implicitly prioritizes recent data, as the learnings from past data are gradually erased by subsequent updates. However in the real world, different kinds of information lose relevance at different rates, so there are two key issues: 1) By design they focus exclusively on the most recent data and lose any signal from older data that is erased. 2) Contributions from data instances decay uniformly over time irrespective of the contents of the data. In our recent work, "Instance-Conditional Timescales of Decay for Non-Stationary Learning", we propose to assign each instance an importance score during training in order to maximize model performance on future data. To accomplish this, we employ an auxiliary model that produces these scores using the training instance as well as its age. This model is jointly learned with the primary model. We address both the above challenges and achieve significant gains over other robust learning methods on a range of benchmark datasets for nonstationary ## Five Threat Model Diagrams for Machine Learning DevFeed: [Five Threat Model Diagrams for Machine Learning]() Original publisher: [Read original article]() Author: Adam Published: 2023-04-13T00:00:00Z Content type: opinion Language: en Sources: [Shostack & Friends Blog]() Topics: [Machine Learning, Security Attacks](), [Machine Learning & Artificial Intelligence](), [Training AI Models](), [Data Quality](), [data]() Tags: [ai-and-cybersecurity](), [cybersecurity](), [data-quality](), [diagram](), [machine-learning](), [training-data]() ### AI overview This article presents five threat-model diagrams for machine learning systems. The diagrams distinguish threats to and from ML systems, illustrate data flows and responses, and examine how training-data sources and system-design decisions can create risks. The author emphasizes that the diagrams are illustrative rather than complete and notes uncertainty about the model of how Twitter content reached Microsoft's Tay. ### Source excerpt Some diagrams to help clarify machine learning threats ## Using Deep Learning to Recognize Handwritten Characters with OCR DevFeed: [Using Deep Learning to Recognize Handwritten Characters with OCR]() Original publisher: [Read original article]() Published: 2023-03-30T10:00:08Z Content type: tutorial Language: en Sources: [Bert Hubert's writings]() Topics: [Deep learning](), [Image processing](), [Neural Network](), [Training AI Models](), [datasets](), [Code]() Tags: [code](), [deep-learning](), [image-processing](), [neural-network](), [ocr](), [testing](), [training](), [training-data]() ### AI overview This tutorial builds and tests an OCR program that recognizes handwritten letters from a photo using a convolutional deep learning model. It shows how differences between real-world input and training data can cause recognition failures, including difficulty recognizing a handwritten "h". ### Source excerpt This page is part of the Hello Deep Learning series of blog posts. You are very welcome to improve this page via GitHub! The previous chapters have often mentioned the chasm between "deep learning models that work on my data" and "it actually works in the real world". It is perhaps for this reason that almost all demos and YouTube tutorials you find online never do any real world testing. ## Hello Deep Learning: Dropout, data augmentation, weight decay and quantisation DevFeed: [Hello Deep Learning: Dropout, data augmentation, weight decay and quantisation]() Original publisher: [Read original article]() Published: 2023-03-30T10:00:07Z Content type: tutorial Language: en Sources: [Bert Hubert's writings]() Topics: [Deep learning](), [data augmentation](), [datasets](), [Image]() Tags: [data-augmentation](), [deep-learning](), [image](), [performance](), [training](), [training-data](), [validation]() ### AI overview This Hello Deep Learning tutorial examines how data augmentation and related techniques can improve the robustness of a convolutional handwritten-digit recognition network. It explains that exposing the network to shifted, corrupted, or otherwise modified inputs helps it handle real-world variation and can enlarge a limited training set. ### Source excerpt This page is part of the Hello Deep Learning series of blog posts. You are very welcome to improve this page via GitHub! In the previous chapter we found ways to speed up our character recognition learning by a factor of 20 by using a better optimizer, and a further factor of four by cleverly using threads using a 'shared nothing architecture'. We also learned how we can observe the development of parameters. ## Experts in-the-Loop at Stitch Fix DevFeed: [Experts in-the-Loop at Stitch Fix]() Original publisher: [Read original article]() Published: 2022-09-02T09:00:00Z Content type: article Language: en Sources: [Stitch Fix]() Topics: [personalization](), [recommendations](), [datasets](), [Training AI Models]() Tags: [customer](), [datasets](), [e-commerce](), [personalization](), [recommendations](), [training](), [training-data]() ### AI overview Stitch Fix describes how human stylists help create high-quality training data for personalized outfit recommendations in its Freestyle direct-shopping experience. The article explains why stylist judgments are needed to assess recommendation quality and client experience at scale. ### Source excerpt Imagine your job is to personalize search results on an e-commerce site for returning customers, classify the presence or absence of pedestrians in street photos, or develop an app that translates languages. In all of these cases, a basic ingredient is a dataset of annotations provided by a human. For any company seeking to personalize experience for its customers, combining human computation with algorithmic computation is essential. This is also true for Stitch Fix. At Stitch Fix, we recently launched Stitch Fix Freestyle, our direct-shopping experience, where our algorithmic recommendations are now directly shared with clients in their own personal shopping feed - a different approach from our original Fix experience, where a team of expert stylists determined what should go in the client's Fix. Central to the Freestyle experience for clients is showing individual items as part of complete outfits, where both items and outfits are personalized based on our clients unique size, fit, style, and price preferences. But, what makes a good outfit? And, how do we balance personalization to a customer with a given level of outfit quality or a particular stylistic slant? In order to bootstrap a new product like this, to offer personalized outfit recommendations at scale to our around 4m clients, we need some high-quality training data. In particular, we need data specific to what it means to have a good client experience - or what makes a good outfit for a given client at Stitch Fix. Datasets like this aren't exactly floating around: assembling them requires an intentional, large-scale effort. The best way to get high-quality data sets is to work with our in-house experts: our stylists. In this post, you'll learn more about the purpose and impact of stylist-in-the-loop projects, and the powerful impact that our stylists have in building the future of personalized shopping. Bringing in our "experts in-the-loop" Working with our expert stylists helps us improve our clients' ## What is Machine Learning? DevFeed: [What is Machine Learning?]() Original publisher: [Read original article]() Author: Fuzzygroup Published: 2020-01-14T00:00:00Z Content type: tutorial Language: en Sources: [Scott Johnson]() Topics: [Machine learning](), [Algorithms](), [data](), [Training AI Models]() Tags: [algorithm](), [data](), [explainability](), [machine-learning](), [neural-network](), [training-data]() ### AI overview This tutorial defines machine learning as a technology and as the study of algorithms and statistical models that learn from labeled training data to perform tasks or make predictions without explicit instructions. It explains the roles of corpora, labeling, training data, data-driven pattern recognition, and neural networks. ### Source excerpt I have written a number of posts about different aspects of Machine Learning but, it occurs to me, not a single one that actually says "Machine Learning is xyz and here are the definitions" so here goes. Machine Learning is a technology that allows a computer ("the Machine") to take a "corpus" of "labeled" "training data" and then "learn" how to do something that the training data teaches the computer. So the above is my definition but let's look at Wikipedia: Machine learning (ML) is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying on patterns and inference instead. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to perform the task.[1][2]:2 Machine learning algorithms are used in a wide variety of applications, such as email filtering and computer vision, where it is difficult or infeasible to develop a conventional algorithm for effectively performing the task.Wikipedia And, while it is a simplification, the underlying thing idea driving machine learning is that of data driven pattern recognition. Basically the Machine Learning process amounts to: Take a bunch of data and construct a set of examples that illustrate what you want the machine to learn. This is called Training Data and the overall collection amounts to a Corpus. Add metadata to these examples in a way that represent what you want the machine to learn. Examples could be blocks of text, images, video files or whatever. I personally work mostly in text but I have machine learning buddies who do things learning on images of cancerous masses. This is called Labeling. Labeling might consist of different things like adding Yes / No / Abstain values to a tweet or even extracting information from a recipe like the ingredie ## Neural Networks and Deep Learning DevFeed: [Neural Networks and Deep Learning]() Original publisher: [Read original article]() Published: 2017-11-09T00:00:00Z Content type: tutorial Language: en Sources: [Vasya Drobushkov]() Topics: [Deep neural networks](), [Neural Network](), [Deep learning](), [Mathematics](), [data](), [NumPy]() Tags: [arrays](), [cpu](), [deep-learning](), [learning](), [mathematics](), [matrix](), [neural-networks](), [training-data]() ### AI overview A concise synopsis of a Coursera course on neural networks and deep learning, covering supervised learning, training data, iterative prediction and correction, and vectorization. It emphasizes that array and matrix operations can be faster than explicit loops, while noting that the computational details are not fully understood by the author. ### Source excerpt Disclaimer It is brief synopsis of Neural Networks and Deep learning course on Coursera. That course is a first part of Deep learning specialization. Though course is great and very useful, I found that it has a lot of mathematics explanation (linear algebra -- matrices, derivatives etc.), so sometimes it was difficult to wait for the actual useful information I didn't know. So, I think I might be interested in reviewing neural networks in the future, but won't be able to review course information as there will be a lot of such noise. So, I decided to briefly put everything into one page, so I will be able to quickly review this topic in the future. There will be no code, just formulas as they were explained in a course with some notes on how I understood it. ## The Boosting Margin, or Why Boosting Doesn't Overfit DevFeed: [The Boosting Margin, or Why Boosting Doesn't Overfit]() Original publisher: [Read original article]() Published: 2015-09-21T11:33:00Z Content type: article Language: en Sources: [Jeremy Kun]() Topics: [machine learning overfitting](), [Machine learning](), [generalization in machine learning](), [Occam's razor machine learning](), [Algorithm]() Tags: [algorithm](), [boosting](), [chernoff-bound](), [classficiation](), [complexity](), [conditional-probability](), [error](), [machine-learning](), [margins](), [mathematics](), [model](), [occam-s-razor](), [overfitting](), [training-data](), [vc-dimension]() ### AI overview This article explains why boosting can continue improving generalization after reaching zero training error. It introduces the margin-based theoretical explanation for this behavior and defines the confidence and margin of AdaBoost classifiers. ### Source excerpt There's a well-understood phenomenon in machine learning called overfitting. The idea is best shown by a graph: overfitting Let me explain. The vertical axis represents the error of a hypothesis. The horizontal axis represents the complexity of the hypothesis. The blue curve represents the error of a machine learning algorithm's output on its training data, and the red curve represents the generalization of that hypothesis to the real world. The overfitting phenomenon is marker in the middle of the graph, before which the training error and generalization error both go down, but after which the training error continues to fall while the generalization error rises.