# Supervised Learning Published articles for Supervised Learning. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Learning the importance of training data under concept drift DevFeed: [Learning the importance of training data under concept drift]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-02-14T18:32:00Z Content type: article Language: en Sources: [Google Research]() Topics: [Artificial Intelligence](), [Training AI Models](), [Benchmark](), [datasets](), [data](), [Google]() Tags: [ai](), [ai-models](), [benchmark](), [continual-learning](), [datasets](), [deep-learning](), [performance](), [research](), [supervised-learning](), [training](), [training-data]() ### AI overview Google Research describes a method for handling slow concept drift in nonstationary learning by assigning training instances importance scores based on their content and age. An auxiliary model jointly learned with the primary model produces the scores, and the authors report up to 15% relative accuracy gains on a large benchmark dataset. ### Source excerpt Posted by Nishant Jain, Pre-doctoral Researcher, and Pradeep Shenoy, Research Scientist, Google Research The constantly changing nature of the world around us poses a significant challenge for the development of AI models. Often, models are trained on longitudinal data with the hope that the training data used will accurately represent inputs the model may receive in the future. More generally, the default assumption that all training data are equally relevant often breaks in practice. For example, the figure below shows images from the CLEAR nonstationary learning benchmark, and it illustrates how visual features of objects evolve significantly over a 10 year span (a phenomenon we refer to as slow concept drift), posing a challenge for object categorization models. Sample images from the CLEAR benchmark. (Adapted from Lin et al.) Alternative approaches, such as online and continual learning, repeatedly update a model with small amounts of recent data in order to keep it current. This implicitly prioritizes recent data, as the learnings from past data are gradually erased by subsequent updates. However in the real world, different kinds of information lose relevance at different rates, so there are two key issues: 1) By design they focus exclusively on the most recent data and lose any signal from older data that is erased. 2) Contributions from data instances decay uniformly over time irrespective of the contents of the data. In our recent work, "Instance-Conditional Timescales of Decay for Non-Stationary Learning", we propose to assign each instance an importance score during training in order to maximize model performance on future data. To accomplish this, we employ an auxiliary model that produces these scores using the training instance as well as its age. This model is jointly learned with the primary model. We address both the above challenges and achieve significant gains over other robust learning methods on a range of benchmark datasets for nonstationary ## Intervening on early readouts for mitigating spurious features and simplicity bias DevFeed: [Intervening on early readouts for mitigating spurious features and simplicity bias]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-02-02T17:49:00Z Content type: article Language: en Sources: [Google Research]() Topics: [AI Research](), [Deep learning](), [responsible-ai](), [generalization in machine learning](), [Model Development](), [Training AI Models]() Tags: [bias](), [dataset](), [deep-learning](), [icml](), [machine-learning](), [ml-fairness](), [research](), [responsible-ai](), [supervised-learning](), [training]() ### AI overview Google Research describes methods for detecting and reducing spurious features and simplicity bias in deep learning models. Early readouts expose confidently wrong predictions associated with spurious features, while feature forgetting helps models identify more predictive features and generalize to unseen domains. ### Source excerpt Posted by Rishabh Tiwari, Pre-doctoral Researcher, and Pradeep Shenoy, Research Scientist, Google Research Machine learning models in the real world are often trained on limited data that may contain unintended statistical biases. For example, in the CELEBA celebrity image dataset, a disproportionate number of female celebrities have blond hair, leading to classifiers incorrectly predicting "blond" as the hair color for most female faces -- here, gender is a spurious feature for predicting hair color. Such unfair biases could have significant consequences in critical applications such as medical diagnosis. Surprisingly, recent work has also discovered an inherent tendency of deep networks to amplify such statistical biases, through the so-called simplicity bias of deep learning. This bias is the tendency of deep networks to identify weakly predictive features early in the training, and continue to anchor on these features, failing to identify more complex and potentially more accurate features. With the above in mind, we propose simple and effective fixes to this dual challenge of spurious features and simplicity bias by applying early readouts and feature forgetting. First, in "Using Early Readouts to Mediate Featural Bias in Distillation", we show that making predictions from early layers of a deep network (referred to as "early readouts") can automatically signal issues with the quality of the learned representations. In particular, these predictions are more often wrong, and more confidently wrong, when the network is relying on spurious features. We use this erroneous confidence to improve outcomes in model distillation, a setting where a larger "teacher" model guides the training of a smaller "student" model. Then in "Overcoming Simplicity Bias in Deep Networks using a Feature Sieve", we intervene directly on these indicator signals by making the network "forget" the problematic features and consequently look for better, more predictive features. This substanti ## Adding Explainability to Machine Learning Routines via Runtime Execution of Labeling Functions DevFeed: [Adding Explainability to Machine Learning Routines via Runtime Execution of Labeling Functions]() Original publisher: [Read original article]() Author: Fuzzygroup Published: 2020-01-10T00:00:00Z Content type: article Language: en Sources: [Scott Johnson]() Topics: [Machine Learning & Artificial Intelligence](), [Neural Network](), [data](), [Kafka]() Tags: [data](), [explainability](), [kafka](), [machine-learning](), [neural-network](), [python](), [snorkel](), [supervised-learning](), [training]() ### AI overview The article discusses the black-box nature of machine-learning outputs and introduces runtime execution of labeling functions as an approach to adding explainability. It also explains basic concepts including supervised learning, labeled datasets, neural networks, training programs, and models. ### Source excerpt I am spending much of my time these days in the world of machine learning and I'm finding that it is profoundly interesting. One of the most interesting aspects of machine learning, for me, is the black box nature of machine learning's output. What we typically get from a machine learning model what the model thinks but not how it arrived at that result. And this lack of explainability is true for machine learning whether the underlying technology is Tensor Flow for image segmentation or Bert / Roberta / Albert for text processing. Note: Bert / Roberta / Albert are machine learning tools that generate linguistically savvy models that understand English language content. As a side note, given how hard it is for most people to understand why they think what they think, is the black box nature of machine learning actually all that surprising? Machine Learning Basics for the Newbie The world of machine learning, which appears intimidating as hell to the newbie, actually isn't that hard but the terms can be terrifying. Someone recently asked me what I was doing and this was my answer: "I am working on a weekly supervised learning model for an anti Semitic classifier that will be used in a Kafka data pipeline. " Here are the very basics of machine learning: Machine Learning is just that - a way for a machine to learn from data. The underlying idea of machine learning is pretty much just pattern recognition based on data fed through a neural network. To make machine learning work you "label" a data set which means you tell the machine that for Example X, the answer you want is Foo and for Example Y, the answer is Bar. And you do this over and over and over (sometimes you have to label 50,000 examples; that's what my current project uses). You give a program, generally called a training program, this labeled data and then it thinks on this for a while and creates what is called a model. The model is a trained neural network which understands how to process data like what yo ## Deep Probabilistic Modelling with Gaussian Processes #NIPS2017 DevFeed: [Deep Probabilistic Modelling with Gaussian Processes #NIPS2017]() Original publisher: [Read original article]() Published: 2017-12-04T12:00:00Z Content type: tutorial Language: en Sources: [Alex Korbonits]() Topics: [Tutorial](), [VAE](), [Machine learning](), [Deep learning](), [Neural Network](), [NeurIPS]() Tags: [analysis](), [bias](), [conference](), [data](), [deep-learning](), [gaussian](), [inference](), [modelling](), [neural-network](), [neurips](), [probabilistic](), [research](), [supervised-learning](), [theory](), [tutorial](), [unsupervised-learning](), [videos]() ### AI overview Lecture notes from a NeurIPS 2017 tutorial introduce deep probabilistic modelling with Gaussian processes, covering probabilistic neural networks, uncertainty, graphical models, and the computational challenge of inference. ### Source excerpt Lecture notes from Neil Lawrence's NIPS 2017 tutorial on deep probabilistic modelling with Gaussian processes -- from GPs to deep GPs and variational inference. ## Using search to guess job categories in FINN småjobber DevFeed: [Using search to guess job categories in FINN småjobber]() Original publisher: [Read original article]() Author: Tor Arne Kvaløy Published: 2017-02-02T14:14:30Z Content type: tutorial Language: en Sources: [Finn.no]() Topics: [Machine learning](), [datasets](), [Forms](), [Development]() Tags: [data](), [forms](), [machine-learning](), [search](), [supervised-learning](), [user](), [using]() ### AI overview The article explains how FINN småjobber explored automatically guessing job categories from user-submitted titles. Although supervised machine learning was considered, the team used existing indexed job data and normal search to rank likely categories. ### Source excerpt The problem Making it easy to fill out forms is essential for getting data into our system at FINN småjobber, which is a service to get help with tasks like house cleaning or renovation, where the user submits a job that non-professionals bid on. Submitting a job is done by filling out a form with a title, a short description, and selecting the category the job belongs to. The category hierarchy has two levels: a main category and a subcategory. For example the main category domestic help, has the subcategories cleaning and babysitting. There are 6 main categories and 26 subcategories, and our assumption was that it was a hassle for the user to go through this long list. We wanted to simplify this by guessing the category. Ideally on the given title. The solution Machine learning A typical solution to this kind of problem is to use machine learning, where the goal is to classify job titles into categories. We had a large set of historical data (130 000 registered jobs) where category already was selected by the user. This was a good starting point for a type of machine learning technique called supervised learning. In supervised learning historical data is used as a training set, and results in a classifier that can classify job titles into categories. Delving into the field of machine learning is quite a complex task, and requires both theoretical understanding and knowhow of various software libraries. And in many cases, it requires access to a special machine learning infrastructure. So, as a lean product development team with limited resources we looked for an easier solution. Good old search Search was a sentral part of our service. All the jobs were indexed, and we had all the necessary libraries and infrastructure for this. So, what about using normal search to find the category? A simple search on the title "Cleaning kitchen", gave the following ranked results: Match rank Category 1 Renovation 2 Cleaning 3 Cleaning 4 Renovation 5 Cleaning 6 Cleaning 7 Cleani ## K-Nearest-Neighbors and Handwritten Digit Classification DevFeed: [K-Nearest-Neighbors and Handwritten Digit Classification]() Original publisher: [Read original article]() Published: 2012-08-26T12:19:43Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Machine Learning & Artificial Intelligence](), [Data Science](), [Math and Logic]() Tags: [dimension](), [linear-algebra](), [machine-learning](), [mathematics](), [metric](), [process](), [programming](), [supervised-learning]() ### AI overview This tutorial introduces supervised classification in machine learning and explains how labeled data, training algorithms, models, and classification algorithms work together. It then examines the assumption that data points in the same class are close under an appropriate metric. ### Source excerpt The Recipe for Classification One important task in machine learning is to classify data into one of a fixed number of classes. For instance, one might want to discriminate between useful email and unsolicited spam. Or one might wish to determine the species of a beetle based on its physical attributes, such as weight, color, and mandible length. These "attributes" are often called "features" in the world of machine learning, and they often correspond to dimensions when interpreted in the framework of linear algebra.