# Representation learning

Representation learning is the process of learning compact, structured data representations that preserve essential information and make downstream tasks easier.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Understanding Deep Learning Algorithms that Leverage Unlabeled Data, Part 1: Self-training

DevFeed: [Understanding Deep Learning Algorithms that Leverage Unlabeled Data, Part 1: Self-training](<https://devfeed.tech/articles/understanding-deep-learning-algorithms-that-leverage-unlabeled-data-part-1-self-training-7590.md>)

Original publisher: [Read original article](<https://ai.stanford.edu/blog/understanding-self-training/>)

Author: A Href; Colin Wei; Jeff Z HaoChen; Tengyu Ma

Published: 2022-02-24T08:00:00Z

Content type: article

Language: en

Sources: [The Stanford AI Lab Blog](<https://devfeed.tech/sources/the-stanford-ai-lab-blog.md>)

Topics: [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [image](<https://devfeed.tech/tags/image.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [predictions](<https://devfeed.tech/tags/predictions.md>), [research](<https://devfeed.tech/tags/research.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This article explains self-training, a semi-supervised learning method that uses a classifier to generate pseudo-labels for unlabeled data and then retrains a new classifier on those labels. It introduces the method and presents a theoretical analysis of why retraining can improve accuracy over the original pseudo-labeler.

### Source excerpt

Deep models require a lot of training examples, but labeled data is difficult to obtain. This motivates an important line of research on leveraging unlabeled data, which is often more readily available. For example, large quantities of unlabeled image data can be obtained by crawling the web, whereas labeled datasets such as ImageNet require expensive labeling procedures. In recent empirical developments, models trained with unlabeled data have begun to approach fully-supervised performance (e.g., Chen et al., 2020, Sohn et al., 2020). This series of blog posts will discuss our theoretical work which seeks to analyze recent empirical methods which use unlabeled data. In this first post, we'll analyze self-training, which is a very impactful algorithmic paradigm for semi-supervised learning and domain adaptation. In Part 2, we will use related theoretical ideas to analyze self-supervised contrastive learning algorithms, which have been very effective for unsupervised representation learning. Background: self-training We will first provide a basic overview of self-training algorithms, which are the main focus of this blog post. The core idea is to use some pre-existing classifier \(F_{pl}\) (referred to as the "pseudo-labeler") to make predictions (referred to as "pseudo-labels") on a large unlabeled dataset, and then retrain a new model with the pseudo-labels. For example, in semi-supervised learning, the pseudo-labeler is obtained from training on a small labeled dataset, and is then used to predict pseudo-labels on a larger unlabeled dataset. A new classifier \(F\) is then retrained from scratch to fit the pseudo-labels, using additional regularization. In practice, \(F\) will often be more accurate than the original pseudo-labeler \(F_{pl}\) (Lee 2013). The self-training procedure is depicted below. It is quite surprising that self-training can work so well in practice, given that we retrain on our own predictions, i.e. the pseudo-labels, but not the true labels.

## Stanford AI Lab Papers at ICCV 2021

DevFeed: [Stanford AI Lab Papers at ICCV 2021](<https://devfeed.tech/articles/stanford-ai-lab-papers-at-iccv-2021-7583.md>)

Original publisher: [Read original article](<https://ai.stanford.edu/blog/iccv-2021/>)

Author: Compiled by Drew A. Hudson

Published: 2021-10-08T07:00:00Z

Content type: article

Language: en

Sources: [The Stanford AI Lab Blog](<https://devfeed.tech/sources/the-stanford-ai-lab-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [object-detection](<https://devfeed.tech/topics/object-detection.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [ai](<https://devfeed.tech/tags/ai.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [fairness-accountability-transparency](<https://devfeed.tech/tags/fairness-accountability-transparency.md>), [learning](<https://devfeed.tech/tags/learning.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [object-detection](<https://devfeed.tech/tags/object-detection.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [video](<https://devfeed.tech/tags/video.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Stanford AI Lab presents its accepted work for ICCV 2021, including research on multimodal medical image recognition, self-supervised learning, diffusion-based shape generation, object pose tracking, human-object relationships in videos, 3D human motion, computational photography, and related vision topics.

### Source excerpt

The International Conference on Computer Vision (ICCV 2021) will be hosted virtually next week. We're excited to share all the work from SAIL that will be presented, and you'll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that's happening at Stanford! List of Accepted Papers GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition Authors: Mars Huang Contact: mschuang@stanford.edu Keywords: medical image, self-supervised learning, multimodal fusion 3D Shape Generation and Completion Through Point-Voxel Diffusion Authors: Linqi Zhou, Yilun Du, Jiajun Wu Contact: linqizhou@stanford.edu Links: Paper | Video | Website Keywords: diffusion, shape generation CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds Authors: Yijia Weng*, He Wang*, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas Contact: yijiaw@stanford.edu Award nominations: Oral Presentation Links: Paper | Video | Website Keywords: category-level object pose tracking, articulated objects Detecting Human-Object Relationships in Videos Authors: Jingwei Ji, Rishi Desai, Juan Carlos Niebles Contact: jingweij@cs.stanford.edu Links: Paper Keywords: human-object relationships, video, detection, transformer, spatio-temporal reasoning Geography-Aware Self-Supervised Learning Authors: Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tanmay, Marshall Burke, David Lobell, Stefano Ermon Contact: kayush@cs.stanford.edu, chenlin@stanford.edu Links: Paper | Website Keywords: self-supervised learning, contrastive learning, remote sensing, spatio-temporal, classification, object detection, segmentation HuMoR: 3D Human Motion Model for Robust Pose Estimation Authors: Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, Leonidas Guibas Contact: drempe@stanford.edu Award nominations: Oral Presentati