# Self-supervised learning Published articles for Self-supervised learning. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## GeoPT helps AI models simulate how objects respond to physical forces DevFeed: [GeoPT helps AI models simulate how objects respond to physical forces]() Original publisher: [Read original article]() Author: Alex Shipps | MIT CSAIL Published: 2026-08-10T19:25:00Z Content type: news Language: en Sources: [MIT AI News]() Topics: [AI Models](), [Simulation](), [AI Research](), [Computer Science and Artificial Intelligence Laboratory (CSAIL)](), [Robotics]() Tags: [3-d-imaging](), [ai](), [ai-models](), [artificial-intelligence](), [computational-fluid-dynamics-cfd](), [computer-graphics](), [computer-modeling](), [computer-science-and-artificial-intelligence-laboratory-csail](), [computer-science-and-technology](), [crash-simulation](), [design](), [electrical-engineering-and-computer-science-eecs](), [geometric-pre-training](), [geopt](), [haixu-wu](), [human-computer-interaction](), [kaiming-he](), [manufacturing](), [minghao-guo](), [mit-csail](), [mit-eecs](), [mit-schwarzman-college-of-computing](), [neural-physics-simulation](), [paper](), [physics](), [physics-aware-ai](), [physics-foundation-models](), [research](), [robotics](), [school-of-engineering](), [self-supervised-learning](), [simulation](), [surrogate-modeling](), [synthetic-dynamics](), [transformer-based-simulators](), [wojciech-matusik]() ### AI overview Researchers at MIT CSAIL and Tsinghua University developed GeoPT, a pre-training approach that uses 3D simulations of mechanical interactions to help AI models learn physics more efficiently. The article reports that models using the approach reached peak performance twice as fast and trained on up to 60 percent less data than leading models. ### Source excerpt "GeoPT" helps AI models understand the basics of physics so they can simulate how objects respond to things like wind and water more efficiently and accurately. ## How controllers from industrial machinery can coordinate multitask machine learning DevFeed: [How controllers from industrial machinery can coordinate multitask machine learning]() Original publisher: [Read original article]() Author: Theodore Vasiloudis Published: 2026-07-30T17:26:47Z Content type: article Language: en Sources: [Amazon Science homepage]() Topics: [Machine Learning & Artificial Intelligence](), [Graphs](), [Training AI Models](), [Neural Network](), [data](), [Algorithm](), [systems]() Tags: [data](), [machine-learning](), [model](), [multitask-learning](), [research](), [self-supervised-learning](), [train](), [training]() ### AI overview ControlG addresses conflicting objectives in graph self-supervised learning by allocating computational capacity to one objective at a time and using a proportional-integral-derivative controller to select which objective receives attention next. ### Source excerpt Instead of compromising among parameter updates dictated by different training objectives, ControlG allocates computational capacity to objectives sequentially and dynamically. ## Improving Embedding-Based Candidate Generation for Recommender Systems with a Two-Tower Model DevFeed: [Improving Embedding-Based Candidate Generation for Recommender Systems with a Two-Tower Model]() Original publisher: [Read original article]() Author: Ahmad Khan Published: 2026-03-18T21:04:58Z Content type: article Language: en Sources: [Glassdoor Engineering]() Topics: [recommendation systems](), [Machine learning](), [Embeddings](), [Neural Network](), [data](), [ordering]() Tags: [ai](), [data](), [embedding](), [machine-learning](), [model-training](), [neural-network](), [ordering](), [ranking](), [recommender-systems](), [self-supervised-learning](), [two-tower-model]() ### AI overview Glassdoor describes a two-tower neural network for candidate generation in its Community feed recommendation system. The model learns user and post embeddings from interaction data to retrieve potentially relevant posts before filtering, ranking, and ordering. ### Source excerpt Overview At Glassdoor (part of Indeed), millions of professionals come to learn about workplaces, ask career questions, and share experiences through our Community platform. When users open the Community Home Feed, they expect to see posts that are relevant to their interests, industry, and career stage. However, generating that feed is not a simple task. At any moment, thousands of new posts across hundreds of community feeds compete for attention. From discussions about interview experiences to workplace culture questions and industry advice, the platform contains a diverse and constantly evolving stream of content. The challenge is to quickly identify which of these posts are most relevant for each user. To address this problem, we built a multi-stage machine learning recommendation system that efficiently sources, filters, and ranks posts before they appear in the user's feed. In a previous post, Inside Glassdoor's Multi-Stage Recommendation System, we described the architecture of this system and how the different stages work together to deliver personalized content. Snapshot of the Glassdoor Community Home Feed At a high level, the system consists of four stages: Candidate Generation -- retrieves a large set of potentially relevant posts Filtering -- removes posts that are undesirable, duplicated, or already seen Ranking -- predicts which posts a user is most likely to engage with Ordering -- applies heuristics to maintain diversity and freshness in the feed Each stage progressively narrows the set of posts until only the most relevant posts remain. 4 Stage ML RecSys Design In this blog post, we focus on the Candidate Generation stage, which is responsible for retrieving an initial pool of potentially relevant posts from a very large pool of posts across many different feeds. For Candidate Generation, we developed a Two-Tower Neural Network Model that learns embeddings for users and posts based on real interaction data. In the rest of this blog post, we will walk ## ScreenAI: A visual language model for UI and visually-situated language understanding DevFeed: [ScreenAI: A visual language model for UI and visually-situated language understanding]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-03-19T20:15:00Z Content type: release Language: en Sources: [Google Research]() Topics: [vlm](), [ui](), [Google](), [datasets](), [multimodal](), [Transformer]() Tags: [datasets](), [google](), [hci](), [model](), [multimodal](), [multimodal-learning](), [self-supervised-learning](), [ui](), [vision]() ### AI overview Google Research introduces ScreenAI, a vision-language model designed to understand, reason about, and interact with user interfaces and infographics. The article describes its PaLI-based multimodal architecture, pix2struct-inspired flexible patching strategy, training approach, reported benchmark results, and three released datasets for screen understanding and question answering. ### Source excerpt Posted by Srinivas Sunkara and Gilles Baechler, Software Engineers, Google Research Screen user interfaces (UIs) and infographics, such as charts, diagrams and tables, play important roles in human communication and human-machine interaction as they facilitate rich and interactive user experiences. UIs and infographics share similar design principles and visual language (e.g., icons and layouts), that offer an opportunity to build a single model that can understand, reason, and interact with these interfaces. However, because of their complexity and varied presentation formats, infographics and UIs present a unique modeling challenge. To that end, we introduce "ScreenAI: A Vision-Language Model for UI and Infographics Understanding". ScreenAI improves upon the PaLI architecture with the flexible patching strategy from pix2struct. We train ScreenAI on a unique mixture of datasets and tasks, including a novel Screen Annotation task that requires the model to identify UI element information (i.e., type, location and description) on a screen. These text annotations provide large language models (LLMs) with screen descriptions, enabling them to automatically generate question-answering (QA), UI navigation, and summarization training datasets at scale. At only 5B parameters, ScreenAI achieves state-of-the-art results on UI- and infographic-based tasks (WebSRC and MoTIF), and best-in-class performance on Chart QA, DocVQA, and InfographicVQA compared to models of similar size. We are also releasing three new datasets: Screen Annotation to evaluate the layout understanding capability of the model, as well as ScreenQA Short and Complex ScreenQA for a more comprehensive evaluation of its QA capability. ScreenAI ScreenAI's architecture is based on PaLI, composed of a multimodal encoder block and an autoregressive decoder. The PaLI encoder uses a vision transformer (ViT) that creates image embeddings and a multimodal encoder that takes the concatenation of the image and text emb ## LinkBERT: Improving Language Model Training with Document Link DevFeed: [LinkBERT: Improving Language Model Training with Document Link]() Original publisher: [Read original article]() Author: A Href; Michihiro Yasunaga; Jure Leskovec; Percy Liang Published: 2022-05-31T07:00:00Z Content type: article Language: en Sources: [The Stanford AI Lab Blog]() Topics: [Large Language Model](), [Natural language processing](), [Training AI Models](), [Self-supervised learning](), [Web](), [data](), [Fine-tuning]() Tags: [data](), [google](), [language-models](), [model](), [model-training](), [natural-language-processing](), [self-supervised-learning](), [training](), [web]() ### AI overview The article introduces LinkBERT in the context of language-model pretraining. It explains how BERT and GPT use self-supervised objectives to learn from large text corpora, then describes a limitation of treating documents independently: hyperlinks and citation links connect knowledge across documents and can support multi-hop understanding. The supplied text ends while introducing this challenge. ### Source excerpt Language Model Pretraining Language models (LMs), like BERT 1 and the GPT series 2, achieve remarkable performance on many natural language processing (NLP) tasks. They are now the foundation of today's NLP systems. 3 These models serve important roles in products and tools that we use every day, such as search engines like Google 4 and personal assistants like Alexa 5. These LMs are powerful because they can be pretrained via self-supervised learning on massive amounts of text data on the web without the need for labels, after which the pretrained models can be quickly adapted to a wide range of new tasks without much task-specific finetuning. For instance, BERT is pretrained to predict randomly masked words in original text (masked language modeling), e.g. predicting the masked word "dog" from "My __ is fetching the ball". GPTs are pretrained to predict the next word given a previous sequence of text (causal language modeling), e.g. predicting the next word "ball" from "My dog is fetching the". In either cases, through pretraining, LMs learn to encode various knowledge from a text corpus that helps to perform downstream applications involving language understanding or generation. In particular, LMs can learn world knowledge (associations between concepts like "dog", "fetch", "ball") from training text where the concepts appear together, and help for knowledge-intensive applications like question answering. 6 Challenges. A challenge with most common LM pretraining strategies is that they model a single document at a time. That is, one would split a text corpus into a list of documents and draw training instances for LMs from each document independently. Treating each document independently may pose limitations because documents often have rich dependencies with each other. For instance, text from the web 7 or scientific literature 8 is often used for LM training, but they all have document links, such as hyperlinks and citation links. Document links are crucial bec