# bert

Published articles for bert.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart

DevFeed: [From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart](<https://devfeed.tech/articles/from-scoring-to-spelling-rebuilding-ads-retrieval-at-instacart-20105.md>)

Original publisher: [Read original article](<https://tech.instacart.com/from-scoring-to-spelling-rebuilding-ads-retrieval-at-instacart-cf36b4e8d1bb?source=rss----587883b5d2ee---4>)

Author: Karuna Ahuja

Published: 2026-06-02T18:50:19Z

Content type: article

Language: en

Sources: [Instacart](<https://devfeed.tech/sources/instacart.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [recommendations](<https://devfeed.tech/topics/recommendations.md>), [systems](<https://devfeed.tech/topics/systems.md>), [context](<https://devfeed.tech/topics/context.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [bert](<https://devfeed.tech/tags/bert.md>), [context](<https://devfeed.tech/tags/context.md>), [generative](<https://devfeed.tech/tags/generative.md>), [inference](<https://devfeed.tech/tags/inference.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [recommendation-system](<https://devfeed.tech/tags/recommendation-system.md>), [recommendations](<https://devfeed.tech/tags/recommendations.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>)

### AI overview

Instacart describes rebuilding its ads retrieval system by moving from a product-scoring encoder to a generative model that predicts product IDs token by token. The system uses contextual shopping sessions and a BERT-like transformer to retrieve sponsored and organic recommendations.

### Source excerpt

Key Contributors: Karuna Ahuja, Marko Avdalovic, Soroush Sobhkhiz, Shrikar Archak, Xiyu Wang, Ji Chao Zhang, Hao Yan Introduction Every time a user opens Instacart, they see product recommendations: on the retailer home page, in search results, and alongside their cart. Many of these recommendations are sponsored products surfaced by a retrieval model that decides which products to show from a vast ads product catalog. A relevant ad helps users discover products they didn't know they needed; a less relevant one generates friction. Two years ago, we introduced Contextual Recommendations (CR), a BERT-based sequence model powering retrieval for both ads and organic recommendations across all major browse surfaces. In this post, we'll focus on our ads retrieval. We will detail how we rebuilt the system, by moving from an encoder that scores products to a generative model that spells them out, token by token. By doing so, we unlocked a new level of contextual matching -- ensuring brands appear exactly when users want them, while simultaneously opening up discovery of thousands of relevant products the previous system couldn't retrieve. Contextual Recommendations: A recap At its core, CR treats grocery shopping as a language modeling task, where atomic product IDs function as tokens and, the finite subset of the catalog it is trained on, acts as its 'vocabulary'. The model leverages the user's real-time session, which includes product views, item page visits, and cart additions, as a sequence of these product tokens. A BERT-like transformer is then trained on millions of authentic shopping sessions to predict the next token (i.e. singular product) in the sequence. This process allows the model to learn and capture complex purchasing patterns, such as the tendency for users who add pasta and olive oil to frequently add garlic next. This single retrieval layer replaced multiple ad-hoc systems and powers recommendation carousels across all major browse surfaces, serving both

## Explore LLM word representations using similarity analysis (part 1)

DevFeed: [Explore LLM word representations using similarity analysis (part 1)](<https://devfeed.tech/articles/explore-llm-word-representations-using-similarity-analysis-part-1-38804.md>)

Original publisher: [Read original article](<https://thepalindrome.org/p/explore-llm-word-representations>)

Author: Mike X Cohen, PhD

Published: 2026-04-22T12:43:43Z

Content type: tutorial

Language: en

Sources: [The Palindrome](<https://devfeed.tech/sources/the-palindrome.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [bert](<https://devfeed.tech/topics/bert.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [bert](<https://devfeed.tech/tags/bert.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [introduction](<https://devfeed.tech/tags/introduction.md>), [llm](<https://devfeed.tech/tags/llm.md>), [using](<https://devfeed.tech/tags/using.md>)

### AI overview

A hands-on introduction to representational similarity analysis using GPT-2 and BERT embeddings to examine word representations in large language models.

### Source excerpt

A hands-on introduction to representational similarity analysis (RSA) with GPT-2 and BERT embeddings

## Qwen3Guard: следующий шаг в модерации и контроле контента

DevFeed: [Qwen3Guard: следующий шаг в модерации и контроле контента](<https://devfeed.tech/articles/qwen3guard-24031.md>)

Original publisher: [Read original article](<https://habr.com/ru/companies/redmadrobot/articles/971388/>)

Author: Martianov (red\_mad\_robot)

Published: 2025-11-28T15:10:54Z

Content type: article

Language: ru

Sources: [Redmadrobot EN](<https://devfeed.tech/sources/redmadrobot-en.md>), [Redmadrobot RU](<https://devfeed.tech/sources/redmadrobot-ru.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [bert](<https://devfeed.tech/tags/bert.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [llm](<https://devfeed.tech/tags/llm.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [rnd](<https://devfeed.tech/tags/rnd.md>), [tag-75a722387627](<https://devfeed.tech/tags/tag-75a722387627.md>), [tag-75dea8ea04f0](<https://devfeed.tech/tags/tag-75dea8ea04f0.md>), [tag-a6ed1cca9b15](<https://devfeed.tech/tags/tag-a6ed1cca9b15.md>)

### AI overview

The article discusses building content moderation for open-text services. It compares using an LLM as a moderator with an embedding-based classifier trained on 40,000 labeled query examples, reporting latency of about 20 ms per request instead of 700-900 ms, while noting that the simpler model struggles with context, irony, hints, and jailbreaks.

### Source excerpt

Всем привет! Меня зовут Миша Мартьянов, я инженер по исследованиям и разработке в лаборатории AI R&D в red_mad_robot. В мои задачи входит проверка гипотез и развитие наших продуктов. Однако недостаточно просто улучшать продукты, необходимо также чтобы они работали устойчиво и безопасно. Ранее я рассказывал разработку идеального контент-фильтра на базе Guardrails. Но время не стоит на месте: появляются новые модели и новые практики их применения. Этому и будет посвящён наш сегодняшний разговор. Читать далее

## Ettin Suite: SoTA Paired Encoders and Decoders

DevFeed: [Ettin Suite: SoTA Paired Encoders and Decoders](<https://devfeed.tech/articles/ettin-suite-sota-paired-encoders-and-decoders-7185.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ettin>)

Author: Orion Weller; K Ricci; Marc Marone; Antoine Chaffin; Dawn Lawrie; Ben Van Durme

Published: 2025-07-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine Learning & Artificial Intelligence](<https://devfeed.tech/topics/machine-learning-artificial-intelligence.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [datasets](<https://devfeed.tech/topics/datasets.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [bert](<https://devfeed.tech/tags/bert.md>), [community](<https://devfeed.tech/tags/community.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model-development](<https://devfeed.tech/tags/model-development.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [open-source-collab](<https://devfeed.tech/tags/open-source-collab.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [research](<https://devfeed.tech/tags/research.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The article introduces Ettin, a suite of paired encoder-only and decoder-only language models ranging from 17M to 1B parameters. The models are trained with identical data, architectures, and recipes, enabling controlled comparisons between masked and causal language modeling. Ettin reports state-of-the-art performance for open-data models and explores converting models between encoder and decoder architectures.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Автоматическая многоязычная модерация сообщений для миллионов пользователей

DevFeed: [Автоматическая многоязычная модерация сообщений для миллионов пользователей](<https://devfeed.tech/articles/article-23645.md>)

Original publisher: [Read original article](<https://habr.com/ru/companies/badoo/articles/648815/>)

Author: punkerpunker (Badoo)

Published: 2022-02-01T10:50:28Z

Content type: article

Language: ru

Sources: [Badoo EN](<https://devfeed.tech/sources/badoo-en.md>), [Badoo RU](<https://devfeed.tech/sources/badoo-ru.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [badoo](<https://devfeed.tech/tags/badoo.md>), [bert](<https://devfeed.tech/tags/bert.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [gpt-3](<https://devfeed.tech/tags/gpt-3.md>), [natural-language-processing](<https://devfeed.tech/tags/natural-language-processing.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [tag-055aee430837](<https://devfeed.tech/tags/tag-055aee430837.md>), [tag-13e2af703774](<https://devfeed.tech/tags/tag-13e2af703774.md>), [tag-1cd610c0e518](<https://devfeed.tech/tags/tag-1cd610c0e518.md>), [tag-5cb789e9aaa3](<https://devfeed.tech/tags/tag-5cb789e9aaa3.md>), [tag-9d8cf70dc46c](<https://devfeed.tech/tags/tag-9d8cf70dc46c.md>), [tag-e69b694e3ce2](<https://devfeed.tech/tags/tag-e69b694e3ce2.md>)

### AI overview

The article describes Badoo and Bumble's Rude Message Detector, a machine-learning system for multilingual moderation of rude messages. It discusses the system's architecture, deployment infrastructure, implementation challenges, and results, including the use of transformer-based architectures and large language models such as BERT, GPT-2, and GPT-3.

### Source excerpt

Как мы все знаем, слова обладают силой. Ими можно вдохновить людей, но можно и навредить. Мы в Badoo и Bumble стараемся оградить пользователей от неприятных ситуаций, поэтому внедрили инструмент Rude Message Detector. Это многоязычный детектор грубых высказываний, работающий на основе машинного обучения. В этой статье мы расскажем о технических особенностях нашего решения, сложностях, с которыми столкнулись в ходе реализации проекта, и его результатах. Читать далее

## 2019 Retrospective

DevFeed: [2019 Retrospective](<https://devfeed.tech/articles/2019-retrospective-40113.md>)

Original publisher: [Read original article](<https://korbonits.com/blog/2020-01-01-retrospective-on-2019/>)

Published: 2020-01-01T00:00:00Z

Content type: opinion

Language: en

Sources: [Alex Korbonits](<https://devfeed.tech/sources/alex-korbonits.md>)

Topics: [Learning](<https://devfeed.tech/topics/learning.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>), [Hugo](<https://devfeed.tech/topics/hugo.md>), [bert](<https://devfeed.tech/topics/bert.md>), [math](<https://devfeed.tech/topics/math.md>)

Tags: [bert](<https://devfeed.tech/tags/bert.md>), [books](<https://devfeed.tech/tags/books.md>), [hugo](<https://devfeed.tech/tags/hugo.md>), [japan](<https://devfeed.tech/tags/japan.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math](<https://devfeed.tech/tags/math.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [poetry](<https://devfeed.tech/tags/poetry.md>), [retrospective](<https://devfeed.tech/tags/retrospective.md>)

### AI overview

A personal retrospective on 2019 covering sailing, travel, reading, writing, language learning, fitness, organization, technology use, and relationships. The author reached a goal of reading 50 books, explored NLP papers including BERT, and attended fiction courses at Hugo House.

### Source excerpt

Looking back on 2019 -- sailing, travel to Japan, books, and the best surprise of the year.