# information retrieval Published articles for information retrieval. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Your agent wants to search like a 2010 quant DevFeed: [Your agent wants to search like a 2010 quant]() Original publisher: [Read original article]() Author: Jon Bratseth Published: 2026-07-07T00:00:00Z Content type: opinion Language: en Sources: [Vespa Blog]() Topics: [AI search](), [information retrieval](), [Language models](), [Google Search]() Tags: [agents](), [ai](), [ai-agents](), [bm25](), [genai](), [google-search](), [hybrid-search](), [information-retrieval](), [rag](), [ranking](), [search]() ### AI overview The article argues that AI agents should retrieve information with more control and sophistication than ordinary human search users. It describes a progression from vector retrieval to hybrid search using methods such as BM25 and machine-learned ranking, and presents search as code as a possible next stage. ### Source excerpt The idea of empowering AI agents to retrieve information like a professional is going mainstream. ## Building Reliable Agentic AI Systems DevFeed: [Building Reliable Agentic AI Systems]() Original publisher: [Read original article]() Author: Martin Fowler (martin@martinfowler.com) Published: 2026-06-16T12:11:00Z Content type: article Language: en Sources: [Martin Fowler]() Topics: [Artificial Intelligence](), [Retrieval Augmented Generation (RAG)](), [text2sql](), [AI search](), [Generative AI](), [AI-generated research reports](), [data](), [Orchestration](), [observability](), [Prompt Engineering]() Tags: [agentic-ai](), [ai](), [building](), [case-study](), [data](), [drug-discovery](), [efficiency](), [engineering](), [generative-ai](), [information-retrieval](), [language-models](), [llms](), [orchestration](), [production](), [rag](), [research](), [retrieval](), [safety](), [sql](), [technical]() ### AI overview This case study describes PRINCE, a cloud-hosted platform developed by Bayer AG with Thoughtworks for pharmaceutical research. It combines Agentic Retrieval-Augmented Generation and Text-to-SQL to help researchers query decades of safety study reports, answer complex questions, and draft regulatory documents. The article focuses on context engineering, orchestration, recovery, observability, transparency, explainability, human oversight, governance, and compliance in production-ready agentic AI systems. ### Source excerpt One of the most interesting projects my colleagues have done with LLMs has been building a system with Bayer to allow pharmaceutical researchers to query decades of information about studies buried in PDF reports. Sarang Sanjay Kulkarni describes its evolution from keyword-based search to an intelligent research assistant capable of answering complex questions and drafting regulatory documents. more... ## Re-autoresearching MSMARCO BM25, on Vespa DevFeed: [Re-autoresearching MSMARCO BM25, on Vespa]() Original publisher: [Read original article]() Author: andreer thomas Published: 2026-05-29T00:00:00Z Content type: article Language: en Sources: [Vespa Blog]() Topics: [Benchmark](), [dataset](), [Python](), [Large Language Model](), [generalization in machine learning](), [pandas](), [Google Search]() Tags: [benchmark](), [bm25](), [dataset](), [evaluation](), [experiment](), [google-search](), [information-retrieval](), [openai](), [pandas](), [python](), [ranking](), [retrieval](), [zero-shot]() ### AI overview This article reproduces an MSMARCO BM25 autoresearch experiment in Vespa. It compares LLM-driven Python reranking with an approach restricted to existing Vespa rank features and reports a comparable improvement on a 650,000-passage subset, with better generalization to the full dataset. ### Source excerpt BM25 is having a moment. We reproduce Doug Turnbull's MSMARCO autoresearch experiment in Vespa and get a comparable MRR@10 lift from existing rank features -- with twice the generalization to full MSMARCO. ## Scaling a Vespa Application: Feeding Fast and Furiously DevFeed: [Scaling a Vespa Application: Feeding Fast and Furiously]() Original publisher: [Read original article]() Author: Kai Borgen Published: 2026-04-28T00:00:00Z Content type: tutorial Language: en Sources: [Vespa Blog]() Topics: [AI search](), [Deployment](), [datasets](), [information retrieval](), [Cloud](), [Homebrew](), [Command-line interface](), [onnx](), [optimum](), [XML]() Tags: [ai](), [ai-search](), [cli](), [cloud](), [dataset](), [genai](), [how-to](), [information-retrieval](), [install](), [metrics](), [onnx](), [optimum](), [performance](), [rag](), [ranking](), [tensors](), [tutorial]() ### AI overview This tutorial demonstrates how to scale a Vespa application while feeding the full MS_marco passages dataset. It covers preparing the dataset, configuring access, deploying a sample application, and using scaling and metrics to improve feed throughput and performance. ### Source excerpt A tutorial on how to scale the resources in a Vespa application to increase feed throughput. Using the metrics dashboard for informed and optimised scaling. ## Modularizing SOLR Query Creation for Multi-Market Scale DevFeed: [Modularizing SOLR Query Creation for Multi-Market Scale]() Original publisher: [Read original article]() Author: Naman Parikh Published: 2026-03-03T12:18:55Z Content type: tutorial Language: en Sources: [Walmart Global Tech]() Topics: [Code](), [configuration](), [implementation](), [Testing](), [debug]() Tags: [article](), [code](), [complexity](), [configuration](), [developer](), [errors](), [exceptions](), [implementation](), [information-retrieval](), [modular-monolith](), [monolithic-architecture](), [regression](), [software-architecture](), [software-engineering](), [technical](), [testing]() ### AI overview This article describes modularizing a 12,000-line SOLR query creation implementation used across multiple markets. It explains how separating query concerns, adding market-specific configuration, and using stronger typing aimed to reduce coupling, regression risk, and runtime errors. ### Source excerpt Introduction When the SOLR query logic expanded into a 12,000-line monolithic implementation, each modification introduced significant risk, making every change feel akin to defusing a critical system. Adding a market-specific override required yet another if block, compounding complexity and slowing time-to-market. In this article, we will deep dive how we broke that SOLR query creation logic, enabling clean configuration per market, stronger typing, reducing technical debts and dramatically reduced runtime errors. Image generated with DALL-E via ChatGPTThe Problem: When SOLR Queries Creation Logic Become Technical Debt The SOLR query logic class was handling filtering logic, boosting logic, boost functions, pagination etc. All the parameters related to SOLR query was getting generated using single class. Overthe time, this core class handled various logics related to different type of queries: Primary search queries Item insertions via business tools Item insertions via semantic sources Thousands of lines tangled edge-case handling, scoring tweaks, and boosting logic. This unscalable approach: Blocked rapid iteration for new markets Tight Coupling: All query-handling logic lived in one massive class, making it difficult to cleanly separate concerns. Market-specific changes could unintentionally affect unrelated logic, requiring exhaustive regression testing. High Risk of Unintended Consequences: Changing business requirements (such as supporting different filtering or boosting strategies for a new market) entailed changing existing code that already served other markets. Developers had to be extremely cautious, as a bug or oversight could break unrelated functionality. No Configuration Flexibility: There was no clear system for externalizing market-specific configuration. Instead, all logic changes happened directly in code, preventing business users or product managers from making simple market changes without developer intervention. Increased Runtime exceptions ## Now Available: Anthropic Claude Opus 4.6 on DigitalOcean's Agentic Inference Cloud DevFeed: [Now Available: Anthropic Claude Opus 4.6 on DigitalOcean's Agentic Inference Cloud]() Original publisher: [Read original article]() Author: DigitalOcean Published: 2026-02-06T19:38:29Z Content type: release Language: en Sources: [DigitalOcean]() Topics: [Anthropic Claude](), [Digital Ocean](), [Inference](), [Serverless](), [AI Platform](), [long-context](), [agentic-coding](), [AI search](), [Automation](), [Development]() Tags: [agentic-coding](), [ai](), [ai-platform](), [automation](), [claude](), [development](), [digitalocean](), [inference](), [information-retrieval](), [long-context](), [product-updates](), [serverless](), [software-development]() ### AI overview DigitalOcean announces that Anthropic's Claude Opus 4.6 is available on the Gradient AI Platform through Serverless Inference. The article describes access through the API or DigitalOcean Cloud Console and highlights capabilities including a 1M-token context, adaptive reasoning, agentic coding, data analysis, information retrieval, and office productivity tasks. ### Source excerpt Claude Opus 4.6 is now available on the DigitalOcean Gradient™ AI Platform via Serverless Inference--giving teams access to Anthropic's most capable model on a platform built to run inference reliably at scale. Start using the new model now, via the API or in the DigitalOcean Cloud Console. With massive 1M-token context, adaptive reasoning, and advanced agentic coding, Claude Opus 4.6 enables teams to analyze huge datasets, refactor entire codebases, and generate high-quality outputs in a single pass. It's also optimized for everyday knowledge work, including reports, spreadsheets, and presentations. What Opus 4.6 unlocks Agentic coding & software development: Plan, debug, and iterate across large codebases; perform root cause analysis; handle multilingual coding and cybersecurity tasks. Knowledge work & research: Analyze financial data, run research, and manage multi-step tasks in documents, spreadsheets, and presentations. Agentic automation: Coordinate multiple AI agents for parallel, read-heavy, or long-running tasks; summarize large contexts and make adaptive reasoning decisions. Information retrieval & long-context reasoning: Retrieve hard-to-find details across vast datasets and reason over hundreds of thousands of tokens. Office productivity: Generate structured reports, spreadsheets, and presentation decks; ingest unstructured data and produce polished outputs in one pass. Why Run Opus 4.6 on DigitalOcean Claude Opus 4.6 runs natively inside your existing DigitalOcean environment--alongside your applications, data, networking, and storage--so inference becomes part of your stack, not another system to integrate or operate. There are no separate model contracts, vendor accounts, or billing surfaces to manage. Usage is billed predictably alongside your other DigitalOcean services, with inference managed by default so you can start running Opus 4.6 quickly without provisioning or tuning infrastructure. Safe defaults are built in from the start. Opus 4.6 runs with ## Silver lining playbook: Likely China-origin activity targeting US persons DevFeed: [Silver lining playbook: Likely China-origin activity targeting US persons]() Original publisher: [Read original article]() Published: 2026-02-01T00:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [OpenAI](), [ChatGPT](), [Social engineering](), [email](), [information retrieval]() Tags: [chatgpt](), [china](), [email](), [information-retrieval](), [openai](), [report](), [research](), [safety](), [social-engineering](), [us]() ### AI overview OpenAI describes banning a small set of likely China-origin ChatGPT accounts that used its models to research US persons and federal locations, draft social-engineering emails, and request guidance related to face-manipulation software. The activity was named "Silver Lining Playbook." ### Source excerpt OpenAI banned likely China-origin accounts using AI to research US persons, locations, and social-engineering tactics. ## Building world-class product search at Shopify: Where C++ excellence meets ML innovation DevFeed: [Building world-class product search at Shopify: Where C++ excellence meets ML innovation]() Original publisher: [Read original article]() Author: Mikhail Shakhray Published: 2025-11-12T16:21:44Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [C++](), [AI Development](), [AI search](), [Latency](), [Large Language Model](), [Inference](), [Framework]() Tags: [apis](), [architecture](), [c-plus-plus](), [engineering](), [evaluation](), [experimentation](), [information-retrieval](), [latency](), [machine-learning](), [ml](), [models](), [performance](), [pipeline](), [ranking](), [real-time](), [retrieval](), [scale](), [search](), [speed](), [systems]() ### AI overview Shopify describes a commerce search platform that combines classical information retrieval with transformers, neural rankers, and gradient boosting while maintaining native C++ speed and millisecond-scale latency. Its RankFlow DSL and TurboDSL execution engine support rapid model deployment, real-time indexing, retrieval, ranking, experimentation, and relevance customization at global commerce scale. ### Source excerpt Learn how we solved a major search engineering dilemma--running machine learning models at native C++ speed. ## Baidu и AI Search Paradigm: мультиагентная структура для интеллектуального поиска информации DevFeed: [Baidu и AI Search Paradigm: мультиагентная структура для интеллектуального поиска информации]() Original publisher: [Read original article]() Author: redmadrobot (red\_mad\_robot) Published: 2025-10-14T18:36:48Z Content type: article Language: ru Sources: [Redmadrobot EN](), [Redmadrobot RU]() Topics: [AI search](), [Artificial Intelligence](), [Large Language Model](), [information retrieval](), [Retrieval Augmented Generation (RAG)]() Tags: [ai](), [ai-search](), [executor](), [information-retrieval](), [llm](), [rag](), [search](), [tag-d89cae10e887](), [tag-e1c0c11ccefc]() ### AI overview The article analyzes Baidu's AI Search Paradigm, a proposed multi-agent architecture for intelligent search built on large language models. It explains how specialized agents coordinate planning, tool use, execution, and answer writing, and contrasts this approach with classical information retrieval, semantic search, learning-to-rank, and RAG systems. ### Source excerpt Аналитический центр red_mad_robot продолжает разбирать ключевые исследования в сфере интеллектуальных систем и генеративного поиска. На этот раз рассказываем про архитектуру AI Search Paradigm от Baidu -- новой системы интеллектуального поиска, построенной на LLM и мультиагентных методах. Читать далее ## Scaling AI-based Data Processing with Hugging Face + Dask DevFeed: [Scaling AI-based Data Processing with Hugging Face + Dask]() Original publisher: [Read original article]() Author: Sarah Johnson; James Bourbeau; Quentin Lhoest; Daniel van Strien Published: 2024-10-09T00:00:00Z Content type: tutorial Language: en Sources: [Hugging Face - Blog]() Topics: [data-processing](), [datasets](), [Apache Spark](), [parquet](), [pandas](), [GPU](), [anthropic](), [Inference](), [Python]() Tags: [data-processing](), [datasets](), [gpu](), [guide](), [hugging-face](), [inference](), [information-retrieval](), [nlp](), [parquet](), [python](), [scale]() ### AI overview This tutorial demonstrates scaling AI-based data processing with Hugging Face and Dask. It processes the FineWeb dataset locally with pandas, then scales text classification to 211 million rows across multiple cloud GPUs using Dask DataFrame and Parquet files. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Ep. 5: Key Techniques for Accurate AI-Driven Information Retrieval DevFeed: [Ep. 5: Key Techniques for Accurate AI-Driven Information Retrieval]() Original publisher: [Read original article]() Published: 2024-08-22T00:00:00Z Content type: tutorial Language: en Sources: [William Kennedy]() Topics: [information retrieval](), [Embeddings](), [Generative AI](), [Artificial Intelligence](), [context](), [data](), [Databases](), [cohere](), [HTML](), [Website]() Tags: [ai](), [ai-and-web-page-content-extraction](), [ai-driven-information-retrieval](), [ai-models](), [ai-models-and-text-data](), [cohere](), [cohere-api-for-embeddings](), [context](), [context-handling-in-ai](), [cosine-similarity-search](), [data](), [databases](), [effective-ai-information-retrieval](), [embeddings](), [enhancing-ai-model-accuracy](), [generative-ai-techniques](), [guide](), [how-to](), [html](), [information-retrieval](), [lancedb-vector-database](), [large-text-data-processing](), [managing-large-scale-text-data-in-ai](), [open-source](), [semantic-search-optimization](), [vector-embeddings-in-ai](), [vectorization-techniques]() ### AI overview Episode 5 of an introductory Generative AI series explains techniques for retrieving relevant information from large text collections. It covers converting web content to Markdown, splitting text into overlapping chunks to preserve context, generating vector embeddings with Cohere's API, storing embeddings in vector databases such as LanceDB, and using cosine similarity for semantic search. ### Source excerpt Introduction: Welcome to Episode 5 of our Intro to Generative AI series! In this episode, Daniel explores practical techniques for enhancing AI models' ability to handle large volumes of text data effectively. He addresses the challenges developers face when working with extensive content, such as entire web pages or internal documents, and provides actionable strategies to optimize the retrieval and processing of relevant information. Context Handling: Splitting large text into manageable chunks while preserving context. Vectorization Techniques: Converting text chunks into vector representations for semantic search. Semantic Search: Implementing cosine similarity to retrieve relevant information efficiently. ## Category Prediction for Search Query Understanding DevFeed: [Category Prediction for Search Query Understanding]() Original publisher: [Read original article]() Author: music and waves Published: 2024-04-21T12:40:48Z Content type: article Language: en Sources: [Myntra]() Topics: [Query (disambiguation)](), [Training AI Models](), [datasets](), [data]() Tags: [ai](), [classification](), [data](), [deep-learning](), [ecommerce](), [information-retrieval](), [model](), [neural](), [nlp](), [search](), [text-classification](), [train](), [training]() ### AI overview This article describes Myntra's multi-label product-category classification model for understanding ambiguous search queries. It covers preparing query-and-category training data and training a neural text classifier to predict relevant categories for live searches. ### Source excerpt Navigating through online shopping platforms can sometimes feel like finding your way through a maze. Take the search bar, for example. You type in "winter upper wear," hoping to find the perfect jacket or cozy sweatshirt. But here's the tricky part: the search engine has to decipher what you mean. Is it jackets you're after? Or maybe sweatshirts? Or both? It gets even more confusing when you consider the overlapping categories. Kurtas can be standalone articles or part of kurta sets. And loafers? They could belong to formal shoes or casual shoes and certainly not sports shoes. See the challenge? To tackle this, Myntra uses a multi-label search to product category classification model. It's like having an assistant that can understand possible intents from your search query. So when you type in something like "whey," the model knows you might be looking for protein or health supplements. But here's the catch: search queries can be short and vague, and they often use words that don't directly match category names. People might search using different terms or even regional variations. So, the model needs to be clever enough to map those words to the right categories internally. The goal is to capture all possible intents without cluttering your search results with irrelevant stuff. After all, nobody likes sifting through pages of irrelevant products. It's a delicate balance between covering all bases and keeping things tidy. Solution The solution has 2 major components. I. Data preparation We prepare ( search query : categories ) data points to be consumed in training by the neural classifier. Ex. ( ethnic wear : kurta, sarees ) II. Training a neural model We train a neural multi-label text classifier that consumes the prepared training data which is used to predicts categories for search queries live. I. Data Preparation We generate the supervised text classification training data in form of a search query and its product category(s) as its labels. This set is enrich ## pgvector: Fewer dimensions are better DevFeed: [pgvector: Fewer dimensions are better]() Original publisher: [Read original article]() Author: Greg Richardson; Oliver Rice; Egor Romanov Published: 2023-08-03T07:00:00Z Content type: article Language: en Sources: [Supabase Blog]() Topics: [Embeddings](), [Supabase](), [AI search](), [hugging face]() Tags: [compute](), [embedding](), [hardware](), [index](), [information-retrieval](), [openai](), [performance](), [pgvector](), [provisioning](), [vectors]() ### AI overview This article explains why using embedding vectors with fewer dimensions can improve pgvector performance. It covers vector storage in Postgres, similarity indexes, memory and compute requirements, and the scaling challenges of high-dimensional embeddings. ### Source excerpt Increase performance in pgvector by using embedding vectors with fewer dimensions ## The power of ChatGPT as a cognitive accessibility assistive technology for Traumatic Brain Injury survivors DevFeed: [The power of ChatGPT as a cognitive accessibility assistive technology for Traumatic Brain Injury survivors]() Original publisher: [Read original article]() Author: Jamie Vanderwall Published: 2023-03-14T00:00:00Z Content type: article Language: en Sources: [The A11Y Project]() Topics: [Accessibility](), [ChatGPT](), [Large Language Model](), [Artificial Intelligence](), [OpenAI](), [Deep learning](), [Algorithms]() Tags: [accessibility](), [algorithms](), [article](), [artificial-intelligence](), [chatgpt](), [communication](), [deep-learning](), [information-retrieval](), [large-language-model](), [memory](), [openai](), [training]() ### AI overview This article examines ChatGPT as a cognitive accessibility assistive technology for traumatic brain injury survivors. It describes potential uses including information retrieval, memory support, communication assistance, and cognitive training, and considers the broader impact on the disability community. ### Source excerpt This article was written by ChatGPT with the below prompt written by Jamie Vanderwall. "Write about how ChatGPT can be used as a cognitive accessibility assistive technology specifically for traumatic brain injury survivors and how this impacts the disability community." Introduction This essay explores the use of ChatGPT as a cognitive accessibility assistive technology for traumatic brain injury (TBI) survivors, discussing the impact it has on the disability community. The essay outlines the cognitive impairments experienced by TBI survivors, and how ChatGPT can provide support in the form of information retrieval, memory aids, communication aids, and cognitive training. Traumatic brain injury (TBI) is a life-changing injury that can affect people of all ages, genders, and backgrounds. TBI can cause a range of physical, cognitive, and emotional impairments, making it difficult for people with TBI to perform daily activities, engage in social interactions, and access information. Cognitive accessibility assistive technology, such as ChatGPT, can help TBI survivors overcome some of the cognitive challenges they face and improve their quality of life. This essay will explore how ChatGPT can be used as a cognitive accessibility assistive technology specifically for TBI survivors and how it impacts the disability community. What is ChatGPT? ChatGPT is a large language model developed by OpenAI that can generate human-like responses to natural language queries. It is a type of artificial intelligence that uses deep learning algorithms to analyze and understand natural language input and generate a relevant and contextually appropriate response. ChatGPT can be used in various contexts, such as customer service, chatbots, and conversational agents. How can ChatGPT be used as a cognitive accessibility assistive technology for TBI survivors? TBI can cause a range of cognitive impairments, including memory loss, attention deficits, and difficulty with executive functioning. ## The Emitter Parameter Pattern for Flexible SPI Contracts DevFeed: [The Emitter Parameter Pattern for Flexible SPI Contracts]() Original publisher: [Read original article]() Published: 2020-05-04T15:30:00Z Content type: tutorial Language: en Sources: [Gunnar Morling]() Topics: [Frameworks](), [Code](), [Programming](), [information retrieval](), [Machine learning]() Tags: [algorithms](), [api](), [code](), [developer](), [framework](), [frameworks](), [information-retrieval](), [interfaces](), [libraries](), [machine-learning](), [spi]() ### AI overview This article explains the emitter parameter pattern as an alternative to return values in service provider interfaces. It discusses how emitter parameters can avoid multiple passes over input, reduce unnecessary allocations, and make SPI contracts easier to evolve when new extracted attributes are added. ### Source excerpt Table of Contents An Example The Emitter Parameter Pattern For libraries and frameworks it's a common requirement to make specific aspects customizeable via service provider interfaces (SPIs): contracts to be implemented by the application developer, which then are invoked by framework code, adding new or replacing existing functionality. Often times, the method implementations of such an SPI need to return value(s) to the framework. An alternative to return values are "emitter parameters": passed by the framework to the SPI method, they offer an API for receiving value(s) via method calls. Certainly not revolutionary or even a new idea, I find myself using emitter parameters more and more in libraries and frameworks I work on. Hence I'd like to discuss some advantages I perceive about the emitter parameter pattern.