# Data Science at Microsoft Lessons learned in the practice of data science and AI at Microsoft - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Progressive Authentication for Production-Ready RAG Architectures DevFeed: [Progressive Authentication for Production-Ready RAG Architectures]() Original publisher: [Read original article]() Author: Aishwarya Murali Padikkal Published: 2026-09-08T07:16:00Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [Authentication](), [Retrieval Augmented Generation (RAG)](), [sensitive data](), [User Experience](), [Security]() Tags: [ai](), [ai-agent](), [authentication](), [data-science](), [privacy](), [rag](), [retrieval](), [sensitive-data](), [user-experience]() ### AI overview The article presents progressive authentication for RAG systems: answer public questions without sign-in, then authenticate within the conversation when a request requires access to sensitive personal data. It argues that static, upfront authentication does not fit conversations that shift between public and private contexts. ### Source excerpt Designing a production-ready RAG architecture where authentication becomes part of the conversation, not a prerequisite for it. By Aishwarya Murali Padikkal & Priyank Solanki From understanding to retrieval, grounding, and response, authentication enters only when the conversation requires access to sensitive data- illustrating the core idea of progressive authentication. Illustration generated using ChatGPT and refined by the authors. Two questions, one help line Imagine two taxpayers opening the same AI-powered tax assistant. The first asks, "When is the filing deadline this year?" The second asks, "What was my refund last year?" Both questions are valid. Both deserve fast, helpful answers. Yet they represent fundamentally different types of requests. Figure 1. The same assistant, two different journeys. Requests are routed according to the type of information being accessed. Illustration generated by AI (ChatGPT) and refined by the authors. The answer to the first lives in a public handbook. Anyone should be able to access it instantly- no sign-in, no friction. The answer to the second is within an individual's private tax records. Before revealing a single detail, the assistant must be certain it is interacting with the right person. Making public information instantly accessible while ensuring private information is revealed only to the right person, all within the same conversation, is the challenge we set out to solve. It is also where conventional RAG systems quietly fall apart. Traditional RAG architectures assume a much simpler interaction model: a user asks a question, the system retrieves relevant documents, and the model generates an answer. Authentication, if it exists at all, is treated as a one-time checkpoint at the start of the interaction. Real conversations are far less predictable. Users naturally shift contexts, starting with a public question, moving to a personal one, and then asking follow-up questions that rely on both. The conversation evo ## A Model Portfolio for cost-efficient AI across the software development lifecycle DevFeed: [A Model Portfolio for cost-efficient AI across the software development lifecycle]() Original publisher: [Read original article]() Author: Praveen Sidda Published: 2026-09-01T07:16:01Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [AI Development](), [AI Models](), [Model Routing](), [sdlc](), [Agile](), [Developer experience](), [Compression](), [Requirements](), [Testing](), [Deployment](), [Code review]() Tags: [agentic-sdlc](), [ai](), [ai-cost-optimization](), [ai-models](), [code-review](), [compression](), [deployment](), [github-copilot](), [llm](), [overhead](), [requirements](), [software-development]() ### AI overview The article examines whether routing software development tasks across a portfolio of AI models can reduce costs compared with using one premium model. It reports that lower-cost models handled well-defined tasks, while premium models were reserved for architecture, implementation, and code review; context compression reduced token usage but risked losing important technical details. ### Source excerpt Image generated by AIWhat one controlled experiment taught me about matching model capability to developer work Topic: Can intelligently routing developer tasks across different AI models outperform relying on a single premium model? In this article, I put that question to the test by mapping software development lifecycle (SDLC) stages to a portfolio of AI models and comparing the outcomes. Motivation As AI becomes embedded throughout the AI-Native Development Lifecycle (AIDLC), an evolution of the traditional Software Development Lifecycle (SDLC), its cost is no longer tied to a single prompt. A single developer task can involve multiple model calls, each carrying source files, conversation history, tool definitions, and generated output. Applying the most capable model to every interaction is straightforward, but it also consumes premium model capacity on tasks that less expensive models can often complete just as effectively. This raises an important question for engineering organizations: How can teams reduce the cost of AI-assisted development without compromising quality, reliability, or the developer experience? My first instinct was to reduce token consumption. Context compression appeared to be the most direct path to lowering inference costs by shortening prompts. Although it reduced token usage, it also introduced risk. Important constraints and technical details could be lost, affecting downstream tasks. Source code, stack traces, and active instructions proved to be especially poor candidates for lossy compression. That experience shifted my focus. The objective was not to process fewer tokens, but to complete developer tasks successfully at a lower overall cost. I then experimented with model allocation. Lower-cost models handled well-defined tasks such as requirements synthesis, planning, routine test generation, deployment artifacts, and final summaries, while premium models were reserved for architecture, implementation, and code review. This appro ## Beyond Relevance: Building a Quality-Aware Retrieval Layer for RAG DevFeed: [Beyond Relevance: Building a Quality-Aware Retrieval Layer for RAG]() Original publisher: [Read original article]() Author: Shay Ben-Elazar Published: 2026-08-18T07:16:01Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [Retrieval Augmented Generation (RAG)](), [Large Language Model](), [Artificial Intelligence]() Tags: [ai](), [data-science](), [education](), [machine-learning](), [microsoft](), [quality](), [rag](), [retrieval]() ### AI overview The article examines how a metadata-aware quality layer could improve educational RAG systems by ranking or filtering retrieved sources for reliability, clarity, evidence, and suitability for teaching. It argues that relevance alone does not ensure educational quality and that filtering weak sources can also reduce context noise, token use, and inference costs. ### Source excerpt How a metadata-aware quality layer can help educational RAG systems retrieve sources that are relevant, reliable, and ready for teaching Shay Ben-Elazar, Principal Applied Data Science Manager, Microsoft Rob Mauceri, Distinguished Engineer, Microsoft A student asks an AI tutor a science question: "Why do flu vaccines need to be updated over time?" A retrieval-augmented generation (RAG) system searches its index, retrieves passages about influenza viruses, vaccine development, immunity, and seasonal outbreaks, then generates an answer grounded in those sources [1]. At first glance, the system appears to have done its job. The passages are relevant, and the answer cites supporting information. But a deeper question remains: were the retrieved sources actually good sources to learn from? Some passages may be outdated, overly technical, weakly supported, or poorly organized for a student audience. Others may provide clearer explanations, stronger evidence, and a more coherent path to understanding. Both may be relevant, yet only one is likely to help a learner build accurate understanding [2]. This distinction matters because relevance alone is not the same as educational quality. Retrieved passages can match the topic while lacking clear definitions, supporting evidence, or an easy-to-follow explanation. In a student-facing product, those differences shape not only what the student learns, but also how much they trust the system. There is also a practical systems reason to care. Web-scale RAG applications operate within limited context windows and real compute budgets. Filtering weak sources earlier, or downranking them before generation, reduces noise, preserves tokens for stronger evidence, and lowers inference costs, compounding across products used by millions of learners [3]. RAG has become one of the most practical ways to make large language model applications more grounded. It can make answers more current, more domain-specific, and easier to connect back to so ## A framework for sequential decisions in daily life and beyond DevFeed: [A framework for sequential decisions in daily life and beyond]() Original publisher: [Read original article]() Author: Nisarg Suthar Published: 2026-08-04T07:16:01Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [decision-making]() Tags: [data-modeling](), [data-science](), [decision-making](), [model](), [operations-research](), [sequential-decision](), [structured](), [supply-chain]() ### AI overview The article introduces a Universal Modeling Framework proposed by Warren Powell for reasoning about sequential decisions under uncertainty and incomplete information. It explains how mathematical modeling can clarify decision problems and support better choices across areas such as finance, healthcare, data-center infrastructure, and supply chains. ### Source excerpt Photo by Sophia Kunkel on Unsplash Decision-making occupies a significant portion of our mental uptime. Whether we work in finance, energy, transportation, healthcare, e-commerce, foreign policy, or global supply chains, we are constantly required to make choices in the presence of uncertainty and incomplete information. As new information arrives, decisions must be revised, refined, and sometimes completely reconsidered. Every decision carries consequences -- some rewarding, others costly. The ability to consistently make better choices is often a defining factor behind successful outcomes. Yet effective decision-making remains notoriously difficult. Describing a problem as "mind-bogglingly complex" is really just a by-product of a failure to think about the problem in a structured way.-- Dr. Warren Powell This article introduces a Universal Modeling Framework for reasoning about sequential decision problems proposed by Dr. Warren Powell, an operations researcher at Princeton University. The same framework can be applied across a remarkably diverse set of problems: buying or selling financial assets, evaluating a new user experience, selecting candidate drugs for clinical trials, investing in data-center infrastructure, or managing large-scale supply chains. Here we shall undertake an approach that focusses on identifying the core elements of a decision-making process. Central to our approach is the creation of a simple mathematical model that eliminates the ambiguity of describing problems in language. Modeling is an art, guided by a mathematical framework, and results in a well-defined problem that we can put on a computer to solve. Even when the ultimate goal is not to automate the decision, the act of modeling itself often leads to deeper understanding and better choices. A dynamic model for sequential ecisions A sequential decision process can be represented as follows: Where: Sₜ is the state variable capturing our state of knowledge at time t. For example, inve ## The grader is the reward: What we learned from reinforcement fine-tuning on Azure Foundry DevFeed: [The grader is the reward: What we learned from reinforcement fine-tuning on Azure Foundry]() Original publisher: [Read original article]() Author: Moid Hassan Published: 2026-07-28T07:16:01Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [Fine-tuning](), [Artificial Intelligence](), [Language models](), [Microsoft]() Tags: [ai](), [azure-foundry](), [data-science](), [data-scientist](), [deep-reinforcement](), [fine-tuning](), [microsoft](), [model](), [reinforcement-learning]() ### AI overview This article describes using reinforcement fine-tuning on Azure Foundry to improve language-model-generated business communications. It explains that prompting a frontier model produced generic drafts with poor context handling, excessive length, repeated questions, and occasional unsupported details, motivating efforts to teach the model what quality means for a specific domain. ### Source excerpt Where this story starts Continue reading on Data Science + AI at Microsoft " ## Modernizing sourcing at scale: How intelligent agents are changing procurement DevFeed: [Modernizing sourcing at scale: How intelligent agents are changing procurement]() Original publisher: [Read original article]() Author: Shreya Peddi Published: 2026-07-21T07:16:01Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [microsoft 365](), [Microsoft]() Tags: [agent](), [agentic-workflow](), [agents](), [generative-ai-finance](), [microsoft](), [microsoft-365](), [procurement](), [procurement-strategy](), [setup](), [structured](), [supply-chain](), [tool]() ### AI overview This article describes how a Microsoft procurement professional built the Supplier Price Interpreter with M365 Chat Agent Builder. The agent uses supplier price sheets stored in a structured Teams channel as a knowledge source and responds to plain-language prompts about the data, aiming to reduce manual comparison work in RFP and RFx processes. ### Source excerpt Image generated by AI Picture this: It's late in the week, an urgent sourcing decision is due, and sitting in your inbox are three separate supplier price sheets -- each formatted differently, each with its own tab structure, its own naming conventions, its own quirks. Your job is to compare them, find the gaps, flag the inconsistencies, and deliver a recommendation. The clock is ticking. This isn't a hypothetical scenario. It happens regularly in procurement teams running large scale RFP and RFx processes. For years, the solution was straightforward: Open each file and compare them line by line. It worked, but it was slow, manual, and far removed from the strategic work that sourcing professionals are hired to do. In a world where business decisions move fast and supplier negotiations are time-sensitive, delays in price review have real consequences. Slower cycles, missed windows, and sourcing teams spending their cognitive energy on extraction instead of analysis. So, we asked the question: What if the extraction part could just happen on its own? Starting small, at the desk The Supplier Price Interpreter didn't start as a grand initiative. It started as a practical frustration. Rupa Shastri, a procurement professional within Microsoft's broader procurement organization, built the agent right at her desk using M365 Chat Agent Builder. No dedicated engineering team, no lengthy development cycle, no new infrastructure. Just a clear problem, an available tool, and the willingness to try something different. Figure 1: M365 Copilot flow and the actionable insights it provides. Working within a large enterprise procurement environment, where supplier bids come in across multiple RFx engagements spanning vendors, scopes, and formats, the repetitive nature of price sheet review was hard to ignore. Every engagement required the same foundational steps. Open the files, read the data, compare the numbers, find what's missing. The methodology never changed. Only the suppliers ## When the bug slips through: How we built an AI feedback loop to strengthen our safety net DevFeed: [When the bug slips through: How we built an AI feedback loop to strengthen our safety net]() Original publisher: [Read original article]() Author: Vasilescu Andreea Published: 2026-07-14T07:16:00Z Content type: article Language: en Sources: [Data Science at Microsoft]() Topics: [Artificial Intelligence](), [bug](), [incident](), [Monitoring](), [Software](), [data](), [Azure]() Tags: [ai](), [analysis](), [artificial-intelligence](), [automation](), [bug](), [data-science](), [engineering](), [errors](), [incident](), [incident-management](), [issue](), [llm](), [monitoring](), [outage](), [production](), [software](), [teams]() ### AI overview The article describes how a large-scale engineering organization built an AI feedback loop to investigate production regressions, compare them with pre-production alerts, and identify gaps in its quality platform. It explains that manual incident sampling was valuable but covered less than 10 percent of incidents, making broader forensic analysis impractical at Azure scale. ### Source excerpt Figure 1: When the bug slips through, or how we built an AI feedback loop to strengthen our safety net. A regression reached production last fall. Not dramatically, no war-room scramble, no cascading outage. It showed up as a cluster of customer-facing errors, traced back through monitoring data to a change that had passed through every pre-production check without triggering a single alert. Somewhere upstream, the safety net had a hole that nobody knew was there. This scenario is not hypothetical. It happens. And when it does, the instinctive response is the same: investigate the incident, fix the issue, and then ask the uncomfortable follow-up question, Why didn't we catch this before it reached production? Answering that question is deceptively hard. A pre-production quality platform generates a constant stream of signals, warnings, and anomalies, many of them noisy, overlapping, or ultimately irrelevant. The regression in question might be buried among production incidents, false signals, and failures the platform was never designed to catch in the first place. While manual sampling and investigation can provide meaningful insight, understanding what was genuinely missed, why it was missed, and what needs to change at platform scale requires the kind of careful forensic analysis that is slow, expert-intensive, and -- at the scale of Azure -- practically impossible to do by hand. This is the problem we set out to solve. A safety net with blind spots Modern large-scale engineering organizations invest heavily in pre-production quality platforms, systems that evaluate software changes before they reach customers. These are systems designed to catch regressions before any code change reaches customers, running controlled experiments, executing targeted tests, monitoring key signals, and alerting engineering teams when something looks wrong. But a safety net is only as good as its coverage. And knowing how good your coverage actually is requires a feedback mechanism: a ## A Career Journey from Cryptanalysis Research to SRE and Chief Editor at Microsoft DevFeed: [A Career Journey from Cryptanalysis Research to SRE and Chief Editor at Microsoft]() Original publisher: [Read original article]() Author: Alexandra Savelieva Published: 2026-07-07T07:16:01Z Content type: opinion Language: en Sources: [Data Science at Microsoft]() Topics: [Data Science](), [Microsoft](), [Cryptography](), [site-reliability-engineering](), [Algorithms, Complexity](), [audit trail]() Tags: [ai](), [algorithms](), [audit-trail](), [careers](), [crack](), [data-science](), [logs](), [machine-learning](), [microsoft](), [paper](), [sre]() ### AI overview The author reflects on becoming chief editor of Microsoft's Data Science + AI online publication and recounts a career spanning cryptanalysis research, Bing Ads R&D, and site reliability engineering. ### Source excerpt Thoughts on taking the helm of DS@MImage by the author (generated with ChatGPT). This week marks an important chapter of Data Science + AI at Microsoft, as well as in my own professional journey, as I step into the role of chief editor of this online publication after the farewell of its wonderful founder, Casey Doyle. This change is not something that I had planned -- rather it's a combination of unexpected circumstances have come together to make it happen, like many other things that have shaped my career and enabled this opportunity. If you read Casey's farewell article from last week, you may see why a metaphor of navigating the ocean came to mind when I was thinking about what's next for DS@M now that I'm "captaining the ship." The first chapter of my journey took place in 2010 as a Ph.D. intern in Microsoft Research. Under the supervision of Dmitry Khovratovich, I studied block hash functions. The paper that I coauthored ended up making a big splash in cryptanalysis (see Biclique attack -- Wikipedia), and a fun fact is that it took two years and several rejections at conferences and workshops for it to be recognized. Our approach involved surprisingly simple math and deterministic algorithms to crack a problem that was previously considered a "puzzle" requiring some craft with a bit of luck to solve. This was my main takeaway from the internship: twist and dissect the complex problems until they get reduced to an intuitively understood form, so that solving them becomes a matter of applying the right calculus. I returned in October 2012 as a full-time employee in Bing Ads R&D. I was expecting an applied research job and ended up as an SRE (Site Reliability Engineer) in the Audit Trail service working with logs collected for customer ads. It was "type 2" fun work -- absolutely not fun in the moment, but exciting when I look back. It served as a practical crash course that left the "bible" of SRE imprinted in my brain: how to design services for reliability, how t ## Reflections on the Evolution of Data Science + AI at Microsoft and a Career Transition DevFeed: [Reflections on the Evolution of Data Science + AI at Microsoft and a Career Transition]() Original publisher: [Read original article]() Author: Casey Doyle Published: 2026-06-30T07:16:00Z Content type: opinion Language: en Sources: [Data Science at Microsoft]() Topics: [Data Science](), [Microsoft](), [Artificial Intelligence](), [Data visualization]() Tags: [ai](), [data-science](), [data-visualization](), [development](), [journey](), [microsoft](), [reflections](), [ship-of-theseus]() ### AI overview The author reflects on the evolution of Microsoft's Data Science + AI publication, their departure from Microsoft, and the career experiences that shaped their approach to communicating complex ideas for business impact. ### Source excerpt Reflections on storytelling, change, and what enduresPhoto by Joseph Barrientos on Unsplash. And so the mythical hero Theseus sailed home to Athens after defeating the Minotaur. To honor him, the Athenians preserved his wooden ship in their harbor down the centuries. As the old planks rotted, builders replaced them with identical new ones. Eventually, they replaced every single original piece of wood. This sparked a famous debate among philosophers: With every part replaced, is it still the same ship? Like the ship of Theseus, what endures also undergoes many changes. That idea applies not only to the Data Science + AI at Microsoft online publication you're reading now, but also to my own career journey. After two stints totaling more than 32 years at Microsoft, and more than six years leading Data Science + AI at Microsoft, I am turning the page on both chapters this month as I prepare to depart. As I mark this transition, I feel many emotions, but one that stands out most is gratitude -- for the people, opportunities, and experiences that have shaped both the work and me. Leading Data Science + AI at Microsoft (which we call DS@M internally) since its inception has been a highlight of my working life. From its initial focus on data science in early 2020 to its evolution to encompass AI starting in 2023, DS@M continues to attract an audience as it approaches 10,000 followers. For me, this is especially remarkable given some of the internal pushback that came in the time before we launched, reflecting skepticism that it was possible, much less advisable, to share our data science expertise outside the company without crossing proprietary lines. That we moved forward -- and ultimately succeeded -- was possible only because of the many authors and collaborators who believed in the idea and were willing to contribute their work. More than 340 articles later, we've amply proved we could do it, and do it well. In many ways, this work reflects the culmination of the paths th ## Five sessions and a hackathon: How we turned skeptics into agent builders DevFeed: [Five sessions and a hackathon: How we turned skeptics into agent builders]() Original publisher: [Read original article]() Author: Jay Garg Published: 2026-06-23T07:16:01Z Content type: opinion Language: en Sources: [Data Science at Microsoft]() Topics: [Artificial Intelligence](), [Learning]() Tags: [adoption](), [agents](), [ai](), [ai-agents](), [artificial-intelligence](), [building](), [engineering](), [engineering-management](), [hackathon](), [leadership](), [productivity](), [software-engineering]() ### AI overview This article presents lessons from a five-week AI enablement series and a two-hour hackathon for an infrastructure engineering team. It argues that low AI adoption was caused less by access or time than by a lack of practical mental models and opportunities to re-engage with AI, culminating in participants building working agents. ### Source excerpt A field-tested playbook for teaching an engineering team to actually work with AI -- not just have access to itIllustration of team members collaborating around a table with laptops and notebooks, discussing holographic charts of network nodes and hexagonal shapes representing AI agents being assembled. Image generated with Microsoft Designer. Three sessions in on teaching a class whose curriculum I'd designed around building with AI, I asked the room a question I'd been avoiding. What's the thing about AI you don't say out loud? The answers came slowly at first. "I don't know where to start." "I'm worried it'll replace me." "I tried it once and it gave me garbage, so I stopped." That last one came up the most. People had given AI a shot a couple of years back, when the first wave hit. The output didn't meet the bar they hold their own work to, so they quietly filed it under "fun toy, not a serious tool" and moved on. That was the moment the curriculum I'd written stopped being a curriculum and became a conversation. This is what we learned running a five-week, thirty-minute-a-week AI enablement series for a high-performing infrastructure engineering team -- and the two-hour hackathon at the end where everyone, including managers and PMs, shipped a working agent. The paradox we started with Picture a senior engineering team. Already shipping. Already busy. Tools available, leadership encouraging use, no policy blockers. And yet -- adoption was low and shallow. A handful of folks were using AI daily. The rest were using it for autocomplete and stopping there or not touching it at all. The same people who would happily spend a weekend learning a new distributed systems primitive weren't spending 15 minutes learning how to make an agent do their service support. That's a paradox worth sitting with for a moment. The barrier wasn't access. It wasn't even time, not really. It was that everybody had tried AI in its bad season, and nobody had been given a reason to try it agai