# GumGum We're hiring! Check out https://gumgum.com/engineering - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Migrating to Nvidia Triton: High-Throughput, Low-Cost Inference at Scale DevFeed: [Migrating to Nvidia Triton: High-Throughput, Low-Cost Inference at Scale]() Original publisher: [Read original article]() Author: Eric Luu Published: 2025-09-19T01:05:13Z Content type: article Language: en Sources: [GumGum]() Topics: [Inference](), [model-serving](), [Nvidia](), [Machine learning](), [Natural language processing](), [databricks](), [Docker](), [Kafka](), [Python]() Tags: [batching](), [databricks](), [docker](), [gpu](), [inference](), [kafka](), [latency](), [machine-learning](), [nvidia](), [optimization](), [performance](), [triton-inference-server]() ### AI overview This article describes GumGum's migration of NLP inference from Databricks-based jobs using Docker images, Kafka streaming, and UDFs toward NVIDIA Triton. It introduces Triton's model-serving features, including batching, ensembles, dynamic model loading, request scheduling, and reduced GPU-CPU data transfers, while outlining limitations of the previous approach such as latency, scalability, GPU-metrics, and PySpark performance issues. ### Source excerpt Photo by Igor Omilaev on Unsplash There are tons of guides on what Nvidia Triton is and how to set it up, so I'll only go over those very briefly, instead focusing on the migration we've had and the small quirks that come from working with Python Backends. Nvidia Triton is an open source inference serving software, capable of serving virtually any machine learning model. Real time processing, batching, dynamic batching, ensembles, specialized backends, all possible with enough knobs and dials to maximize hardware use and minimize cost. Core features include File based model repository to store your models. If wanted, can be dynamically loaded onto the server Ensemble capabilities, can define whole inference pipelines, even ones that branch, on triton Ability to define each step in the inference pipeline as its own model (e.g., tokenizer -> model -> decoder), which can get hit as its own endpoint Dynamic batching between models, even in ensembles Automatic scheduling of requests between models and ensembles Minimal data transfers between GPU & CPU Our Previous Solution At GumGum we've previously deployed the bulk of our NLP inference on Databricks. These would run 24 hour jobs loading custom Docker images, streaming inputs from a Kafka topic and relying on UDF's to contain our inference logic. This was a move to mitigate issues we had on ECS related to starvation, scalability, and parallelism. Workers being paired on consumers lead to starvation. As a result, scalability of the service suffered and maintaining threading code became a pain. We've also experienced other issues as well: GPU metrics were not easily available, as we need to use an ML runtime, which was not compatible with docker containers (Although modern runtime do support this more). Latency was high as with Kafka structured streaming, microbatches must wait for all workers to finish. So if one worker hangs, the entire batch hangs. Performance on PySpark was always tough. Serializing between Java to Pyth ## Engineering a Scalable Topic Pipeline: A BERTopic and GenAI Case Study DevFeed: [Engineering a Scalable Topic Pipeline: A BERTopic and GenAI Case Study]() Original publisher: [Read original article]() Author: Saniya Naphade Published: 2025-08-28T06:25:05Z Content type: tutorial Language: en Sources: [GumGum]() Topics: [genai](), [Artificial Intelligence](), [Algorithm](), [data](), [GPU](), [datasets]() Tags: [ai](), [algorithm](), [case-study](), [clustering](), [data](), [datasets](), [engineering](), [genai](), [gpu](), [llm-applications](), [topic-modeling](), [workflow]() ### AI overview GumGum describes a scalable hybrid topic-modeling pipeline that combines deterministic BERTopic clustering with generative AI post-processing. The workflow processes impression data weekly, uses cuML for GPU acceleration, and addresses BERTopic's high outlier rate to recover more topic-related insights. ### Source excerpt Introduction: In a world where trends change in an instant, relying on static taxonomies to target ad campaigns is no longer effective. At GumGum, we face a constant torrent of digital content, ingesting millions of web pages every day. This massive flow of information must be turned into actionable insights, but because trends are dynamic, traditional methods like hard-coded taxonomies and slow hand-labeling are simply not viable at this scale. While large language models (LLMs) can offer some insight, using them at this volume is often cost-prohibitive and can lead to inconsistent results. To solve this, we've developed a lightweight and repeatable hybrid pipeline. This approach combines a deterministic clustering method -- specifically BERTopic¹ -- with a strategic post-processing layer using generative AI. The result is a more efficient and powerful topic modeling workflow that gives us the stability needed for large datasets and provides valuable, human-centric insights. Let's now walk through the key stages of this pipeline, beginning with data preparation. Data preparation The pipeline's first step is a critical data preparation phase built to handle immense scale. We process a comprehensive collection of impression data on a weekly basis to ensure we can capture not only major trends like the Summer Box Office but also the smaller, more nuanced shifts within that topic. For instance, in a given week, the pipeline might detect a specific surge of interest around a new movie's opening weekend, which would be lost in a less frequent processing schedule. This raw data is then thoroughly cleaned and prepared for the next stage, laying the groundwork for accurate and timely topic modeling. Dynamic Topic Modeling With this clean, prepared data, the real work of topic modeling begins. The pipeline then leverages BERTopic¹, a topic modeling algorithm, in conjunction with cuML³ for GPU acceleration, that is both powerful and efficient, and serves as our deterministic co ## How AI May Change the Role and Craft of Product Designers DevFeed: [How AI May Change the Role and Craft of Product Designers]() Original publisher: [Read original article]() Author: Chris Jones Published: 2025-08-12T05:39:54Z Content type: opinion Language: en Sources: [GumGum]() Topics: [Artificial Intelligence](), [prompt](), [context]() Tags: [ai](), [creative](), [design](), [designer](), [product-design]() ### AI overview An AI-generated conversation considers how AI may change product design, shifting designers toward directing, framing, and curating intelligent systems while raising concerns about the possible loss of hands-on craft and attention to detail. ### Source excerpt We are the things that others fear. Now, I know you were all hoping for a detailed monologue on the 1994 film starring Tom Cruise and Brad Pitt, or perhaps the new Netflix series. (I haven't seen it.) Sadly, this isn't that. It's something else. I fed my previous articles and thoughts on AI and product design into AI and asked for their take on the future of product design. Below is part of a transcript of our conversation. You can decide whether it's a genius, a parasite, or it just wants me to feel like I'm not replaceable. Me: OK, let's start with the big one. How do you see the role of a product designer changing in a world with AI? AI: Well, it's a fascinating question, and the shift may be less about the tools and more about the designer's entire mindset. Previously, designers were makers who created static blueprints for a fixed journey. Now, the role is more like a creative director or a coach for an intelligent system. Instead of creating every option yourself, your job is to provide the AI with the right direction, context, and constraints -- teaching it your taste and shaping its character. This moves the focus from pixel-perfect execution to the quality of the prompt and framing the problem. The skill becomes asking the right questions and curating the output to find the magic within the noise. You hold the vision, but now you have a new, incredibly powerful, and very literal-minded creative partner to help you explore it. Me: OK! First, love the phrase 'coach for an intelligent system' -- nicely done. Two things here, aren't the best Product Designers already working this way? We already know, it isn't about a series of individual moments or a fixed experience. Also, are we in danger of removing an essential part of a designer's DNA? We are the details folk, famed for our bad spelling, but we can spot a misaligned pixel from 100 meters away through a crowded room. Are we in danger of losing part of our craft? AI: Those are two excellent and very fair poin ## How AI Interfaces and Prompts Are Changing User Experience Design DevFeed: [How AI Interfaces and Prompts Are Changing User Experience Design]() Original publisher: [Read original article]() Author: Chris Jones Published: 2025-08-04T11:30:11Z Content type: opinion Language: en Sources: [GumGum]() Topics: [prompt](), [ui](), [User interface design](), [User experience (UX)](), [AI Models](), [Open Source Models & Datasets]() Tags: [ai](), [ai-models](), [interface](), [open-source](), [product-design](), [prompt](), [ui](), [ux]() ### AI overview This opinion article considers how AI may change interface and user experience design. It argues that as powerful models become widely available, well-designed interfaces and conversational prompting can distinguish products and improve the quality of AI-generated results. ### Source excerpt AI spells like a designer! Let's jump straight in here. Mobile arrived with its associated apps, and we adapted -- 'mobile-first' for the win. Social media, with its infinity (doom) scroll, changed a few habits too; ask any bored teenager. Big things come along, and we expect a paradigm shift in the UI. And yes, our screens and habits have changed, but for the most part, the interactions still feel familiar. So, will this time be different? Sure, AI is helping us work faster, smarter. But the question for me, as a Product Designer, is how it will change the nature of design itself. A wise colleague recently shared a thought that hit a chord: with open-source AI models becoming incredibly powerful, fast, and (relatively) cheap, does the true value lie in the models themselves, or in the interfaces we use to interact with them? For all its fantasia, AI is already mundane. Predictive text on your phone -- it's a form of AI we use dozens of times a day without a second thought. As AI becomes this commonplace, the real opportunity shifts. If powerful models are ten-a-penny, then an elegant, well-thought-out user interface is an opportunity to elevate one product above another. Designing for a Conversation So what does that AI interface look like? A simple text box is often a familiar starting point. It isn't the ultimate UX, but we shouldn't forget that Google did alright by sticking one in the middle of a blank screen. Removing all the noise and letting the results be the magic is a powerful foundation. The thing is, AI may start with a text box, but it expects us to behave differently. For the past two decades, search engines have trained us to be terse -- to use keywords, not sentences. AI asks us to unlearn that habit entirely. It wants to converse. I treat my interactions with AI as a back-and-forth collaboration; the real breakthrough comes when you stop issuing commands. We all feel the pain of a bad prompt as it delivers generic, lifeless results that drive you away ## How AI Tools Reshape the UX Design Process DevFeed: [How AI Tools Reshape the UX Design Process]() Original publisher: [Read original article]() Author: Chris Jones Published: 2025-07-28T22:49:08Z Content type: opinion Language: en Sources: [GumGum]() Topics: [Artificial Intelligence](), [User experience (UX)](), [Agile](), [Development]() Tags: [agile](), [ai](), [development](), [product-design](), [ux]() ### AI overview A designer reflects on adapting the UX design process to AI tools. The article argues that AI changes how designers handle automatable work while preserving the broader cycle of research, prototyping, testing, and design, and describes using NotebookLM to synthesize interview transcripts during discovery. ### Source excerpt AI enabled and ready to go! I've spent my career navigating the major shifts in tech and design, tailoring my approach along the way: Web 1.0 and Macromedia Flash, the mobile-first revolution, Agile development, Lean UX, the Double Diamond, and Design Sprints. But then, just like that, with a few new tools, I was AI-enabled. Note, I said tailored. At each stage, the next best thing or new way of working is often discussed as a radical development. However, the impact on the design process is often far less radical. We research, prototype, test, and design; we go broad, then narrow, and sometimes we even go broad again. This stuff takes time to get right (anyone want to rewatch 10 hours of user interviews?) But, as designers, our aim has always been to deliver well-thought-out solutions that approach a challenge with a unique perspective, born out of a desire to be visual in our thinking. Then, along comes AI, all innocent-like, with its polite manners and slightly sycophantic ways. "Work with me," it says. "That's a fantastic idea," it encourages. "Let me make it better for you." FINE! So I've spent the last few months ripping up my design process -- and a few long-held beliefs along the way -- to create an entirely new way of working, which also feels strangely familiar. Familiar because, like all the shifts that came before it, this one is once again forcing me to shed the tasks that can be automated and focus on what makes me human. Here we go. Discovery with AI Don't get me wrong here; I love those users. But let's be honest: the manual synthesis during the discovery phase can be a grind. We write the scripts, find the users, interview each one, watch back the recordings, make notes, lose the will to live, cover a room (or a country-sized MIRO board) in Post-its, benchmark competitors, create themes, and then finally identify and prioritise the key findings. Today? Do the interviews, then drop the transcripts into NotebookLM along with details about the Product, c ## Building smarter, not just faster: Product Discovery in the GenAI Era DevFeed: [Building smarter, not just faster: Product Discovery in the GenAI Era]() Original publisher: [Read original article]() Author: Karla Belista Published: 2025-07-03T04:22:51Z Content type: opinion Language: en Sources: [GumGum]() Topics: [genai](), [Product Management](), [Requirements]() Tags: [ai](), [engineering](), [genai](), [product](), [product-management](), [requirements]() ### AI overview This article argues that generative AI makes product delivery faster but increases the risk of building products without product-market fit. It emphasizes product discovery, human judgment, strategic thinking, and broader product-management responsibility as essential to building valuable products. ### Source excerpt GenAI has recently unlocked a super-power we could only dream about a few years ago: the ability for anyone -- including this 91-year old grandpa -- to spin up a new working product in a weekend. It's incredible and game-changing. But for those hoping for real business outcomes, it's problematic. The past 5 years has consistently shown us that 90% of start-ups fail. And most don't fail because they can't build fast enough; they fail because they lack product-market fit. GenAI will only amplify this problem. Because while it's never been faster or easier to build products, it's never been harder to build a valuable one. Core to product-market fit is investing in product discovery -- the messy, strategic, human process of figuring out what's worth building in the first place. And though GenAI has also helped to speed up discovery, it still requires deep, strategic thinking (that only humans do well) to result in quality output. Now with delivery cycles accelerating faster than ever, the risk of discovery being rushed or skipped entirely is at an all-time high. To build valuable products in our new reality, we need to do more than use GenAI to build faster. We need to fundamentally change how we think about product management: from a crucial skill required by more than just product people, to how we use GenAI to support discovery that can keep up with delivery, to the way we structure product teams. Product management skills matter more than ever If I had to point to the biggest shift I've observed recently -- it's this: judgment now matters more than execution. What used to take a team of engineers, multiple sprints, and a round of seed funding, can now be vibe-coded by one person in a single afternoon. In just one scroll of my LinkedIn feed, I'm bound to hear about the next person whose done just this. That's why product management is no longer just the Founder or PM's job. Designers, Engineers, Analysts -- if you're part of building something, you're part of shaping it. ## How AI Coding Agents Are Changing Software Engineering Roles DevFeed: [How AI Coding Agents Are Changing Software Engineering Roles]() Original publisher: [Read original article]() Author: Vaibhav Puranik Published: 2025-06-17T05:08:18Z Content type: opinion Language: en Sources: [GumGum]() Topics: [AI-assisted coding](), [AI Agent](), [GitHub Copilot](), [ai-coding](), [pull-requests](), [Tech Lead](), [Testing]() Tags: [ai](), [ai-agent](), [ai-coding-agent](), [ai-coding-agents](), [github-copilot](), [pull-request](), [quality](), [review](), [software-development](), [software-engineer](), [software-engineering]() ### AI overview The article argues that AI coding agents are becoming regular members of software teams, shifting developers' work toward task delegation, code review, feedback, and architectural decisions. It also discusses changing hiring expectations and AI-assisted test generation. ### Source excerpt Your New Coding Teammate is an AI Agent -- Are You Ready? AI coding agents like GitHub Copilot and OpenAI's Codex aren't just novelties anymore -- they're rapidly becoming your daily teammates. Surveys show that 97% of developers have tried AI coding tools at work , and GitHub Copilot now helps millions write code faster, with studies showing developers complete tasks up to 55% faster with it . This isn't the future -- it's now. And it's changing what it means to be a software engineer. Think Like a Team Lead, Not Just a Coder AI tools aren't magic wands -- they're more like junior engineers who need guidance. That means every developer, including juniors, needs to start thinking like a tech lead. Just like a lead assigns tasks and reviews code from teammates, you'll now delegate work to AI agents and manage their output. GitHub's Copilot Agents can even spin up a dev environment and create a pull request for you . So instead of grinding through repetitive code, your job is shifting to: Breaking down problems into clear instructions for AI Reviewing AI-generated code for quality Making key architectural decisions AI can't handle You're not just coding -- you're leading a small army of fast, tireless interns. Your value? Knowing how to steer them well. AI = Your New Junior Developer Treat your AI tools like you would a junior teammate: Give them clear instructions Review their code carefully Provide feedback to improve the output For example, Copilot can generate tests or refactor code, but it needs your input to do it right. It often improves drastically when you give specific guidance, just like mentoring a junior dev . Tools like Copilot Chat even let you leave comments on its pull requests, and the agent will revise code in response . This isn't sci-fi -- it's daily life on modern engineering teams. Hiring is Changing -- Are You Ready? Tech hiring is also evolving. Companies aren't just testing if you can write code -- they want to know if you can review AI-written code, ## Bridging the Gap between Business Stakeholders and Engineering Teams: A Business Analyst's Playbook... DevFeed: [Bridging the Gap between Business Stakeholders and Engineering Teams: A Business Analyst's Playbook...]() Original publisher: [Read original article]() Author: Shelby Boyle Published: 2025-05-30T06:49:43Z Content type: tutorial Language: en Sources: [GumGum]() Topics: [Support](), [Data Science](), [jira](), [dashboards](), [data]() Tags: [bridging](), [business-analytics](), [collaboration](), [communication](), [data-science](), [engineering](), [guide](), [practical](), [product-development](), [teams](), [ticketing]() ### AI overview A business analyst playbook explains how to translate stakeholder needs into product solutions. It covers clarifying problems and impacts, using JIRA to support engineering work, and applying data science models and dashboards to analyze variables, customer behavior, revenue, and conversions. ### Source excerpt Bridging the Gap between Business Stakeholders and Engineering Teams: A Business Analyst's Playbook to Creating Quality Product SolutionsDoes your bridge look like this?! Tsingy Bemaraha National Park, Madagascar (my own photo August 2024) In today's evolving technology world, communication and collaboration are vital between business and engineering teams to deliver functional, scalable solutions. The Business Analyst (BA) is the problem-solver who ensures that business needs are translated into practical products to fuel powerful strategic decisions. What does this process look like? This is a detailed playbook BAs can follow to guide from stakeholder need to fully materialized product. 1. Meet with Stakeholders on Problem and the Impact The first step in solving any business problem is understanding the problem itself. What is the problem? What would be the desired outcome? What is the business impact of the problem? What would be the business impact of a possible solution? Ask these questions of your stakeholders and encourage them to distill the problem to its core. Meeting with stakeholders guarantees no details are left out: not just what may be documented, but what may not be documented as the problem. Using an engineering ticketing system like JIRA can facilitate this initial step. The better scoped the problem is, the sooner you can start working on analytic solutions. These initial conversations set the foundation for all subsequent steps. It makes your work easier too! 2. Define the Problem and Impact After initial stakeholder conversations, the BA defines the core problem, the present consequences of not having a solution, and the impact of constructing a product solution. These are some examples of this step: We need to understand variable likeness e.g. finding types of words associated with a holiday event or grouping customers based on shopping behavior. Without knowing this, we cannot derive target strategies for success or resolve weakness. Cue the ## Lightweight User Moderation with Slack and AWS Cognito DevFeed: [Lightweight User Moderation with Slack and AWS Cognito]() Original publisher: [Read original article]() Author: Mauricio Massaia Published: 2025-05-20T23:39:54Z Content type: tutorial Language: en Sources: [GumGum]() Topics: [Slack](), [web applications](), [Users]() Tags: [account](), [application](), [aws](), [email](), [management](), [onboarding](), [real-time](), [slack](), [test](), [testing](), [users](), [web](), [web-development](), [workflow]() ### AI overview This article describes how Creative Studio replaced email-based user approval with Slack integration during its closed alpha testing phase. The system used a private Slack channel and interactive components to give authorized moderators real-time visibility and control over approving, rejecting, and updating user access. ### Source excerpt We are building a web application to make a single and consistent way for our designers to build ad creatives with a seamless workflow from Photoshop to animated HTML, accounting for cropping, uploading, applying effects and making their lives easier. As we prepared to launch Creative Studio in a closed alpha testing phase -- where access was limited to a select group of users -- we recognized the importance of having a reliable user moderation process in place. In this invite-only environment, every new user account had to be reviewed and approved to maintain control over who could access and test the platform. In our previous tool, this moderation process was handled via email. When someone signed up, an email would be sent to a small group of administrators. These team members had to manually review each request and decide whether to approve it. This workflow quickly proved to be inefficient, especially with users signing up across different time zones. The responsible admin might miss the email, and days could pass without any feedback reaching the user. Often, frustrated users had to send follow-up emails to report the issue, which would then escalate and require IT support to resolve. Learning from that experience, we knew Creative Studio needed a more efficient and transparent system. Slack integration offered a smarter and more streamlined solution, giving us real-time visibility and control over the user onboarding process without building a full-scale user management system. User Moderation Challenges Previously, user moderation faced several critical issues due to the reliance on manual email checks: Manual Approval Process: Only a few team members had access to the moderation email. User approvals were frequently overlooked, causing significant delays. No Rejection Mechanism: There was no clear method for explicitly rejecting users, leaving many uncertain about their status. Difficulty Managing Approvals: Adjusting user statuses once set was cumbersome and ## Integrating Xandr's Real-Time Data Provider On-Premises: Infrastructure Requirements and Engineering Challenges DevFeed: [Integrating Xandr's Real-Time Data Provider On-Premises: Infrastructure Requirements and Engineering Challenges]() Original publisher: [Read original article]() Author: Guy Watson Published: 2025-04-22T22:57:37Z Content type: article Language: en Sources: [GumGum]() Topics: [real-time](), [Requirements](), [on-prem](), [Latency](), [Low Latency](), [Hardware](), [data](), [digital]() Tags: [ads](), [advertising](), [challenges](), [clients](), [data-processing](), [decision-making](), [hardware](), [infrastructure](), [latency](), [low-latency](), [metric](), [on-prem](), [performance](), [platform](), [real-time]() ### AI overview The article describes GumGum's experience integrating Xandr's Real-Time Data Provider on-premises, focusing on infrastructure requirements, scaling challenges, latency, traffic volume, and the decision to own hardware. ### Source excerpt How We Slashed Our Infrastructure Costs by 80% While Successfully Integrating Xandr's Real-Time Ad Platform!Introduction In the fast-paced world of digital advertising, maximizing return on investment (ROI) for ad spend is paramount. Buyers need to make data-driven decisions in real time to ensure their ads reach the right audience at the right moment. For us, integrating Xandr's Real-Time Data Provider (RTDP) was essential to providing the real-time optimization signals that would help our clients maximize their ROI by boosting Attention Time -- the key metric driving the effectiveness of programmatic ad campaigns. Our story is one of navigating through technical hurdles, experimenting with new infrastructure, and finding creative solutions to challenging engineering problems. As we embarked on integrating Xandr Real Time Data Provider, the path was far from straightforward. From vendor selection to scaling issues, and even the decision to own our hardware, every step had its complications. Yet, each obstacle presented an opportunity for innovation and refinement. In this article, we'll take you through the key challenges we faced, how we overcame them, and the lessons we learned from integrating Xandr on-prem. Whether you're considering a similar project or are simply curious about the behind-the-scenes of large-scale advertising tech deployments, our experience might just offer the insights you need. Xandr Integration and Its Requirements Xandr, a leading programmatic advertising platform, provides powerful tools to streamline and optimize ad delivery, targeting, and performance tracking. For us, integrating with Xandr RTDP meant integrating with the real time optimisation Xandr platform. Xandr's system, built to handle vast amounts of data in real time, comes with certain infrastructure demands. These include: Volume: The platform must be capable of handling massive amounts of traffic at any given time. For example, billions of ad impressions are processed daily,