# Thumbtack Engineering - Medium Stories from the Engineering team at Thumbtack - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Three principles for building a vector platform at Thumbtack DevFeed: [Three principles for building a vector platform at Thumbtack]() Original publisher: [Read original article]() Author: John Zhu Published: 2026-09-10T15:45:00Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Embeddings](), [AI search](), [Database](), [data](), [Machine learning](), [API]() Tags: [api](), [data](), [embeddings](), [engineering](), [etl](), [infrastructure](), [machine-learning](), [ml-platform](), [pgvector](), [retrieval](), [search](), [vector](), [vector-search]() ### AI overview This article explains how Thumbtack built a vector platform that lets ML engineers deploy production vector search without managing database access, custom ETL, or query services. It describes three guiding principles: reuse existing infrastructure, treat embeddings as data, and reduce adoption costs for future teams. ### Source excerpt Reusing what we already had, treating embeddings as data, and lowering the next team's cost Today, an ML engineer at Thumbtack can stand up production vector search without negotiating database access, building a custom ETL, or writing a query service. The team brings their choice of embedding model, the data, and the query; the platform handles what connects them. It took several iterations to get to this point. In this post we'll walk through how we got there and the three principles that shaped what we built. A vector database stores high-dimensional numeric arrays (embeddings) and serves nearest-neighbor queries against them. It's how an ML system asks "what's most similar to this?" instead of "what matches this exact key?" The shift from exact lookup to semantic retrieval is what makes vectors useful: a search can return results that mean the same thing, not just results that spell the same. At Thumbtack, embeddings sit between the models that produce them and the services that consume them: language models for text, multimodal models for images, retrieval models for ranking. The platform we describe here is where those embeddings live and how teams reach for them when they need to. Three principles shaped what we built. Reuse what we have: extend the infrastructure we already run rather than stand up a new system. Treat embeddings as data: flow them through the same pipelines that move every other dataset at the company. Lower the next team's cost: make the platform easier to adopt than to work around. Each principle shaped one layer of the system, and together they took vector search from a one-off project to a platform that any team can build on. Architecture at a glance The platform has four moving parts: where embeddings come from, how they reach the database, where they live, and how consumers query them. Each is a layer, and together they form a pipeline that produces vectors and serves similarity searches as a typed API call. The diagram below traces a ## How We Moved 1,500 Android Screenshot Tests to Roborazzi DevFeed: [How We Moved 1,500 Android Screenshot Tests to Roborazzi]() Original publisher: [Read original article]() Author: Zachary Wander Published: 2026-08-08T05:03:41Z Content type: tutorial Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [screenshot-testing](), [Android](), [Testing](), [Compose](), [Kotlin](), [Firebase](), [Git]() Tags: [android](), [compose](), [consistency](), [firebase](), [git](), [jetpack-compose](), [kotlin](), [mobile](), [robolectric](), [roborazzi](), [screenshot](), [screenshot-testing](), [software-testing](), [technology](), [testing](), [tests]() ### AI overview Thumbtack Engineering describes migrating 1,500 Android screenshot tests from Firebase Test Lab to Roborazzi, which runs locally on the JVM. The article covers porting existing suites, comparing screenshots in Git, and addressing rendering, test-data injection, and dependency-injection differences. ### Source excerpt How we moved 1,500 Android screenshot tests to Roborazzi.Introduction If you're a mobile developer you're probably no stranger to screenshot tests. Making sure that layouts remain consistent as time goes on and code changes is important for ensuring functionality and accessibility. Since Thumbtack uses Kotlin and Jetpack Compose for its Android apps, we had historically been using Firebase Test Lab to capture and compare layouts. However, using Firebase can be slow, tedious, and error-prone, so we decided it was time for a change. Instead of Firebase, we decided to migrate to using Roborazzi, a screenshot testing framework that runs fully locally inside the local JVM instead of relying on remote emulators and devices. For more details on why we migrated and the alternatives we evaluated, check out the previous blog post from Brian. This post is going to go into the details of how we migrated, and the challenges we faced along the way. Porting Loop Because the purpose of screenshot tests is ensuring consistency over time, we wanted to bring over our existing test suites into Roborazzi. Deleting the old ones and just creating new tests as we built new layouts was a potential option, but it would wipe out a lot of our automated verification. Instead, we ported our existing Firebase Test Lab screenshot test suites over to Roborazzi. In general, the porting process was pretty simple: Delete the old screenshots. Move the test suites over from androidTest to test. Adjust the code for any API differences. Record new screenshots. Compare the old and new screenshots in the git diff to make sure nothing is broken. Fix broken screenshots. Since Roborazzi uses Robolectric, and Robolectric implements the Android API, most of the test suites needed very few changes to work, with updating the main test rule and test annotations being enough to get them running. A few, like those that relied on permission granting rules or interacted directly with the test Activity, needed some more ## Treating Pricing Changes Like Code Deploys DevFeed: [Treating Pricing Changes Like Code Deploys]() Original publisher: [Read original article]() Author: Ananda Kanagaraj Sankar Published: 2026-07-22T18:09:48Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Deployment](), [DynamoDB](), [Go Language](), [Code review]() Tags: [code-review](), [deployment](), [devops](), [dry-run](), [dynamodb](), [engineering](), [go](), [marketplaces](), [monitor](), [platform-engineering](), [pricing-strategy]() ### AI overview This article explains how Thumbtack handled large pricing changes by comparing them with code deployments. Pricing bids stored in DynamoDB were rewritten in bulk using custom Go scripts, but those scripts lacked built-in resumability, runtime control, and durable records for undoing changes. ### Source excerpt Making large pricing changes safe to run, safe to resume, and safe to undo. Most days, shipping code to production at Thumbtack is a non-event. You merge your change, a deployment pipeline takes it from there and it rolls out while you monitor the rollout. If a deploy stalls, the system already knows which step it stalled on. If it goes wrong, rolling back is a button away. It's the return on years of platform work, so routine now that we mostly forget it's there. Which is the point: Safety is built into the road, not into how carefully each person drives. Changing pricing data never had that platform at Thumbtack. Pricing is how a marketplace like ours sustains itself. It lets us capture a share of the value we help create, and it's something we tune on an ongoing basis rather than set once. We test pricing changes, measure their impact and roll out the ones that work. Mechanically, this runs on an auction. When a customer creates a project on Thumbtack, pros compete for the lead, and what a pro is willing to pay shows up as a bid. Those bids are stored per pro's service offering (a specific offering a pro provides, like house cleaning or landscaping) as bid settings in DynamoDB. Rolling out a pricing change means rewriting those settings in bulk, often a few hundred thousand of them at a time, several times a year. For a long time, each of the pricing rollouts was a custom Go script written for that specific change. The scripts were production grade. They went through code review, ran in staging first, did a dry run before any live write, and were rolled out in stages. That was a reasonable way to work: the changes are infrequent, each one is genuinely different from the last, and a script you can read top to bottom is easy to trust. But what reviews and a good runbook can't give you is the runtime machinery. A script doesn't remember where it was when it died, so it can't safely pick up where it left off. It has no real sense of how hard it's pushing the database ## Building a Transformer-Based Category Recommender at Thumbtack DevFeed: [Building a Transformer-Based Category Recommender at Thumbtack]() Original publisher: [Read original article]() Author: Andrew Morss Published: 2026-07-20T20:23:27Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [recommendation systems](), [Transformer](), [Machine learning](), [email](), [push-notification]() Tags: [catalog](), [data-science](), [email](), [engineering](), [machine-learning](), [model](), [push-notification](), [recommendations](), [recommender-systems](), [torch]() ### AI overview This engineering article describes Thumbtack's category recommender, which selects home-service categories for push and email notifications. It explains how the system addresses popularity bias through negative sampling and how Thumbtack's small category catalog and sparse user activity shape the recommendation problem. ### Source excerpt A look at compensating for popularity bias in recommender systems using negative sampling strategies By: Andrew Morss, Senior Applied Scientist Introduction A recommender system is a machine learning model that, given a user and a catalog of items, predicts which items that user is most likely to want. Recommenders set your YouTube playlist, determine what items Amazon suggests for you, push you songs on Spotify and customize your Steam store. If you're a homeowner, Thumbtack's recommender systems can suggest home projects for you such as house cleaning or lawn mowing. Thumbtack connects users with professionals spread across almost 500 different job categories. Job categories are a kind of occupational bucket that Thumbtack uses as a taxonomy to categorize local service professionals (pros). Pros have the opportunity to choose categories of jobs that they want to receive leads on; examples include things like "Electrical and Wiring Repair", "Local Moving" or "Snow Plowing". Thumbtack's Lifecycle marketing team uses a category recommender to choose which categories to surface to users in our push and email notifications. Choosing the right ones is critical. A well-chosen suggestion is how a customer starts a search for a new project, finds the right service professional, and ultimately gets work done. An example of a push notification and an in-email category recommendation. The Problem Space: Category Recommendation There are several features of category recommendations at Thumbtack that are fairly unique in the item recommender space. The number of categories that we have is relatively small (~500) compared to the number of items that are usually seen in recommender systems, which simplifies the problem considerably. Usually recommenders require a two stage approach where an initial, low compute, filtering stage reduces the number of items to some "candidate" list of items before a more complicated model ranks that smaller set of items. With a relatively small cat ## AI as the Next Abstraction Layer: How I see engineering evolving at Thumbtack DevFeed: [AI as the Next Abstraction Layer: How I see engineering evolving at Thumbtack]() Original publisher: [Read original article]() Author: Ananda Kanagaraj Sankar Published: 2026-05-22T23:53:03Z Content type: opinion Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Artificial Intelligence](), [agentic workflows](), [Development](), [Software Engineering](), [engineering-leadership]() Tags: [agentic-workflows](), [ai](), [ai-agent](), [chatgpt](), [copilot](), [development](), [engineering](), [engineering-culture](), [generative-ai-tools](), [software-engineering]() ### AI overview A personal perspective on how AI-assisted development is evolving at Thumbtack, from informal experimentation with ChatGPT and Copilot to end-to-end agentic workflows. The article frames AI as a new abstraction layer and highlights its non-determinism as a major difference from earlier abstractions. ### Source excerpt Over the past year, the way we use AI at Thumbtack has gone through a few phases. Early on it was mostly curiosity, people experimenting with ChatGPT and Copilot on side projects, sharing tips in Slack. Then the models got noticeably better at working inside real, mature codebases (not just greenfield projects) and the conversation shifted. It stopped being about whether we should adopt AI-assisted development and became about how. Lately, it is moving towards the adoption of end to end agentic workflows for development. I've been thinking a lot about what this shift means, not just for our codebase or our velocity, but how it impacts what it actually feels like to be an engineer here. What follows is my personal perspective, shaped by leading engineering on our monetization teams and informed by how Thumbtack's engineering leadership has been approaching this across teams. Another layer of abstraction If you zoom out, software engineering has always been a story of rising abstraction. We went from assembly to C, from C to Java, from hand-rolled servers to cloud-native infrastructure. And each time, the craft shifted rather than shrinking. For example, managing memory manually was replaced by designing distributed systems. I think AI-assisted development is the next version of that pattern.The difference this time isn't that the new layer takes on implementation work. Compilers always did some of that, with their own undefined behavior and implementation-defined choices. The difference is the leap in non-determinism. The earlier abstractions were designed to be mostly deterministic and mostly non-leaky, and AI tools break that pattern. The same prompt produces different code on different days, with different trade-offs and different bugs -- that's a bigger shift than swapping languages. I keep coming back to the same historical pattern: the engineers who thrive are the ones who can operate at the new layer, not the ones who insist on staying anchored to the old one. ## Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI DevFeed: [Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI]() Original publisher: [Read original article]() Author: Thumbtack Engineering Published: 2026-04-29T00:16:16Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Human-AI evaluation](), [genai](), [trust & safety]() Tags: [ai](), [ai-engineering](), [ai-evaluation](), [data-science](), [evaluation](), [genai](), [research](), [safety](), [technical]() ### AI overview Thumbtack describes a learning-driven, exploratory approach to evaluating generative AI experiences. Its strategy combines cross-functional insights with a parallel-path MVP evaluation system to address probabilistic outputs, unsupported claims, harmful content, changing model behavior, and trust-related risks. ### Source excerpt A practical look at how Thumbtack navigates evaluation for emerging AI experiences and what we've learned along the way. By: Shishir Dash, Director of Applied Science & Teja Venkat Kolli, Senior Applied Scientist Evaluating AI at ScaleIntroduction AI is reshaping how people interact with products, and Thumbtack is no exception. We're introducing AI into more aspects of our customer and local service professional (pro) experiences -- from helping customers articulate what they need, to generating helpful summaries, to offering clearer explanations of how pros may fit those needs. But evaluating generative AI is uniquely challenging. Unlike traditional software, its outputs are probabilistic, wide-ranging, and capable of subtle errors: mistakes in tone, inaccuracies, unsupported claims, or harmful assumptions. Rather than attempt to formalize a single rigid evaluation framework, we've taken a learning-driven, exploratory approach, pairing cross-functional insights with a parallel-path MVP evaluation system. This balanced strategy allows us to move quickly while staying grounded in safety, responsibility, and quality. Why AI Evaluation Matters Evaluation is essential because generative AI can produce unsupported or overly strong claims. Sometimes it can misinterpret user intent or vary in style or tone from one version to the next. It can sometimes generate harmful, biased, or inappropriate content. It can also drift over time due to model updates or prompt changes. For a marketplace built on trust, these challenges matter. Customers need accurate guidance; pros need fair, clear representation. Evaluation helps ensure every AI interaction strengthens and not undermines that trust. Our Approach: Exploration, Learning, and MVP Paths The landscape of AI evaluation is still evolving. New research, tooling, and patterns emerge every month. Rather than over-commit to a single approach, we've adopted a mixed strategy rooted in: Exploration and fast learning across multiple pro ## Leveraging AI to build a faster iOS CI system DevFeed: [Leveraging AI to build a faster iOS CI system]() Original publisher: [Read original article]() Author: Muhieddine EL Kaissi Published: 2026-04-21T18:16:01Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [CI/CD](), [Testing](), [iOS](), [AI-assisted coding]() Tags: [ai](), [ai-assisted-coding](), [ci-cd](), [continuous-integration](), [developer](), [developer-productivity](), [ios](), [ios-development](), [mobile](), [pipeline](), [testing]() ### AI overview Thumbtack describes Smart Test Selection, an iOS CI system that reduces pipeline time by running only tests relevant to each pull request. The article explains why parallelization stopped scaling, including node contention, large test suites, and slow snapshot and UI tests, while noting that AI-assisted coding could increase CI demand. ### Source excerpt Every iOS pull request at Thumbtack used to wait 30 to 40 minutes for Continuous Integration (CI) -- sometimes over an hour on Fridays. We built Smart Test Selection (STS) to cut that in half. Here's how. I'm a Senior Software Engineer on the iOS Infrastructure team. We own the CI/CD pipelines, build tooling, and shared frameworks that keep Thumbtack's apps shipping reliably. We also build AI-powered developer tooling -- 35+ AI Agent Skills, MCP integrations for Jenkins, Figma, Firebase, and Xcode -- that help the broader iOS team ship faster. Over the past few quarters, we built Smart Test Selection (STS), a system that reduces CI time by running only the tests that matter for each pull request. Why Full-Suite Testing Stopped Scaling Before STS, we had already optimized CI through parallelization -- running consumer, pro, and shared framework tests on separate nodes, and splitting snapshot tests across multiple simulator clones. But parallelization hit a ceiling. The more nodes we spread CI across, the more engineers had to wait for free nodes, cancelling out the speed gains. Snapshot tests dominated the suite but couldn't be split further without diminishing returns, and UI tests were inherently slow per-test. We needed a fundamentally different approach: run fewer tests, not run the same tests faster. Every iOS PR ran the full test suite for each affected app. We have three main targets -- a consumer app, a pro app, and a shared framework -- totaling 1,500+ test classes. The pipeline already skipped unaffected apps (a change to the consumer app would not run pro app tests), but within each affected app, every test ran regardless of what specifically changed. A one-line bug fix in one screen would trigger all 400+ tests for that entire app. This was becoming unsustainable. Human-generated code was already putting CI at the limit due to ever-increasing code size and test counts. We had increased our CI node count multiple times to keep up with demand, but each expansion ## Headless Screenshot Testing in Thumbtack's Android Apps DevFeed: [Headless Screenshot Testing in Thumbtack's Android Apps]() Original publisher: [Read original article]() Author: Brian Terczynski Published: 2026-03-31T22:47:00Z Content type: tutorial Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [screenshot-testing](), [Testing](), [Android](), [Emulator](), [Firebase](), [Jenkins]() Tags: [android](), [android-app-development](), [emulator](), [firebase](), [jenkins](), [mobile](), [robolectric](), [roborazzi](), [screenshot-testing](), [sdk](), [testing]() ### AI overview Thumbtack explains how it chose Roborazzi to run high-fidelity screenshot tests for its Android apps without an emulator. The article describes how emulator-based testing was slow and difficult to use for Jenkins verification, especially with more than 1,500 tests. ### Source excerpt How we chose Roborazzi for fast, easy, and high-fidelity screenshot tests of our Android apps, without needing to run them on an emulator. As mentioned in our earlier blog post, screenshot tests are a fundamental part of our testing pyramid at Thumbtack. They validate that our UI code renders correctly. They catch UI regressions as our code evolves. They make it easy to share incremental progress with designers. In code reviews, they provide a great way for reviewers to "see" what the UI code is doing. Finally, they are very easy to write! But they were not always easy to run. Because they tested Android UI code, they had to run in a "real" Android environment so that the drawing operations would work. This meant running them on an Android device or emulator. To ensure consistent renderings, developers would have to run their tests on the same device used by our CI job. In the past, this meant having to run a script that would connect to our cloud device testing provider to generate the reference image, which would then be downloaded for inclusion in the local Git branch. Such a process would take minutes to run (not including build time). This got easier when we switched to Firebase Test Lab (FTL), because their emulators were the same as the emulators provided in the Android SDK used on developers' machines. Developers could then simply run their tests on a local emulator which sped up execution time significantly. But it still meant the emulator profile needed to match that of the device on FTL; in particular, the screen size and pixel density. Another problem was, because these were run on an emulator, they could not run with our CR verification job on Jenkins because their execution time was too slow (especially since we have over 1,500 of these tests!). This therefore meant that we had to run our screenshot tests separately, and they could not be used to gate code pushes because they were just too slow to run. So what would often happen was developers would me ## Working at the intersection of data and AI with Kenza Boulisfane DevFeed: [Working at the intersection of data and AI with Kenza Boulisfane]() Original publisher: [Read original article]() Author: Thumbtack People Team Published: 2026-03-05T19:31:06Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Artificial Intelligence](), [data](), [recommendations](), [real-time]() Tags: [agent](), [ai](), [analytics](), [careers](), [data](), [real-time](), [recommendations](), [software-development](), [technology]() ### AI overview In a Q&A, Thumbtack software engineer Kenza Boulisfane discusses building an AI-powered Marketing Analytics Agent. The tool is intended to help people across the company ask questions about marketing performance, receive data-backed recommendations, and explore results in real time without requiring SQL expertise. She also describes the challenges of unifying fragmented, differently defined marketing data and translating it into clear insights. ### Source excerpt Kenza Boulisfane, Software Engineer at Thumbtack, works at the intersection of data, AI, and real business impact. In this Q&A, she shares how she's building an AI-powered Marketing Analytics Agent designed to make complex marketing data accessible to everyone. She also reflects on team culture, technical challenges, and why diverse perspectives make engineering stronger. What are you currently working on? I'm working on building a Marketing Analytics Agent. It's an AI-powered marketing expert that provides companywide support, regardless of technical background. The idea is simple: marketing data shouldn't require a SQL expert to interpret it. Whether someone is in Data Science, Growth, Brand, or Product, they should be able to ask questions like, "How did paid social perform last week?" or "Why did conversions dip in this channel?" and get a clear, reliable answer. Even better, they should receive thoughtful, data-backed recommendations. Instead of waiting on reports or digging through multiple tools, teams can explore performance in real time, ask follow-up questions, and move faster with confidence. I love that my work sits at the intersection of data, AI, and real business impact. How would you describe the culture of your team? I'd describe my team culture as genuinely supportive. People are always willing to jump in, answer questions, share context, or sanity-check an idea, regardless of how busy they are. We're in it together, and that makes tackling ambitious projects feel a lot less daunting. At the same time, we don't take ourselves too seriously. Some of my favorite moments have been at offsites when we play games and you suddenly see everyone's competitive side come out. It's funny how the same people who are thoughtfully debating technical implementation during the day are fully locked in trying to win a Mafia/Werewolf round. What are some of the unique technical challenges you get to solve? One of the biggest challenges is turning fragmented marketi ## A Q&A on Site Reliability Engineering and Observability at Thumbtack DevFeed: [A Q&A on Site Reliability Engineering and Observability at Thumbtack]() Original publisher: [Read original article]() Author: Thumbtack People Team Published: 2026-02-05T18:01:47Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [site-reliability-engineering](), [observability](), [Software as a service](), [Self-hosted](), [Incident response](), [telemetry]() Tags: [careers](), [company](), [culture](), [employee-experience](), [engineering](), [growth](), [incident-response](), [observability](), [saas](), [scale](), [self-hosted](), [site-reliability](), [software-engineer](), [software-engineering](), [systems](), [technical](), [technology](), [work]() ### AI overview In this Q&A, Thumbtack Senior Software Engineer Brett Shouse discusses his site reliability engineering work, including migrating self-hosted observability services to a unified SaaS platform. He also describes addressing technical debt, improving incident response, and unifying logs, traces, and metrics. ### Source excerpt A Q&A with Brett Shouse, Senior Software Engineer.As an engineer, what drew you to Thumbtack? I joined Thumbtack in March of 2025, and from day one it's felt like the right place to grow, build, and make a real impact. I was ready for a fresh challenge and wanted to help evolve a business that truly aligned with my values. I took a very intentional approach to my search. I reviewed the career sites of 106 companies to deeply understand their cultures, and from that process I applied and interviewed with four outstanding organizations. Each had strong teams and compelling missions, but Thumbtack stood out immediately. The values resonated with me, the opportunities for growth were clear, and I could see countless ways to meaningfully contribute -- not just within site reliability engineering, but across engineering as a whole. Choosing Thumbtack felt both exciting and obvious. What work are you currently focused on? I'm currently leading work to migrate a collection of self-hosted observability services into a unified SaaS platform. This effort is reducing the ongoing burden of systems administration, application support, and alert fatigue, while also lowering costs. Most importantly, it frees up engineering time and energy so teams can focus on the projects that directly support Thumbtack's customers. It's deeply satisfying work because the benefits compound across the entire organization. How would you describe the culture of your team? Supportive, driven, and genuinely fun. My teammates are endlessly curious, highly skilled, and always learning. We love automating toil away so we can focus on meaningful, high-impact work. It's a team that pushes for excellence while lifting each other up along the way. What are some of the unique technical challenges you're solving? Tackling technical debt that naturally accumulates when a company leans in early and then grows rapidly. Defining what effective incident response means for the business and enabling fast, confident res