# Leveraging PyFixest for High-Cardinality Marketplace Modeling at Instacart

DevFeed: [Leveraging PyFixest for High-Cardinality Marketplace Modeling at Instacart](<https://devfeed.tech/articles/leveraging-pyfixest-for-high-cardinality-marketplace-modeling-at-instacart-20107.md>)

Original publisher: [Read original article](<https://tech.instacart.com/leveraging-pyfixest-for-high-cardinality-marketplace-modeling-at-instacart-3913df91a04b?source=rss----587883b5d2ee---4>)

Author: Benjamin Knight

Published: 2026-06-29T16:06:24Z

Content type: article

Language: en

Sources: [Instacart](<https://devfeed.tech/sources/instacart.md>)

Topics: [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [math](<https://devfeed.tech/topics/math.md>), [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>), [Software](<https://devfeed.tech/topics/software.md>), [Programming](<https://devfeed.tech/topics/programming.md>), [Routing (disambiguation)](<https://devfeed.tech/topics/routing.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cardinality](<https://devfeed.tech/tags/cardinality.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [fixed-effects-model](<https://devfeed.tech/tags/fixed-effects-model.md>), [instacart](<https://devfeed.tech/tags/instacart.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [memory](<https://devfeed.tech/tags/memory.md>), [precision](<https://devfeed.tech/tags/precision.md>), [pyfixest](<https://devfeed.tech/tags/pyfixest.md>), [regression](<https://devfeed.tech/tags/regression.md>), [routing](<https://devfeed.tech/tags/routing.md>), [speed](<https://devfeed.tech/tags/speed.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [techniques](<https://devfeed.tech/tags/techniques.md>)

## AI overview

This Instacart article explains why ordinary least squares regression becomes computationally impractical for marketplace experiments with high-cardinality categories. It presents the mathematical basis for using Fixest and Pyfixest, discusses switchback experiment designs for addressing treatment spillover, and describes benchmarks comparing processing speed, memory efficiency, and estimator precision.

## Source excerpt

Benjamin S. Knight Scaling Marketplace experiments requires specialized statistical techniques. We examine why standard ordinary least squares regression (OLS) becomes computationally intractable when controlling for high-cardinality categories. We then dive into the underlying math and demonstrate how modern packages -- specifically Fixest and Pyfixest -- bypass these limitations. We conclude by benchmarking these methods to show their real-world impact on processing speed, memory efficiency, and estimator precision. At Instacart we strive to give our customers access to all the fresh foods and ingredients that they would normally get from a trip to the grocery store, but without the hassle of driving, finding parking, waiting in line, etc. Instacart's Marketplace team is responsible for surfacing customers' orders to shoppers, aligning Instacart's delivery windows with shoppers' projected availabilities as efficiently as possible. This entails a careful balancing act. If we offer delivery windows that are sooner / more popular, then we risk overextending shoppers' ability to fulfill those orders on time. If we are too conservative in our delivery option offerings, then we risk losing potential orders. Accurately measuring the impact of changes in our batching and routing algorithms requires thoughtful experiment design and software. Better predictions of future demand / time-to-fulfill allow Instacart to offer more convenient delivery windows.Experimentation on Marketplace One of our primary concerns in Marketplace is treatment spillage. For example, if we adjust our batching algorithm and increase the rate at which multiple orders are combined into batches in Brooklyn and Queens, then we face a real risk of also influencing the rate of batch creation / completion in Staten Island, the Bronx, and Manhattan. In this case the treatment impacts the control group -- a classic source of measurement bias as a consequence of violating the Stable Unit Treatment Value Assumpt