# Groupon Engineering -- Medium Tech blog from Groupon's Product & Engineering team - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## How Groupon Tracks Cloud Costs Across AWS and Google Cloud DevFeed: [How Groupon Tracks Cloud Costs Across AWS and Google Cloud]() Original publisher: [Read original article]() Author: Michael Published: 2023-06-16T20:11:47Z Content type: article Language: en Sources: [Groupon Engineering -- Medium]() Topics: [Cloud](), [Amazon Web Services](), [Databases](), [migration](), [Amazon S3](), [data]() Tags: [amazon](), [aws](), [cloud](), [cloud-cost-management](), [cost](), [data](), [database](), [infrastructure](), [migration](), [s3](), [scale](), [services](), [team]() ### AI overview Groupon describes its cloud migration and its approach to tracking infrastructure costs across AWS and Google Cloud. The article explains how service tags connect cloud resources to individual systems and engineering teams, enabling more detailed cost analysis. ### Source excerpt Groupon completed its migration to the cloud in February 2023. Shortly thereafter our data centers were completely shut down and the equipment cleaned up. The move to the cloud has brought many advantages to Groupon engineering. Our infrastructure can now dynamically scale with our traffic throughout the year and engineering resources can be easily started up or turned off as our needs change. The move to the cloud has also brought a wealth of information regarding our infrastructure costs and utilizations. Groupon operates in both Amazon AWS and Google GCP. Our main consumer and merchant applications reside in AWS while our backend data warehouse and reporting systems are in GCP. Both clouds provide extensive data on service costs including granular usage information. As more Groupon systems migrated to the cloud it became clear there were many opportunities for making our infrastructure and systems more efficient. To do this though we wanted to ensure we were tracking the costs appropriately and making the best decisions with our infrastructure and systems. Cost Tracking The first part of this journey involved improving our cost tracking. While the cloud provides extensive cost data it's important to match this data with individual systems to get a more complete picture. Understanding that we're spending a lot on databases is useful but being able to map that to particular systems and teams gives us more information on the course of actions we can take. Tags are an oft recommended approach to tracking service costs and Groupon is no exception. Groupon applied "service" tags across all our cloud resources. This allows us to map cloud resources to individual services and with it the individual engineering teams. So an RDS database may have the tag service=payments to map that to our Orders and Payments team while an S3 bucket may have service=images to map it to our global image management team. This now allows us to see not only how much we spend overall on databas ## LoadBalancer Services using Kubernetes in Docker (kind) DevFeed: [LoadBalancer Services using Kubernetes in Docker (kind)]() Original publisher: [Read original article]() Author: Owain Williams Published: 2022-09-20T20:08:29Z Content type: tutorial Language: en Sources: [Groupon Engineering -- Medium]() Topics: [Kubernetes](), [Docker](), [Orchestration](), [container](), [Development]() Tags: [container](), [development](), [docker](), [ingress](), [kind](), [kubernetes](), [load-balancer](), [orchestration]() ### AI overview This tutorial introduces Kubernetes and compares local options for experimenting with it, including Docker's single-node cluster, minikube, and kind. It explains that kind runs cluster nodes in containers and supports multi-node clusters, but the supplied text ends during cluster creation. ### Source excerpt Kubernetes is a very popular open-source container orchestration system for deploying, scaling, and updating your software, as well as automatically recovering from failures. If your code can be packaged into a container, e.g. via Docker, then it can run on Kubernetes, or k8s for short (which is sometimes pronounced 'kates' and refers to the 8 letters between the first and last letter). You can read more about it on the Kubernetes Documentation page. You can run a Kubernetes cluster in your own data centre as well as on many of the popular cloud providers, for example there's the Amazon Elastic Kubernetes Service (EKS), the Azure Kubernetes Service (AKS), and the Google Kubernetes Engine (GKE) to name a few. You can also create clusters that span your data centre and a cloud provider or even span multiple cloud providers. Here at Groupon, we have several Kubernetes clusters around the world hosted on both EKS and GKE. This is great for production, or even as a staging cluster, or shared development environment. But what if you just want to try it out and see what all the hype is about before you spend money on a new cluster? The good news is there are several options, if you've installed Docker you can enable a single-node Kubernetes cluster simply by ticking the option in the Docker settings. However, you will be tied to the version of Kubernetes that ships with your version of Docker. Another option is minikube which allows you to run specific versions of Kubernetes, or even multiple clusters with different versions. However, you'll still only be able to run single-node clusters. A single-node cluster is probably fine, and could well be all you ever need. But you won't be able to experiment with pods running on different nodes, and the various like-live scenarios that enable, for example using hostPath volumes with different files on each node. Before you fire up a load of VMs and try installing Kubernetes The Hard Way there is one more option, kind, which stands ## Migrating to Optimizely at Groupon DevFeed: [Migrating to Optimizely at Groupon]() Original publisher: [Read original article]() Author: Andres Otarola Published: 2022-09-12T21:17:34Z Content type: article Language: en Sources: [Groupon Engineering -- Medium]() Topics: [experiments](), [Mobile](), [Cloud](), [API](), [Caching]() Tags: [api](), [article](), [cache](), [cloud](), [compatibility](), [components](), [data](), [experimentation](), [experiments](), [mobile](), [pipeline]() ### AI overview Groupon describes migrating its in-house experimentation pipeline to Optimizely's cloud-based solution. The article outlines the existing data transport and experiment engine components, the migration challenges of maintaining backward compatibility and supporting web and mobile clients, and the mobile experiment flow using a public-facing API and cached in-memory data. ### Source excerpt Sharing experiences and learnings 🧪Photo by Alex Kondratiev on Unsplash Experimentation is a key indicator of whether a feature will be appealing to users, maybe fall flat, or worse: cause a negative impact on them. Here at Groupon, we have been data driven for years, so it is in our nature to A/B Test everything that we present to our customer base. The objective of this article is to give you some insights into how we migrated our existing experimentation pipeline onto Optimizely's solution by swapping certain components from our pipeline. Let's dive in. As mentioned, we already had an existing experimentation pipeline, which can be generally split into two main components: Data transport Experiment engine Generalised diagram of experiment eventsData transport This is the layer that is responsible for transporting the events of an experiment. "What is an impression event?" you may inquire. It's an interaction of a unique user with the treatment of an experiment. For Example, an experiment that presents either a red button or a green button to a user will be counted as an impression event with the chosen variation (either a red button or a green button). Such events are handed over to specialized channels which end up being cleaned up by a central process in charge of canonicalizing the events so upstream services can easily parse the data. It is important to emphasize that the Data transport layer is ignorant of the details of the experiment engine. Experiment engine This is the actual component that decides how an experiment is performing based on the events it is receiving, as well as providing the information of the currently available experiment to query from. (In our case this implementation varies per client; more on this later on.) Enter Optimizely When we decided to switch from our in-house experimentation solution to a cloud-based one, in this case, Optimizely, we immediately faced various challenges: Keep backward compatibility (Even more so for Mobile!) ## Exporting Node Modules in 2022 DevFeed: [Exporting Node Modules in 2022]() Original publisher: [Read original article]() Author: David Bushong Published: 2022-09-01T20:01:47Z Content type: tutorial Language: en Sources: [Groupon Engineering -- Medium]() Topics: [CommonJS](), [npm](), [ECMAScript](), [TypeScript](), [Webpack](), [import]() Tags: [babel](), [commonjs](), [compatibility](), [import](), [javascript](), [node](), [npm](), [typescript](), [webpack]() ### AI overview This tutorial explains CommonJS and native ES Module exports in Node.js, including how modules can be imported across module systems. It also compares the output of Babel, Webpack, and TypeScript with native Node.js module behavior, highlighting compatibility concerns and TypeScript's non-standard export syntax. ### Source excerpt Groupon maintains literally hundreds of NPM modules, both open source and internal. Many of these are consumed by our custom NodeJS-based middleware web layer we call "The Interaction Tier" (itself a topic for another post someday). As folks write new modules, a common question is "what's the best way to export things from our published modules to maximize compatibility?" -- and that is the topic of this post. First, some background and history: Flavors of exported modulesCommonJS in Node In the beginning, there was CommonJS: Here are 3 sample files with exports: // export1.js - exporting individual properties w/ CommonJS 'use strict';function foo() { } exports.foo = foo; exports.bar = 42;// export2.js - exporting a single object w/ CommonJS 'use strict';function baz() { } const garply = 88; module.exports = { baz }; // dynamically (conditionally!) exported! if (Math.random() > 0.5) module.exports.garply = garply;// export3.js - exporting a bare function w/ CommonJS 'use strict';function quux() { } module.exports = quux;// sometimes there are extra properties added to the bare function quux.yadda = 42; Those CommonJS exports can be imported either into other CommonJS files or into ES Module (more on that below) files: // import.js - importing CommonJS modules into a CJS file 'use strict';const { foo, bar } = require('./export1'); const { baz, garply } = require('./export2'); if (Math.random > 0.9) { // can also dynamically decide when to import const quux = require('./export3'); // can poke into properties tacked onto functions const { yadda } = require('./export3'); }// import.mjs - importing CommonJS modules into an ESM file // node is willing to turn exported objects into named exports import { foo, bar } from './export1.js'; import { baz, garply } from './export2.js'; import quux from './export3.js'; // cannot access added property "yadda" directly; hence: const { yadda } = quux; You can read more elsewhere, but the key features are: The module files are synchron ## Varnish to Redis Migration DevFeed: [Varnish to Redis Migration]() Original publisher: [Read original article]() Author: Ravikumar Published: 2022-03-22T15:39:20Z Content type: article Language: en Sources: [Groupon Engineering -- Medium]() Topics: [migration](), [Caching](), [Amazon EKS](), [Redis](), [Amazon Web Services]() Tags: [aws](), [cache-invalidation](), [caching](), [cloud](), [distributed-transaction](), [haproxy](), [kafka](), [kubernetes](), [migration](), [nginx](), [proxy](), [redis](), [varnish]() ### AI overview Groupon describes migrating its Varnish caching infrastructure to Redis while moving application workloads to AWS EKS. The article focuses on a high-traffic place read service, the limitations of TTL-based cache invalidation, and the planned shift to event-based expiration. ### Source excerpt At Groupon, we are in the process of migrating our app workloads to AWS EKS. As part of the migration, we are also in the process of re-architecting our services to make them cloud & Kubernetes optimised. In this article, we will focus on the impact of cloud migration on Varnish caching at Groupon. For the uninitiated, Varnish is a high-performance reverse caching proxy. For simplicity, you can think of it as a general proxy such as HAProxy or Nginx with caching support so that it doesn't have to go to a backend service every time it receives a request. Internally, Varnish cache uses pthreads heavily(to the tune of 1000s) to serve the requests. In contrast, Redis is single-threaded. Varnish at Groupon & Place Read Service We have been using Varnish for many of our high-traffic services. These read-heavy services get traffic to the tune of millions of RPM. One such service is our place read service which is called on each & every deal page visit; it will be the focus of this article. Place read service stores all the locations a deal can be redeemed. Ex: Redemption location details of the deal https://www.groupon.com/deals/parent-big-littles-3 is highlighted below. Traffic pattern of place read service The service is called more than a billion times a day with peak traffic of ~2 million requests per minute. Current architecture (simplified) The traffic is served by a cluster of 4 Varnish servers (40GB RAM). The current Varnish hit rate is 95%. This is admittedly a low hit rate for a read-heavy service, the main reason being our TTL-based cache invalidation logic. Varnish cache invalidation We are using TTL-based cache invalidation. This usually leads to a problem where Varnish starts bombarding the backend service when most of the keys expire at the same time. Request coalescing is not very helpful when the unique keys count is in millions (which is the case with our service). As a workaround, we use randomised TTLs (between 30 to 60 mins) so that all the keys don't ## CX90: Rethinking, Redesigning and Reimplementing the Groupon User Experience DevFeed: [CX90: Rethinking, Redesigning and Reimplementing the Groupon User Experience]() Original publisher: [Read original article]() Author: Avleen Kaur Published: 2021-12-22T16:57:17Z Content type: opinion Language: en Sources: [Groupon Engineering -- Medium]() Topics: [User Experience](), [Web](), [browsers](), [Mobile](), [modules]() Tags: [agile](), [browsers](), [interface](), [mobile](), [modules](), [product](), [product-analytics](), [user-experience](), [user-interface-design](), [web]() ### AI overview This article describes Groupon's CX90 project, a 90-day effort to redesign and reimplement the user experience. The team changed the interface from a deal-focused feed to a category-focused experience across the Home, Browse, and Search pages for desktop and mobile browsers. ### Source excerpt All Grouponers who haven't been living under a rock this past year probably know what CX90 is. But for the few that missed out on this extraordinary project and the rest of the world, CX90 was a huge effort across the Product and Engineering Teams at Groupon where all of us came together to redesign our user experience and interface within a period of 90 days. It was an "all hands on deck" situation and probably one of the greatest collaborations we've seen across the organization where people from different backgrounds and areas of expertise joined forces to transform a simple vision into reality. Why? The old Groupon interface focused primarily on a deal feed that allowed our users to scroll through the entire depth of our inventory either for all types of deals or for specific categories of interest. We soon realized that our users were missing out on all the wonderful opportunities we had to offer. We wanted to help them discover new experiences and "Grab Life by the Groupon" by going out of their comfort zones. Our new interface solved exactly this problem. We changed our feed from being deal focused to category focused in order to understand user intent and preferences, expose them to the breadth of the inventory that we had to offer, and to help them explore experiences they could potentially enjoy if only they took a chance. And to be honest, we were also long overdue for a design refresh. 😅 What? Since I'm a part of Web Search and Discovery team, also internally known as Ion, I'll focus on my own experiences and that of my team. We were responsible for reimplementing the Groupon Home, Browse and Search pages both for desktop and mobile browsers. For the Homepage, we transitioned from a plain deal feed and deal carousels to bigger and bolder modules with engaging text and images that highlighted the different categories from our inventory, allowing users to explore specific deals that might interest them instead of aimlessly scrolling through a generic feed. ## Building Felix, the Design System for Groupon DevFeed: [Building Felix, the Design System for Groupon]() Original publisher: [Read original article]() Author: Junmin Liu Published: 2021-12-20T16:23:45Z Content type: article Language: en Sources: [Groupon Engineering -- Medium]() Topics: [Design system](), [Development]() Tags: [design-system](), [design-systems](), [designer](), [developer](), [devops]() ### AI overview Groupon describes building Felix, a design system intended to improve collaboration between design and engineering teams. The initial implementation uses reusable components and design tokens to support Groupon and LivingSocial across web and mobile apps. ### Source excerpt -- Building Felix, the Design System for Groupon Several new features have been released on Groupon.com recently, such as the QR code in the navigation bar to download the app, and a banner carousel to display multiple banner messages within a single view. In the past, similar product features might take 2-3 sprints to complete, but now all of these features are developed and tested in a single sprint. The reason why we can finish the development and testing efficiently is mainly credited to Felix, the design system we're building for Groupon. What is a design system? Design Systems are not new to the world of product development. There are many successful cases in the industry, such as Google's Material Design, Microsoft's Fluent Design System, IBM's Carbon, Adobe's Spectrum, and so on. With their design systems, these companies have successfully changed the way they design and develop their software products by establishing a set of reusable components and usage guidelines to inform design and development. The result? Delivering faster and better product experiences to our customers, at scale. Felix, the early days In 2020, Groupon's consumer experience design and engineering teams began partnering to bring to life a comprehensive design system-now named Felix-to be used throughout Groupon's many brands, platforms, and products. Initially, I believed a design system to be simply a set of design specifications and component libraries -- an unoriginal concept. After diving into the architectural design and project management of Felix's development with Lead Product Designer, Michelle Witkowski, I realized Felix's true potential: a fundamental change in collaboration between designers and developers. This year, we have built out the initial implementation of Felix, and several of our development teams have begun leveraging it. Through the use of design tokens within Felix, we have developed themes to support Groupon's multiple brands (Groupon and LivingSocial) and plat ## The Responsibility of Tech: Reflections from Grace Hopper Celebration DevFeed: [The Responsibility of Tech: Reflections from Grace Hopper Celebration]() Original publisher: [Read original article]() Author: Nida Pervez Published: 2021-11-11T15:33:52Z Content type: opinion Language: en Sources: [Groupon Engineering -- Medium]() Topics: [Development](), [Google](), [Artificial Intelligence]() Tags: [conference](), [ethics](), [google](), [product-development](), [reflections](), [tech](), [tech-industry](), [women-in-tech]() ### AI overview A reflection on Grace Hopper Celebration sessions about ethics, responsibility, gender inequality, and disability in technology. The author considers how technology companies and product teams should support women and take responsibility for the effects of their work. ### Source excerpt The end of September was marked by one of the most exciting events for women in technology: Grace Hopper Celebration of Women in Computing (GHC). For those who do not know, GHC is a yearly conference that highlights women's contributions/involvement in different aspects of the tech industry. In a field that is overwhelmingly dominated by men, it's important to acknowledge the feats of women which otherwise might be drowned out. The week-long conference is packed with speakers, panels, interactive sessions, networking events, and a career fair. In my college days most of my attention admittedly was on the career fair -- but this year I had the opportunity to attend virtually through Groupon, a platinum sponsor of the conference. This year's theme was #DareTo, and I saw that play out in many of the talks, but the sessions that impacted me the most dared me to take responsibility for tech I produced. Three talks that especially left an impression on me were Dr.Timnit Gebru's keynote, Integrating Ethics and Responsibility in the Product Development Cycle by Kathy Pham, and Disability is an Opportunity for Innovation by Haben Girma. Each speaker speaks on the effect of taking responsibility (or lack thereof) for one's technological feats. Dr.Gebru's talk highlighted how large tech corporations can take advantage of freedom for the sake of technological "advancement". In her talk, Gebru explained in detail about her dismissal from the co-lead position of Google's ethical AI team. There were two main aspects to this song that I understood. One being the sexism she faced and the other was the blatant unethical behavior of Google. Gebru addresses the inequality between men and women in the workforce at Google and highlighted that she was fired for doing the right thing, whereas many of her male counterparts could do ethically wrong things and still get by. As a company that is an Emerald sponsor (highest degree of sponsorship) of GHC, she asked what kind of message does that ## Pinion -- The Load Framework Part-2 DevFeed: [Pinion -- The Load Framework Part-2]() Original publisher: [Read original article]() Author: Saurabh Jain Published: 2021-10-29T16:50:24Z Content type: article Language: en Sources: [Groupon Engineering -- Medium]() Topics: [data lake](), [Framework](), [Data Quality](), [Transactions](), [Streaming](), [Open Source]() Tags: [acid](), [audit](), [aws](), [cloud](), [data-validation](), [delta-lake](), [deltalake](), [hdfs](), [logging](), [optimization](), [reporting](), [s3](), [schema](), [science](), [spark](), [streaming](), [transactions]() ### AI overview This second post in the Pinion -- The Load Framework series explains how Pinion extends Apache Delta Lake APIs for slowly changing dimension operations. It covers data validation, compaction, auditing, streamlined logging, and chained APIs, and introduces Delta Lake capabilities such as ACID transactions, schema enforcement, batch and streaming interfaces, and time travel. ### Source excerpt Pinion -- The Load Framework Part-2 This post is the 2nd part of the "Pinion -- The Load Framework" series. In case you have not read the 1st post, you can read it here. In this post, we are going to cover the following topics. How does Pinion use Delta Lake for SCD operations? Small file problem with Delta Lake and its resolution. Before we dive into the topics of this post, let's look at the definition of DeltaLake to set the context right. Apache Delta Lake - Apache Delta Lake is an open-source framework that enables the addition of ACID transactions support to a new data lake or an existing data lake created on top of S3, GCS, and HDFS. In addition to this, it provides other features such as scalable metadata handling, unified interface for both batch and streaming application, schema enforcement, time travel, and a rich interface of APIs to enable complex use cases like change-data-capture (CDC) and slowly-changing-dimension (SCD) operations. To keep the post concise and to the point, I won't go into much detail here about Delta Lake, since there is already great documentation available about it, that you can read it here. How does Pinion use Delta Lake for SCD operations? - Apache Delta Lake provides a rich set of APIs to handle slowly-changing dimensions, however, those APIs were not enough alone to build the features that we want to have in The Pinion Framework. So, we decided to enrich the APIs provided by Delta Lake by adding the following features to it: Data Validation Compaction Audit Streamlined logging infrastructure to make the data engineer's life easier during debugging of a failed job Chained APIs Let's dive a little further into the features that we had listed above. Data Validation -- By default, schema enforcement is enabled in Pinion for all the APIs where we have a need of inserting rows from source data(LRFs) into the target table. In case of a schema mismatch, Pinion raises an error and stops processing of further stages. It ensures the data a ## 3 Guidelines to Reduce Implementation Flaws DevFeed: [3 Guidelines to Reduce Implementation Flaws]() Original publisher: [Read original article]() Author: Pratik Shekhar Published: 2021-09-03T17:06:05Z Content type: opinion Language: en Sources: [Groupon Engineering -- Medium]() Topics: [Code quality](), [coding](), [implementation](), [Testing](), [Unit testing]() Tags: [code-quality](), [coding](), [coding-style](), [function](), [google](), [implement](), [software-development](), [team](), [tech](), [test](), [testing](), [unit-testing]() ### AI overview The article presents coding guidelines intended to reduce implementation flaws. It emphasizes code quality, readability, team practices, breaking functions into smaller single-purpose units, and making code easier to test and reuse. ### Source excerpt 3 Rules/Guidelines to Reduce Implementation Flaws When I hear the word "rule", I think back to grade school. "Be quiet when the teacher is talking." "No interrupting, you must raise your hand with a question." But what makes this different than following a set of guidelines? There was always that one kid who never listened to the rules. In coding, we may talk about rules, whereas in reality, we're following some sort of guideline. Think of guidelines as a scene from the Ghostbusters movie when Bill Murray told Sigourney Weaver that he never sleeps with a possessed woman. Then she kissed him and he said, "Actually, this is more of a guideline than a rule." A standard guideline is important because it helps improve the existing code quality, a guideline also makes it more readable and also somewhere sets the team culture. Over the last few years of gaining more experience in coding, rules have changed into guidelines; however, there are three top guidelines I always follow. After being curious if other colleagues follow any specific coding guidelines, I reached out to several within Groupon and companies like Google, Morgan Stanley, Apple, Lutron, L Brands, Capitol One, GoPuff, and Vertex Inc. Specifically, I asked if they follow any guidelines that reduce implementation flaws. To not much surprise, we all follow similar guidelines. Take a moment and ask yourself, do you have any specific rules or guidelines? We may all have similar guidelines that we follow, mine are listed below- Guideline 1 -> Break down your functions My first rule is merely following the single responsibility principle. A function should be responsible for doing one specific thing. If a function has too many responsibilities, then it should be broken down into multiple smaller functions. Its logic is limited in scope, which makes it easier to implement and test its end-to-end functionality. A friend of mine once said that "If the function you end up unit testing requires tests for multiple separat