# Feature Engineering Feature engineering is the process of transforming, selecting, and creating data features for machine learning models. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## 🍔🧠 Pinterest's Fix for the Hardest Problem in ML Infra DevFeed: [🍔🧠 Pinterest's Fix for the Hardest Problem in ML Infra]() Original publisher: [Read original article]() Author: Alexandre Zajac Published: 2026-09-14T15:31:30Z Content type: article Language: en Sources: [Hungry Minds]() Topics: [recommendation systems](), [Feature Engineering](), [Inference](), [Streaming](), [configuration](), [Framework](), [data](), [Low Latency]() Tags: [batch](), [configuration](), [data](), [feature-engineering](), [low-latency](), [ml](), [pinterest](), [ranking](), [recommendation-systems](), [streaming]() ### AI overview Pinterest redesigned its user-sequence platform for ranking, retrieval, and recommendation systems by defining signals once and instantiating them consistently across streaming, batch, and serving workloads. The approach uses Python configuration with validated schemas, a shared execution engine, cooperating streaming and batch paths, and columnar time-partitioned storage to improve freshness, completeness, consistency, and operational efficiency. ### Source excerpt PLUS: OpenAI agents beat math 🧮, Test techniques for agents ⚡, Postgres survival guide 📖 ## Planetary prediction engine: Automating global models via Earth AI DevFeed: [Planetary prediction engine: Automating global models via Earth AI]() Original publisher: [Read original article]() Published: 2026-08-27T17:37:00Z Content type: article Language: en Sources: [The latest research from Google]() Topics: [Earth AI](), [Artificial Intelligence](), [data-engineering](), [Google](), [Machine learning](), [AI Development](), [Feature Engineering](), [Model Development]() Tags: [agents](), [ai](), [analytics](), [data-engineering](), [earth-ai](), [evaluation](), [feature-engineering](), [generative-ai](), [google](), [insights](), [machine-intelligence](), [mapping](), [research](), [training](), [validation](), [workflow](), [workflows]() ### AI overview Google Research introduces the Planetary Prediction Engine, an experimental Earth AI capability that autonomously performs geospatial data discovery, cleanup, feature engineering, model training, evaluation, and report generation from natural-language queries. The system targets applications including public health, food security, environmental risk, and socioeconomic analysis, reducing the stated workflow from weeks of manual data engineering to minutes. ### Source excerpt Earth AI ## Data Engineering Weekly #283 DevFeed: [Data Engineering Weekly #283]() Original publisher: [Read original article]() Author: Ananth Packkildurai Published: 2026-08-17T02:59:40Z Content type: article Language: en Sources: [Data Engineering Weekly]() Topics: [data-engineering](), [AI, ML & Data Engineering](), [Data Quality](), [Microservice](), [Apache Spark](), [Kafka](), [Feature Engineering](), [audit trail](), [Protocol (disambiguation)]() Tags: [ai](), [anthropic](), [article](), [data-engineering](), [data-quality](), [engineering](), [feature-engineering](), [kafka](), [newsletter](), [pipelines](), [services](), [testing]() ### AI overview Data Engineering Weekly #283 is a newsletter covering data platform fundamentals, multiagent system coordination, payments platform data contracts, financial data quality, declarative data engineering, and cost-efficient export workloads. ### Source excerpt The Weekly Data Engineering Newsletter ## Introducing TabFM: A zero-shot foundation model for tabular data DevFeed: [Introducing TabFM: A zero-shot foundation model for tabular data]() Original publisher: [Read original article]() Published: 2026-06-30T10:26:00Z Content type: article Language: en Sources: [The latest research from Google]() Topics: [foundation-models](), [datasets](), [Machine learning](), [Google](), [Feature Engineering](), [Hyperparameter optimization](), [Language models](), [BigQuery](), [GitHub](), [hugging face](), [AI Inference]() Tags: [bigquery](), [classification](), [data](), [data-management](), [feature-engineering](), [github](), [google](), [hugging-face](), [hyperparameter-optimization](), [machine-intelligence](), [model](), [product](), [zero-shot]() ### AI overview Google Research introduces TabFM, a zero-shot foundation model for tabular-data classification and regression. It frames prediction as in-context learning, reducing the need for dataset-specific training, hyperparameter optimization, and feature engineering, with availability through Hugging Face, GitHub, and BigQuery. ### Source excerpt Data Management ## Making User-Sequence Data More Cost-Efficient, Faster, and Easier to Use DevFeed: [Making User-Sequence Data More Cost-Efficient, Faster, and Easier to Use]() Original publisher: [Read original article]() Author: Pinterest Engineering Published: 2026-05-21T16:01:00Z Content type: article Language: en Sources: [Pinterest Engineering Blog - Medium]() Topics: [data](), [recommendation systems](), [datasets](), [Machine learning](), [Embeddings](), [Feature Engineering](), [Inference](), [Latency](), [systems]() Tags: [data](), [data-infrastructure](), [datasets](), [embeddings](), [engineering](), [feature-engineering](), [inference](), [latency](), [machine-learning](), [pinterest](), [production](), [recommendation-system](), [recommendation-systems](), [retrieval](), [systems](), [train](), [transformers]() ### AI overview Pinterest describes a redesign of its user-sequence platform for ranking, retrieval, and recommendation workloads. The article explains how enriched event sequences support training datasets, offline analysis, online inference, and latency-sensitive production use cases, with goals of reducing cost, improving extensibility, and simplifying debugging. ### Source excerpt Authors (listed alphabetically) Ads Feature Engineering Infra team: Ajay Venkatakrishnan, Le Zhang Core ML Infra team: Eric Shang, Pihui Wei ML Data team: Connor Votroubek, Yi He User Understanding team: Camilo Munoz, Simin Li If you work on ranking, retrieval, or recommendation systems, you've probably asked for some version of the same thing: "Give me the last N meaningful actions this user took, with the right enrichments, in a format that's easy to train and serve ML models." On paper, that sounds simple. In practice, "user sequences" often become one of the most expensive and fragile parts of the ML data stack. They end up powering everything from training datasets to offline analysis and online inference, so they need to be fresh and complete at the same time. They must remain consistent as you add new events and enrichments. And they have to do all of this while serving latency-sensitive production workloads. This article walks through how we redesigned our user-sequence platform to make these sequences cheaper to run, faster to extend, and easier to debug, while still supporting demanding production use cases. What We Mean by "User Sequence" In this context, a user sequence is an ordered list of recent, relevant events for a user, along with the enrichments (signals) attached to each event. Here, enrichments mean all the extra signals we attach to raw events, so they're useful for models: embeddings (for example, Pin or query representations), contextual features (such as surface, device, or country), and derived attributes or counters that describe how the user interacted with a piece of content over time. A concrete example helps. Imagine a sequence made up of the last 500 engagements a user had with Pinterest Pins. Each event in that sequence might carry a timestamp, an action type, the surface where the action occurred, and a handful of embedding features or categorical attributes. As a data primitive, user sequences are powerful. They capture temporal b ## 5 Career Myths From Women in Engineering DevFeed: [5 Career Myths From Women in Engineering]() Original publisher: [Read original article]() Author: Emily Majewski Published: 2026-03-17T13:28:39Z Content type: opinion Language: en Sources: [Two Sigma Engineering]() Topics: [Job](), [Software Engineering](), [engineering-culture](), [Amazon Web Services](), [Feature Engineering](), [Finance](), [Users]() Tags: [advice](), [career](), [careers](), [engineering](), [skills](), [software-engineer](), [team](), [users]() ### AI overview The article examines five common career myths through the experiences of three Two Sigma engineering leaders. Their nonlinear careers illustrate how lateral moves, detours, and varied roles can build useful skills and inform later work in software engineering, architecture, and technical leadership. ### Source excerpt The post 5 Career Myths From Women in Engineering appeared first on Two Sigma. ## Engineering Features at Scale: Inside the Darwin Feature Store DevFeed: [Engineering Features at Scale: Inside the Darwin Feature Store]() Original publisher: [Read original article]() Author: Dream Blog Published: 2026-01-27T15:38:25Z Content type: article Language: en Sources: [Dream11 Engineering]() Topics: [Feature Engineering](), [Low Latency](), [Apache Cassandra](), [Machine learning](), [Open Source]() Tags: [cassandra](), [data](), [engineering](), [feature-engineering](), [feature-store](), [inference](), [kafka](), [latency](), [machine-learning](), [offline](), [open-source](), [pipelines](), [software-engineering](), [tech]() ### AI overview Dream Horizon's Darwin Feature Store is an open-source, low-latency feature platform designed to support real-time machine learning and offline training at Dream11. The article describes the challenges that led to its development, including batch-only pipelines, manual schema changes, and scaling issues. It reports serving more than 200 million feature requests per minute with p99 read latency below 5 milliseconds and no production incidents over the preceding year. ### Source excerpt By Mohit Jain and Ujjwal Bagrania Dream Horizon, our open-source effort to make Dream11's battle-tested tech available to every developer, brings you the Darwin Feature Store -- a unified, low-latency feature platform built to power real-time ML at scale, and shaped to help teams build, manage, and trust features in production. Explore the Darwin Feature Store here. At Dream11, data and ML have always powered how millions of users experience sports in real time. From personalisation and relevance to ensuring efficiency under peak match traffic, ML sits deep in the critical path of the product. But models are only half the story. The real challenge is managing features -- how they're defined, versioned, and accessed consistently across training and live traffic, especially under peak sports pressure. And at the scale we operate at, feature access doesn't mean the occasional lookup; it means hundreds of millions of requests per minute, spanning both real-time inference and offline training pipelines. As Dream11 grew, feature engineering stopped being something we could manage with pipelines and tables, and became infrastructure that everything else depended on. That shift forced us to build the Darwin Feature Store: a system designed not just to tackle sports-scale traffic, but to make feature engineering predictable, reliable, and developer-friendly in production. Today, that translates into serving 200M+ feature requests per minute, delivering p99 read latencies under 5 ms, and doing so reliably through the most demanding live sports moments, without a single production incident over the last year. In this post, we'll walk through how we built the Feature Store, and what it takes to serve features reliably. The Early Days: Batch-Only, Fragile, and Manual Before the Darwin Feature Store existed, our feature engineering was mostly a few pipelines, a lot of Cassandra tables, and an increasing number of engineers building ML models. Here's what the first version looked li ## Platform Thinking: Three Views from Two Sigma Leaders DevFeed: [Platform Thinking: Three Views from Two Sigma Leaders]() Original publisher: [Read original article]() Author: Emily Majewski Published: 2025-10-23T14:49:45Z Content type: article Language: en Sources: [Two Sigma Engineering]() Topics: [data-engineering](), [data](), [Large Language Model](), [Development](), [Feature Engineering](), [Artificial Intelligence](), [BigQuery](), [CI/CD](), [AI research agents](), [Open Source Models & Datasets]() Tags: [ai](), [ai-agents](), [bigquery](), [ci-cd](), [data](), [data-engineering](), [datasets](), [feature-engineering](), [llms](), [open-source](), [platform](), [post]() ### AI overview Two Sigma leaders describe the company's platform-oriented approach to data, technology, and management. The article highlights parallel delivery of raw and curated datasets, BigQuery and CI/CD practices, reusable data contracts, LLM-assisted feature engineering, controls for temporal leakage, and possible uses of multiple AI agents in quantitative research. ### Source excerpt The post Platform Thinking: Three Views from Two Sigma Leaders appeared first on Two Sigma. ## Optimizing AI Code for Performance DevFeed: [Optimizing AI Code for Performance]() Original publisher: [Read original article]() Author: admin Published: 2024-05-27T13:47:23Z Content type: tutorial Language: en Sources: [Posts on Chris Banes]() Topics: [AI Development](), [Optimization](), [Code](), [Feature Engineering](), [data]() Tags: [ai](), [batching](), [code](), [feature-engineering](), [format](), [machine-learning](), [optimization](), [pandas](), [parquet](), [performance](), [uncategorized]() ### AI overview This guide explains techniques for optimizing AI and machine learning code for faster execution and lower computing costs. It covers data loading and preprocessing, algorithm selection, hardware acceleration, distributed training, and batching. ### Source excerpt AI code optimization is about getting almost everything out of your algorithms and models. It's about making sure that your AI projects run smoothly, quickly, and cost-effectively. In this guide, we'll dive into the world of AI code optimization, exploring different techniques and best practices that can help you speed up your machine learning projects. [...] ## Declarative Feature Engineering at PayPal DevFeed: [Declarative Feature Engineering at PayPal]() Original publisher: [Read original article]() Author: Marina Lyan Published: 2023-12-11T15:56:51Z Content type: tutorial Language: en Sources: [PayPal Technology]() Topics: [Feature Engineering](), [Machine learning](), [data](), [Requirements]() Tags: [declarative-programming](), [engineering](), [feature](), [feature-engineering](), [fraud](), [machine-learning](), [paypal](), [production](), [real-time](), [requirements](), [scale]() ### AI overview PayPal describes its declarative, or config-based, feature engineering approach for machine learning. The approach lets data scientists declare feature definitions while execution details are handled separately, helping engineers address scale, predictable time to market, and total cost of ownership. The article introduces feature-cost metrics and strategies for reusing existing features. ### Source excerpt Photo by fabio on Unsplash PayPal supports over 400 million active consumers and merchants worldwide. Every minute there are several thousand payment transactions. To prevent fraud in real-time at such a scale, we need to streamline our ML workflow and feature engineering processes to build strong predictors of behaviors and risk indicators. On top of that, it must be done with consistently predictable Time to Market (TTM) and sustainable Total Cost of Ownership (TCO). What is declarative feature engineering? While the declarative feature engineering term was first introduced by Zipline AirBnB in 2019, we have successfully used this paradigm in PayPal for the last decade, though we know it as config-based feature engineering. The idea is to allow data scientists to write a declaration of what their features look like rather than explicitly specify how to construct them on top of different execution platforms. In this way, the concerns of feature construction and execution are abstracted away from scientists so that engineers can worry about those complexities. This post is the first in a series of two posts that will outline how the declarative feature engineering approach helps our engineers to address scale, TTM, and TCO requirements. Let's start with the definitions of metrics that measure our success. TTM of the ML Feature TTM of the Machine Learning (ML) feature is the length of time from the feature concept until it is released to production. Predictable TTM is a paramount need for overall AI maturation and business-first AI strategies. Cost of the ML Feature To qualify and estimate TCO, we break down the cost of the feature as follows: Layers of feature cost The main insight from the feature cost analysis is that we have to reuse existing features across the teams whenever possible. Otherwise, we just pay twice for every aspect. To approach the above challenges, we separate features into three levels of complexity so we can define tailored strategies for scal ## Pre-processing temporal data made easier with TensorFlow Decision Forests and Temporian DevFeed: [Pre-processing temporal data made easier with TensorFlow Decision Forests and Temporian]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2023-09-11T20:14:00Z Content type: article Language: en Sources: [The TensorFlow Blog]() Topics: [Temporal data](), [Temporian](), [TensorFlow Decision Forests](), [Feature Engineering](), [Python](), [datasets]() Tags: [blog](), [data](), [dataset](), [feature-engineering](), [logs](), [machine-learning](), [python](), [sales](), [temporal-data](), [temporian](), [tensorflow-decision-forests](), [time-sequences](), [time-series](), [transactions](), [user-event]() ### AI overview This article demonstrates preprocessing temporal sales data with Temporian and forecasting weekly sales with TensorFlow Decision Forests. It explains event sets, aggregation, moving sums, indexing, and the trade-off between transactional detail and uniformly sampled time series. ### Source excerpt Posted by Google: Mathieu Guillame-Bert, Richard Stotz, Robert Crowe, Luiz GUStavo Martins (Gus), Ashley Oldacre, Kris Tonthat, Glenn Cameron, and Tryolabs: Ian Spektor, Braulio Rios, Guillermo Etchebarne, Diego Marvid, Lucas Micol, Gonzalo Marín, Alan Descoins, Agustina Pizarro, Lucía Aguilar, Martin Alcala Rubi Temporal data is omnipresent in applied machine learning applications. Data often changes over time or is only available or valuable at a certain point in time. For example, market prices and weather conditions change constantly. Temporal data is also often highly discriminative in decision-making tasks. For example, the rate of change and interval between two consecutive heartbeats provides valuable insights into a person's physical health, and temporal patterns of network logs are used to detect configuration issues and intrusions. Hence, it is essential to incorporate temporal data and temporal information in ML applications. INFO: Temporian is a new open-source Python library for preprocessing and feature engineering temporal data for machine learning applications. It is developed in collaboration between Google and Tryolabs. Check the sister blog post for more details. This blog post demonstrates how to train a forecasting model on transactional data. Specifically, we will show how to forecast the total weekly sales from individual sales records. For the modeling part, we will use TensorFlow Decision Forests as they are well suited to handle temporal data. To feed the transaction data to our model, and to compute temporal specific features, we will use Temporian, a newly released library designed for ingesting and aggregating transactional data from multiple non-synchronized sources. Time series are the most commonly used representation for temporal data. They consist of uniformly sampled values, which can be useful for representing aggregate signals. However, time series are sometimes not sufficient to represent the richness of available data. Instead ## Thoughts from AAAI 2019 DevFeed: [Thoughts from AAAI 2019]() Original publisher: [Read original article]() Author: Loren Puchalla Fiore Published: 2019-05-13T00:00:00Z Content type: opinion Language: en Sources: [Jane Street]() Topics: [Machine Learning & Artificial Intelligence](), [Deep neural networks](), [Feature Engineering]() Tags: [ai](), [conference](), [fairness](), [feature-engineering](), [machine-learning](), [neural-networks]() ### AI overview Jane Street colleagues reflect on themes and projects presented at AAAI 2019, including neural-network interpretability, adversarial weaknesses in interpretation techniques, classical AI, AI for social good, social AI, and fairness in machine learning. ### Source excerpt At Jane Street, for the last several years, we have been increasingly interested in machine learning and its many use cases. This is why it was exciting when earlier this year myself and a few of my colleagues had the opportunity to attend the AAAI 2019 conference. We'd like to take this space to share with you some of the interesting projects and themes we saw at the conference. ## A Peek into Machine Learning at Square DevFeed: [A Peek into Machine Learning at Square]() Original publisher: [Read original article]() Author: Sara Vera Published: 2017-08-04T17:34:16Z Content type: article Language: en Sources: [Square Corner Blog RSS Feed]() Topics: [Machine Learning & Artificial Intelligence](), [datasets](), [Development](), [Feature Engineering](), [Instrumentation]() Tags: [business](), [data](), [data-science](), [dataset](), [development](), [feature-engineering](), [instrumentation](), [machine-learning]() ### AI overview Square describes how it uses machine learning across its business, including internal tools and customer-facing experiences. The article outlines a workflow covering problem definition, data collection and validation, model selection and evaluation, and iterative improvements such as refactoring and feature engineering. ### Source excerpt We'll be posting more about specific projects, methods, and how we use machine learning in our day-to-day work -- stay tuned! For now... ## Tricks and tips for feature engineering DevFeed: [Tricks and tips for feature engineering]() Original publisher: [Read original article]() Author: Mrinal Parekh Published: 2016-07-20T18:32:44Z Content type: tutorial Language: en Sources: [CleverTap]() Topics: [Feature Engineering](), [data]() Tags: [ai-and-ml](), [ai-and-ml-data-and-analytics-data-insights-engineering-data-science](), [data](), [data-and-analytics](), [data-insights](), [data-science](), [engineering](), [feature-engineering](), [tips](), [tricks]() ### AI overview The article explains feature engineering as the creation of meaningful new input variables from existing fields for predictive modeling. It describes how domain expertise, reasoning, intuition, or automation can improve data insights and predictive model performance. ### Source excerpt Predictive modeling is a formula that transforms a list of input fields or variables into some output of interest. Feature The post Tricks and tips for feature engineering first appeared on CleverTap.