# groupby

A data-processing technique that divides rows into groups and applies aggregations or other operations to each group.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Reading Postgres EXPLAIN ANALYZE when the planner expects 300 rows and gets 15,000

DevFeed: [Reading Postgres EXPLAIN ANALYZE when the planner expects 300 rows and gets 15,000](<https://devfeed.tech/articles/reading-postgres-explain-analyze-when-the-planner-expects-300-rows-and-gets-15-000-136913.md>)

Original publisher: [Read original article](<https://hackernoon.com/reading-postgres-explain-analyze-when-the-planner-expects-300-rows-and-gets-15000?source=rss>)

Author: Anatolii Zhadan

Published: 2026-11-17T06:40:19Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [JOIN](<https://devfeed.tech/topics/join.md>), [groupby](<https://devfeed.tech/topics/groupby.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [amd-epyc](<https://devfeed.tech/tags/amd-epyc.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [arm](<https://devfeed.tech/tags/arm.md>), [b-tree](<https://devfeed.tech/tags/b-tree.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backend-development](<https://devfeed.tech/tags/backend-development.md>), [batch](<https://devfeed.tech/tags/batch.md>), [batches](<https://devfeed.tech/tags/batches.md>), [cache](<https://devfeed.tech/tags/cache.md>), [clock](<https://devfeed.tech/tags/clock.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cost](<https://devfeed.tech/tags/cost.md>), [count](<https://devfeed.tech/tags/count.md>), [database](<https://devfeed.tech/tags/database.md>), [epyc](<https://devfeed.tech/tags/epyc.md>), [explain](<https://devfeed.tech/tags/explain.md>), [explain-analyze](<https://devfeed.tech/tags/explain-analyze.md>), [node](<https://devfeed.tech/tags/node.md>), [performance](<https://devfeed.tech/tags/performance.md>), [plan](<https://devfeed.tech/tags/plan.md>), [planner](<https://devfeed.tech/tags/planner.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [programming](<https://devfeed.tech/tags/programming.md>), [query-optimization](<https://devfeed.tech/tags/query-optimization.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

Start at the lowest plan node where estimated and actual rows diverge: in the example, PostgreSQL expected 300 Lisbon users but found 15,000, leading to 15,000 index lookups and a sort spill. That Nested Loop was faster in memory (159 ms versus 344 ms for a Hash Join) but slower from disk (2.15 s versus 455 ms), showing why plain queries should be timed warm and cold before comparing plans. Read plans bottom-up, account for per-loop row and time averages by multiplying by loops, and subtract child times to locate work. `Buffers` read counts mean pages requested from the OS, not necessarily disk; `track_io_timing` can show read waits. `Disk:` and multi-batch hashes indicate spills, whose cost depends on the serving machine and memory settings. `EXPLAIN ANALYZE` adds measurement overhead: in the reported tests it slowed `count(*)` 2.4-3.0 times, while `TIMING OFF` retains rows, loops and buffers but still has overhead. Wrap analyzed writes in a transaction and roll them back; their work and sequence increments can persist.

### Source excerpt

A bad plan starts at the lowest node where expected and actual rows split, and every node above it is built on that wrong number.

## CALQA: Conformal Screening of Forecast-Ensemble Configurations from One Mergeable State

DevFeed: [CALQA: Conformal Screening of Forecast-Ensemble Configurations from One Mergeable State](<https://devfeed.tech/articles/calqa-conformal-screening-of-forecast-ensemble-configurations-from-one-mergeable-state-113635.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.13249>)

Author: Xiufeng Liu, Ruyu Liu, Yusen Liu, Huan Huo, Per Sieverts Nielsen

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [groupby](<https://devfeed.tech/topics/groupby.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [polynomials](<https://devfeed.tech/topics/polynomials.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [databases](<https://devfeed.tech/tags/databases.md>), [faster](<https://devfeed.tech/tags/faster.md>), [forecast](<https://devfeed.tech/tags/forecast.md>), [greedy](<https://devfeed.tech/tags/greedy.md>), [retail](<https://devfeed.tech/tags/retail.md>), [screen](<https://devfeed.tech/tags/screen.md>), [search](<https://devfeed.tech/tags/search.md>), [smart](<https://devfeed.tech/tags/smart.md>), [split](<https://devfeed.tech/tags/split.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

CALQA screens large searches over forecast-ensemble weights using a fixed-size, mergeable integer aggregate built with one GROUP BY. It stores histograms of normalized block scores at a net of weight vectors, supports addition and deletion by subtraction, and scores candidates without rereading rows. Its score is at least the exact split-conformal shift and exceeds it by at most the net distance plus a grid gap; searched-weight coverage is stated for i.i.d. blocks with union-bound slack. The system ranks candidates, then exactly rechecks the top 100. Across five public panels, it achieved 99.95% of exact search's held-out Winkler score at 1.3-59x speedup; on two roughly 10^8-row panels, it achieved 99.98-100.01% at 44-54x speedup and outperformed sample screens in regret at lower total time. Exact-shift deployments still need a held-out coverage check. The state is not beneficial for small or greedy searches, off-net weights with ample memory, or full twelve-member support.

### Source excerpt

arXiv:2610.13249v2 Announce Type: replace Abstract: AutoML systems fit ensemble weights over model libraries; a search can score, per category, market or time pane, thousands of weightings of the quantile-averaged (Vincentized) ensemble for the narrowest conformally calibrated interval; exact calibration scores a group's rows per candidate. We present CALQA, a fixed-size integer aggregate per group and pane that stores histograms of normalised block scores at a net of weight vectors. One GROUP BY builds it; it merges by addition, retracts deletions by subtraction and scores any candidate without reading a row. We prove that its score never falls below the exact split-conformal shift and exceeds it by at most the net distance plus a grid gap; that coverage carries over to each fixed weight vector and, for i.i.d. blocks and up to a union-bound slack, to a searched weight vector deployed with the state's shift; and how small any such state can be. The state is thus a screen: it ranks all candidates, and exact recomputation re-checks the top 100. On five public panels (604 groups, 10,000 candidates each) this two-stage plan matches exact search on held-out blocks (99.95% of its Winkler score), 1.3-59 times faster. On two real panels of about $10^8$ rows (smart meters, retail sales) it is 44-54 times faster at 99.98-100.01% of that score and beats sample screens with the same re-check in regret at less total time. Picks deployed with their exact shift need a held-out coverage check. The state does not pay off for small or greedy searches, off-net weights at ample memory or full twelve-member support.

## Snowpark Python Supports Native to\_polars()

DevFeed: [Snowpark Python Supports Native to\_polars()](<https://devfeed.tech/articles/snowpark-python-supports-native-to-polars-106578.md>)

Original publisher: [Read original article](<https://medium.com/snowflake/snowpark-python-supports-native-to-polars-b84e03721b5d?source=rss----34b6daafc07---4>)

Author: Qinyi Ding

Published: 2026-10-29T17:22:34Z

Content type: release

Language: en

Sources: [Snowflake Builders](<https://devfeed.tech/sources/snowflake-builders.md>)

Topics: [Python](<https://devfeed.tech/topics/python.md>), [groupby](<https://devfeed.tech/topics/groupby.md>), [Parquet files in Amazon S3](<https://devfeed.tech/topics/parquet-files-in-amazon-s3.md>), [apache-arrow](<https://devfeed.tech/topics/apache-arrow.md>), [pandas](<https://devfeed.tech/topics/pandas.md>)

Tags: [data-processing](<https://devfeed.tech/tags/data-processing.md>), [etl](<https://devfeed.tech/tags/etl.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [polars](<https://devfeed.tech/tags/polars.md>), [polars-dataframe](<https://devfeed.tech/tags/polars-dataframe.md>), [python](<https://devfeed.tech/tags/python.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [snowpark](<https://devfeed.tech/tags/snowpark.md>)

### AI overview

Snowpark Python's `DataFrame.to_polars()` is generally available in Snowpark Python 1.55.3 with Polars 1.0 or later. It executes a Snowpark query and returns a fully materialized Polars DataFrame; filter and select data in Snowpark before conversion because later Polars work runs in Python and the result must fit in memory. Parquet is the default transport: Snowpark stages query results as Parquet files and reads them in parallel, which can speed large transfers, especially in stored procedures. For timezone-aware timestamps or full timestamp and floating-point precision, use `to_polars(transport="arrow")`: Parquet lacks support for `TIMESTAMP_LTZ` and `TIMESTAMP_TZ`, rounds `TIMESTAMP_NTZ` to milliseconds, and reduces `FLOAT` to `float32`. The example filters out South in Snowflake, then groups the remaining rows in Polars to yield North revenue of 320.

### Source excerpt

Photo by Annie Spratt on Unsplash Snowpark Python's DataFrame.to_polars() is now generally available, bringing Snowflake query results into a Polars DataFrame with a single call. Its default Parquet transport reads files in parallel to accelerate large data transfers, connecting data preparation in Snowflake with analytics and data processing in Polars. Why it matters Snowpark already supports pandas through to_pandas(). With to_polars(), we are extending that interoperability to developers who use Polars in their Python workflows. You can use Snowpark to filter, join, and aggregate large datasets in Snowflake, then return the results as a Polars DataFrame for further processing. This lets you combine Snowflake's data processing with your existing Polars application, using the same Snowpark session. How to use it With a configured Snowpark session and Polars installed, call to_polars() on a Snowpark DataFrame. Here is a small example: import polars as pl from snowflake.snowpark.functions import col snowpark_df = session.create_dataframe( [["North", 120], ["South", 80], ["North", 200]], schema=["REGION", "REVENUE"], ).filter(col("REVENUE") >= 100) polars_df = snowpark_df.to_polars() # Continue processing with Polars. summary = polars_df.group_by("REGION").agg( pl.col("REVENUE").sum() ) print(summary) In this example, calling to_polars() executes the Snowpark query, filtering out rows with revenue above 100 in Snowflake and returning the remaining rows as an eager Polars DataFrame. Polars then groups those rows by region and sums their revenue, producing one result: North with a total of 320. South is excluded by the filter. Note DataFrame.to_polars() uses Parquet transport by default. Snowpark unloads query results to the session stage as Parquet files and reads them in parallel into Polars. This can speed up large, transfer-heavy workloads, especially in Snowpark Python stored procedures. Use to_polars(transport="arrow") when you need to preserve timezone-aware time

## Introducing group activity pages

DevFeed: [Introducing group activity pages](<https://devfeed.tech/articles/introducing-group-activity-pages-86565.md>)

Original publisher: [Read original article](<https://www.w3.org/blog/2026/introducing-group-activity-pages/>)

Author: Jean-Guilhem Rouel, Senior Principal DevOps Engineer

Published: 2026-10-14T10:28:00Z

Content type: news

Language: en

Sources: [W3C](<https://devfeed.tech/sources/w3c-blog.md>)

Topics: [Website](<https://devfeed.tech/topics/website.md>), [groupby](<https://devfeed.tech/topics/groupby.md>), [software-development](<https://devfeed.tech/topics/software-development.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [blogs](<https://devfeed.tech/tags/blogs.md>), [pages](<https://devfeed.tech/tags/pages.md>), [systems](<https://devfeed.tech/tags/systems.md>), [that-s](<https://devfeed.tech/tags/that-s.md>), [w3c](<https://devfeed.tech/tags/w3c.md>), [work](<https://devfeed.tech/tags/work.md>)

### AI overview

W3C has introduced beta activity pages that bring a group's recent work together month by month, including participation, meetings, publications, reports and GitHub activity. The pages show the latest three months, offer yearly archives and filters, and may later add faster GitHub updates, more data sources and an API. W3C is seeking feedback on the pages and their future uses.

### Source excerpt

W3C groups now have an Activity page that shows, month by month, their charters, publications, meetings, participants and GitHub work in one place. The pages are in beta, and we would like your feedback.

## GIS in Go

DevFeed: [GIS in Go](<https://devfeed.tech/articles/gis-in-go-22178.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2021/11/gis-in-go.html>)

Published: 2021-11-01T00:00:00Z

Content type: article

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [CSV](<https://devfeed.tech/topics/csv.md>), [groupby](<https://devfeed.tech/topics/groupby.md>), [timezone](<https://devfeed.tech/topics/timezone.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [plotting](<https://devfeed.tech/topics/plotting.md>)

Tags: [ardan-labs](<https://devfeed.tech/tags/ardan-labs.md>), [blog](<https://devfeed.tech/tags/blog.md>), [csv](<https://devfeed.tech/tags/csv.md>), [data](<https://devfeed.tech/tags/data.md>), [gis](<https://devfeed.tech/tags/gis.md>), [go](<https://devfeed.tech/tags/go.md>), [go-programming](<https://devfeed.tech/tags/go-programming.md>), [golang](<https://devfeed.tech/tags/golang.md>), [programming](<https://devfeed.tech/tags/programming.md>)

### AI overview

The article outlines a Go workflow for turning a 740-row jogging track CSV into a map: parse timestamp, latitude, longitude, and height fields; convert UTC times to a supplied time zone; group rows by a time interval and average each field to reduce points; then generate HTML using Leaflet. The excerpt describes CSV loading through an io.Reader for flexibility and testing, and uses time.Parse with a custom time unmarshaler. It does not include the complete resampling implementation or the map-generation details.

### Source excerpt

Introduction You are jogging and want to show off your route to your friends. Let's imagine the data you have for your route is a CSV file in the following format: Listing 1: track.csv time,lat,lng,height 2015-08-20 03:48:07.235,32.519585,35.015021,136.1999969482422 2015-08-20 03:48:24.734,32.519606,35.014954,126.5999984741211 2015-08-20 03:48:25.660,32.519612,35.014871,123.0 2015-08-20 03:48:26.819,32.519654,35.014824,120.5 2015-08-20 03:48:27.828,32.519689,35.014776,118.9000015258789 2015-08-20 03:48:29.720,32.519691,35.014704,119.9000015258789 2015-08-20 03:48:30.669,32.519734,35.014657,120.9000015258789 Listing 1 shows the first few lines of track.csv. Each line contains a time stamp in UTC, latitude, longitude and height above sea level in meters.

## Effective Kotlin Item 56: Consider using groupingBy instead of groupBy

DevFeed: [Effective Kotlin Item 56: Consider using groupingBy instead of groupBy](<https://devfeed.tech/articles/effective-kotlin-item-56-consider-using-groupingby-instead-of-groupby-39280.md>)

Original publisher: [Read original article](<https://kt.academy/article/ek-grouping>)

Published: 2021-10-11T00:00:00Z

Content type: tutorial

Language: en

Sources: [Kt. Academy](<https://devfeed.tech/sources/kt-academy.md>)

Topics: [groupby](<https://devfeed.tech/topics/groupby.md>), [Kotlin](<https://devfeed.tech/topics/kotlin.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [performance-optimization](<https://devfeed.tech/topics/performance-optimization.md>), [Collections](<https://devfeed.tech/topics/collections.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [collections](<https://devfeed.tech/tags/collections.md>), [groupby](<https://devfeed.tech/tags/groupby.md>), [kotlin](<https://devfeed.tech/tags/kotlin.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [standard-library](<https://devfeed.tech/tags/standard-library.md>), [workshop-learning-programming](<https://devfeed.tech/tags/workshop-learning-programming.md>)

### AI overview

This tutorial explains how Kotlin's groupBy and groupingBy functions organize elements from an iterable by key. It describes groupBy as the simpler option that creates collections for each key, while groupingBy can reduce work and memory use in performance-critical code by processing groups through operations such as counting and aggregation.

### Source excerpt

What Grouping is, and how groupingBy instead of groupBy can be a performance optimization.