# spark-dataframes

Published articles for spark-dataframes.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## My Glue Job Read Half a CSV From a Strongly Consistent S3

DevFeed: [My Glue Job Read Half a CSV From a Strongly Consistent S3](<https://devfeed.tech/articles/my-glue-job-read-half-a-csv-from-a-strongly-consistent-s3-64695.md>)

Original publisher: [Read original article](<https://hackernoon.com/my-glue-job-read-half-a-csv-from-a-strongly-consistent-s3?source=rss>)

Author: Kiran

Published: 2026-10-04T19:13:24Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [AWS Glue](<https://devfeed.tech/topics/aws-glue.md>), [CSV](<https://devfeed.tech/topics/csv.md>)

Tags: [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-glue](<https://devfeed.tech/tags/aws-glue.md>), [csv](<https://devfeed.tech/tags/csv.md>), [csv-processing](<https://devfeed.tech/tags/csv-processing.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [object-storage](<https://devfeed.tech/tags/object-storage.md>), [s3](<https://devfeed.tech/tags/s3.md>), [spark-dataframes](<https://devfeed.tech/tags/spark-dataframes.md>)

### AI overview

A Glue job returned incomplete CSV results from an S3 object that was overwritten in place. The investigation could not establish a single root cause, but identified two plausible mechanisms: Spark may have planned a read using the old object length before fetching a newer version, or a retry may have crossed object versions. Giving each run a separate partition stopped the short reads.

### Source excerpt

A Glue job silently dropped records from a 6 MB CSV. Strong S3 consistency made the real cause harder to find, not easier.