# Scraping

Published articles for Scraping.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Claims About RubyGems Caching and RubyDoc Code Execution

DevFeed: [Claims About RubyGems Caching and RubyDoc Code Execution](<https://devfeed.tech/articles/what-a-time-to-be-alive-39007.md>)

Original publisher: [Read original article](<https://tenderlovemaking.com/2026/09/11/what-a-time-to-be-alive/>)

Published: 2026-09-12T00:02:11Z

Content type: opinion

Language: en

Sources: [Aaron Patterson](<https://devfeed.tech/sources/aaron-patterson.md>)

Topics: [rubygems](<https://devfeed.tech/topics/rubygems.md>), [Caching](<https://devfeed.tech/topics/caching.md>), [vulnerability](<https://devfeed.tech/topics/vulnerability.md>), [Docker](<https://devfeed.tech/topics/docker.md>), [Web Scraping](<https://devfeed.tech/topics/web-scraping.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [docker](<https://devfeed.tech/tags/docker.md>), [rubygems](<https://devfeed.tech/tags/rubygems.md>), [scraping](<https://devfeed.tech/tags/scraping.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

An opinion post discusses alleged RubyGems activity involving a caching vulnerability and web scraping. It also describes how published gems can cause RubyDoc.info to run YARD-loaded code in a networked Docker container.

### Source excerpt

Today Reuters and the Wall Street Journal both reported about rogue AI agents at OpenAI attacking RubyGems.org. https://www.rubyhack.ai/ has an amazing writeup, and you should read it. I just wanted to make a quick post about it because it's wild. TL;DR: It seems like OpenAI Bots knew about the RubyGems caching vulnerability, tried to take advantage of it, and at the same time ran some weird web scraping code on RubyDoc.info. Back in May, socket.dev reported about a "GemStuffer Campaign" where someone (I guess OpenAI) was uploading tons of junk gems to RubyGems.org. For some reason, the gems would scrape UK government websites, then repackage the data as gems, and attempt to upload them to RubyGems. I honestly didn't think much about this (or even look into it) until Sydney Von Arx and Spencer Kitts (both co-authors on https://www.rubyhack.ai) contacted me asking about RubyGems. I thought the claims they were making were completely outlandish until I actually read the code in these "GemStuffer" gems. After reading the code in these gems, a couple things stood out to me. YARD Documentation First, the gems leverage YARD documentation to execute arbitrary code on host machines. In most of the examples you'll see a .yardopts file that looks like this: --load ./script.rb README.md lib/**/*.rb Here's a link to an example. If you have YARD installed, and you install this gem, then YARD will load and run whatever is in ./script.rb from inside the gem. I think it's pretty common knowledge that C extensions will execute extconf.rb (so you basically have an RCE vector), but I was surprised to find out that a documentation tool would do that too. Nobody is going to install a gem named slnleaker5 though, so why would this matter? Well, any time a Gem is published RubyDoc.info will download the gem and process the YARD documentation. RubyDoc.info will execute the arbitrary code inside a Docker container. The Docker container still has network access though, so these gems could ha

## Building an AI Agent for LinkedIn Scraping & Candidate Sourcing

DevFeed: [Building an AI Agent for LinkedIn Scraping & Candidate Sourcing](<https://devfeed.tech/articles/building-an-ai-agent-for-linkedin-scraping-candidate-sourcing-39409.md>)

Original publisher: [Read original article](<https://blog.pranshu-raj.in/posts/linkedin-scraping-full/>)

Author: Pranshu Raj

Published: 2026-07-23T00:00:00Z

Content type: tutorial

Language: en

Sources: [Pranshu Raj - blog on backend systems, performance and sidequests](<https://devfeed.tech/sources/pranshu-raj-blog-on-backend-systems-performance-and-sidequests.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [FastAPI](<https://devfeed.tech/topics/fastapi.md>), [Python](<https://devfeed.tech/topics/python.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [GitHub](<https://devfeed.tech/topics/github.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [async](<https://devfeed.tech/tags/async.md>), [fastapi](<https://devfeed.tech/tags/fastapi.md>), [github](<https://devfeed.tech/tags/github.md>), [llms](<https://devfeed.tech/tags/llms.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [python](<https://devfeed.tech/tags/python.md>), [recruiting](<https://devfeed.tech/tags/recruiting.md>), [scraping](<https://devfeed.tech/tags/scraping.md>)

### AI overview

A developer describes building an AI-powered LinkedIn candidate-sourcing agent for a hackathon. The Python and FastAPI application combines LinkedIn and GitHub profile discovery, six-factor candidate scoring, and Llama-powered personalized outreach, using external services for search and LinkedIn data retrieval.

### Source excerpt

How I built an AI agent that scrapes LinkedIn, scores candidates, and generates personalized outreach, a full recruiting sourcing pipeline in Python and FastAPI.

## One honojs file for entire web scraping API

DevFeed: [One honojs file for entire web scraping API](<https://devfeed.tech/articles/one-honojs-file-for-entire-web-scraping-api-39140.md>)

Original publisher: [Read original article](<https://ihatereading.in/t/one-honojs-file-for-entire-web-scraping-api>)

Author: iHateReading

Published: 2026-04-13T16:54:35Z

Content type: tutorial

Language: en

Sources: [iHateReading](<https://devfeed.tech/sources/ihatereading.md>)

Topics: [Hono](<https://devfeed.tech/topics/honojs.md>), [Web Scraping](<https://devfeed.tech/topics/web-scraping.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [backend](<https://devfeed.tech/tags/backend.md>), [blockchain](<https://devfeed.tech/tags/blockchain.md>), [coding](<https://devfeed.tech/tags/coding.md>), [development](<https://devfeed.tech/tags/development.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [github-repository](<https://devfeed.tech/tags/github-repository.md>), [honojs](<https://devfeed.tech/tags/honojs.md>), [ihatereading](<https://devfeed.tech/tags/ihatereading.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [node-js](<https://devfeed.tech/tags/node-js.md>), [one-honojs-file-for-entire-web-scraping-api](<https://devfeed.tech/tags/one-honojs-file-for-entire-web-scraping-api.md>), [product](<https://devfeed.tech/tags/product.md>), [programming](<https://devfeed.tech/tags/programming.md>), [react](<https://devfeed.tech/tags/react.md>), [scraping](<https://devfeed.tech/tags/scraping.md>), [typescript](<https://devfeed.tech/tags/typescript.md>), [web-development](<https://devfeed.tech/tags/web-development.md>), [web-scraping](<https://devfeed.tech/tags/web-scraping.md>), [web-scraping-github-repository-scraping-backend-honojs](<https://devfeed.tech/tags/web-scraping-github-repository-scraping-backend-honojs.md>)

### AI overview

A tutorial about building web-scraping endpoints in a single Honojs file.

### Source excerpt

Building scrape endpoints using one honojs file

## Hacking Displate (For Fun!)

DevFeed: [Hacking Displate (For Fun!)](<https://devfeed.tech/articles/hacking-displate-for-fun-38395.md>)

Original publisher: [Read original article](<https://blog.danlew.net/2025/11/18/hacking-displate-shenanigans/>)

Author: Dan Lew

Published: 2025-11-18T15:00:46Z

Content type: opinion

Language: en

Sources: [Dan Lew Blog](<https://devfeed.tech/sources/dan-lew-blog.md>)

Topics: [Hacking](<https://devfeed.tech/topics/hacking.md>), [Website](<https://devfeed.tech/topics/website.md>), [API](<https://devfeed.tech/topics/api.md>), [Browser Extension](<https://devfeed.tech/topics/browser-extension.md>), [browser](<https://devfeed.tech/topics/browser.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [browser](<https://devfeed.tech/tags/browser.md>), [extension](<https://devfeed.tech/tags/extension.md>), [hacking](<https://devfeed.tech/tags/hacking.md>), [order](<https://devfeed.tech/tags/order.md>), [scraping](<https://devfeed.tech/tags/scraping.md>), [tracking](<https://devfeed.tech/tags/tracking.md>), [website](<https://devfeed.tech/tags/website.md>)

### AI overview

A developer documents experiments with Displate's website, including tracking Limited Edition resale prices and using its API to discover early-access links. The article also describes ordering a Limited Edition before its public release and competing with another person using a browser extension to find links and monitor inventory.

### Source excerpt

During the pandemic my weird vice/obsession became Displates (metal wall art hung up by magnets): Some of my Displates (along with other tchotchkes in my office) For me, part of the hobby was poking holes in their website. I've long stopped trying to give their devs heartburn,

## Mathematical Genealogy

DevFeed: [Mathematical Genealogy](<https://devfeed.tech/articles/mathematical-genealogy-40413.md>)

Original publisher: [Read original article](<https://www.jeremykun.com/2017/06/22/mathematical-genealogy/>)

Published: 2017-06-22T09:00:10Z

Content type: opinion

Language: en

Sources: [Jeremy Kun](<https://devfeed.tech/sources/jeremy-kun.md>)

Topics: [Graphs](<https://devfeed.tech/topics/graphs.md>), [Database](<https://devfeed.tech/topics/database.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [JavaScript](<https://devfeed.tech/topics/javascript.md>), [GitHub](<https://devfeed.tech/topics/github.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [database](<https://devfeed.tech/tags/database.md>), [genealogy](<https://devfeed.tech/tags/genealogy.md>), [github](<https://devfeed.tech/tags/github.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [scraping](<https://devfeed.tech/tags/scraping.md>)

### AI overview

A side project explores the Mathematical Genealogy database as a directed graph of mathematicians and their thesis-advisor relationships. The article describes scraping a database snapshot, publishing the raw data and scripts, and building a browser-based visualizer with search and graph algorithms.

### Source excerpt

As a fun side project to distract me from my abysmal progress on my book, I decided to play around with the math genealogy graph! For those who don't know, since 1996, mathematicians, starting with the labor of Harry Coonce et al, have been managing a database of all mathematicians. More specifically, they've been keeping track of who everyone's thesis advisors and subsequent students were. The result is a directed graph (with a current estimated 200k nodes) that details the scientific lineage of mathematicians.