Skip to content

Skill

Web Scraping

jamditis

Pulls data off web pages that don't hand it over cleanly - cascading through fallback fetch strategies and coping with 403s, CAPTCHAs, and rate limits, while validating every URL and treating whatever comes back as untrusted. It is a disciplined way to ACQUIRE raw data from the live web, not a tool for analyzing, charting, or synthesizing what you collect.

Situations it fits

You need a current list of all product pages and prices from a dozen competitor storefronts that don’t offer an API, and some storefronts block automated requests.
Web Scraping. web-scraping is the skill for obtaining live content from websites that don’t provide structured feeds, especially when you must work around blocking measures and multiple fetch strategies. The other skills handle analysis, visualization, or broad research rather than robustly acquiring live site content.
A reporter needs every city-council meeting date and its agenda link from a municipal site that lists them across dozens of paginated pages, collected into one clean table.
Web Scraping. Pulling structured records - dates and links - out of many paginated web pages into one table is the web-scraping skill's job. The spreadsheet skill would hold the table once you have it but cannot fetch it; EDA profiles a dataset you already possess; and chart-designer visualizes numbers rather than gathering them.
A marketing leader asks you to build a nightly process that collects product availability from multiple vendor websites that sometimes return a blank page or temporarily block your bot IPs.
Web Scraping. web-scraping is appropriate because the task requires reliably acquiring live information from several sites while handling intermittent blocks and malformed responses. EDA and charting assume you already have cleaned data, and deep research is about synthesizing multi-site findings rather than building a robust, fault-tolerant fetch pipeline.

6 scenarios in the bank answer to Web Scraping. The rest are in the game.

Skills it gets confused with

These share a family with Web Scraping, which is another way of saying they are the ones you might reach for by mistake.

Knowing what Web Scraping does is the easy half. Telling it apart from the others under time pressure is the game.

Today's session

Web Scraping is part of jamditis/claude-skills-journalism. Licence: MIT. The description above was written for this game, not taken from the skill.