Scrape blog posts via RSS feeds with Apify fallback

Blog feed monitor

by gooseworks-ai · MIT license · ★ 1,220 on GitHub · updated Sep 21

Scrape blog posts via RSS feeds (free, no API key) with Apify fallback for JS-heavy sites. Use when you need to monitor competitor blogs, track industry content, or aggregate blog posts by keyword.

---
name: blog-feed-monitor
description: >
  Scrape blog posts via RSS feeds (free, no API key) with Apify fallback for
  JS-heavy sites. Use when you need to monitor competitor blogs, track industry
  content, or aggregate blog posts by keyword.
---

# Blog Feed Monitor

Scrape blog posts via RSS/Atom feeds (free) with optional Apify fallback for JS-heavy sites.

## Quick Start

No API key needed for RSS mode.

```bash
# Scrape a blog's RSS feed

Overview

Scrape blog posts via RSS/Atom feeds (free) with optional Apify fallback for JS-heavy sites.

Manual install (npx, ZIP)

Claude Codenpx skills add gooseworks-ai/goose-skills --skill blog-feed-monitor -g -a claude-code

Claude Code (manual)

mkdir -p ~/.claude/skills && curl -sL https://codeload.github.com/gooseworks-ai/goose-skills/tar.gz/615c685f34f8 \
  | tar -xz -C ~/.claude/skills --strip-components=4 goose-skills-615c685f34f8/skills/monitoring/capabilities/blog-feed-monitor

Windows (PowerShell)

New-Item -ItemType Directory -Force "$HOME\.claude\skills" | Out-Null; Invoke-WebRequest "https://m.agentalley.io/zip/29d1b89ef314de24f1d2/blog-feed-monitor.zip" -OutFile "$env:TEMP\blog-feed-monitor.zip"; Expand-Archive "$env:TEMP\blog-feed-monitor.zip" -DestinationPath "$HOME\.claude\skills" -Force

Claude.ai / desktopDownload the ZIP and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md2.3 KB
skill.meta.json256 B
scripts/1 file · 15.0 KB

Read the files here

gooseworks-ai/615c6853 files

SKILL.md

Show the full text66 lines

blog-feed-monitor/SKILL.md66 lines · 2.3 KB

PreviewCode

Outline

RawView on GitHub

Blog Feed Monitor

Scrape blog posts via RSS/Atom feeds (free) with optional Apify fallback for JS-heavy sites.

Quick Start

No API key needed for RSS mode.

# Scrape a blog's RSS feed
python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
  --urls "https://example.com/blog" --days 30

# Multiple blogs with keyword filter
python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
  --urls "https://blog1.com,https://blog2.com" --keywords "AI,marketing" --output summary

# Force Apify for JS-heavy sites
python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
  --urls "https://example.com" --mode apify

How It Works

Auto Mode (default)

For each URL, tries to discover an RSS/Atom feed:
- Checks HTML <link rel="alternate"> tags
- Probes common paths: /feed, /rss, /atom.xml, /feed.xml, /rss.xml, /blog/feed, /index.xml
Parses discovered feeds (supports RSS 2.0 and Atom)
If any URLs fail, falls back to Apify jupri/rss-xml-scraper (if token available)
Applies date and keyword filtering client-side

Note: The Apify fallback actor jupri/rss-xml-scraper may need updating -- it has not been verified recently. RSS mode works reliably without it.

RSS Mode

Only tries RSS feeds, no Apify fallback.

Apify Mode

Uses Apify actor directly, skipping RSS discovery.

CLI Reference

Flag	Default	Description
`--urls`	required	Blog URL(s), comma-separated
`--keywords`	none	Keywords to filter (comma-separated, OR logic)
`--days`	30	Only include posts from last N days
`--max-posts`	50	Max posts to return
`--mode`	auto	`auto` (RSS + fallback), `rss` (RSS only), `apify` (Apify only)
`--output`	json	Output format: `json` or `summary`
`--token`	env var	Apify token (only needed for Apify mode/fallback)
`--timeout`	300	Max seconds for Apify run

Cost

RSS mode: Free (no API, no tokens)
Apify mode: Uses jupri/rss-xml-scraper -- minimal Apify credits

1	---
2	name: blog-feed-monitor
3	description: >
4	Scrape blog posts via RSS feeds (free, no API key) with Apify fallback for
5	JS-heavy sites. Use when you need to monitor competitor blogs, track industry
6	content, or aggregate blog posts by keyword.
7	---
8
9	# Blog Feed Monitor
10
11	Scrape blog posts via RSS/Atom feeds (free) with optional Apify fallback for JS-heavy sites.
12
13	## Quick Start
14
15	No API key needed for RSS mode.
16
17	```bash
18	# Scrape a blog's RSS feed
19	python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
20	--urls "https://example.com/blog" --days 30
21
22	# Multiple blogs with keyword filter
23	python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
24	--urls "https://blog1.com,https://blog2.com" --keywords "AI,marketing" --output summary
25
26	# Force Apify for JS-heavy sites
27	python3 skills/blog-feed-monitor/scripts/scrape_blogs.py \
28	--urls "https://example.com" --mode apify
29	```
30
31	## How It Works
32
33	### Auto Mode (default)
34	1. For each URL, tries to discover an RSS/Atom feed:
35	- Checks HTML `<link rel="alternate">` tags
36	- Probes common paths: `/feed`, `/rss`, `/atom.xml`, `/feed.xml`, `/rss.xml`, `/blog/feed`, `/index.xml`
37	2. Parses discovered feeds (supports RSS 2.0 and Atom)
38	3. If any URLs fail, falls back to Apify `jupri/rss-xml-scraper` (if token available)
39	4. Applies date and keyword filtering client-side
40
41	> Note: The Apify fallback actor `jupri/rss-xml-scraper` may need updating -- it has not been verified recently. RSS mode works reliably without it.
42
43	### RSS Mode
44	Only tries RSS feeds, no Apify fallback.
45
46	### Apify Mode
47	Uses Apify actor directly, skipping RSS discovery.
48
49	## CLI Reference
50
51	\| Flag \| Default \| Description \|
52	\|------\|---------\|-------------\|
53	\| `--urls` \| required \| Blog URL(s), comma-separated \|
54	\| `--keywords` \| none \| Keywords to filter (comma-separated, OR logic) \|
55	\| `--days` \| 30 \| Only include posts from last N days \|
56	\| `--max-posts` \| 50 \| Max posts to return \|
57	\| `--mode` \| auto \| `auto` (RSS + fallback), `rss` (RSS only), `apify` (Apify only) \|
58	\| `--output` \| json \| Output format: `json` or `summary` \|
59	\| `--token` \| env var \| Apify token (only needed for Apify mode/fallback) \|
60	\| `--timeout` \| 300 \| Max seconds for Apify run \|
61
62	## Cost
63
64	- RSS mode: Free (no API, no tokens)
65	- Apify mode: Uses `jupri/rss-xml-scraper` -- minimal Apify credits
66