Extract leads from developer forums by detecting intent

Community Signals

Extract high-intent leads from developer community forums by detecting buying signals in public discussions. Currently supports Hacker News and Reddit.

When to Use

• User wants to find leads from developer communities or forums
• User wants to identify people publicly expressing pain with competitors
• User wants to find people asking "what tool should I use for X"
• User mentions Hacker News, Reddit, Stack Overflow, or developer forums as lead sources
• User describes prospects who discuss tools, complain about solutions, or ask for recommendations in public forums
• User wants to find developers who built DIY/hacky solutions for problems the user's product solves

Prerequisites

• Python 3.9+ with requests and optionally python-dotenv
• Apify API token in .env (for Reddit scraping)
• No auth needed for Hacker News (free Algolia API)
• Working directory: the project root containing this skill

Phase 1: Collect Context

Step 1: Gather Product & ICP Information

Ask the user for the following. Do NOT proceed without this — the entire query generation depends on it.

"To find the right leads from developer communities, I need to understand:

1. What does your product do? (one-liner)
2. Who are your competitors? (list the main ones)
3. What specific problems does your product solve? (the pain points)
4. Who is your ideal buyer? (role, company type, tech stack)
5. Any specific technologies or keywords associated with your space?"

If the user has already provided this context (e.g., from running the github-repo-signals skill), use that — don't ask again.

Phase 2: Generate Search Queries

Step 2: Generate Queries Across 9 Categories

Based on the user's product info, generate 3-5 search queries per category. These are the fixed categories — do not skip any:

Category 1: Alternative Seeking (intent score: 9) People actively looking to switch tools.

• Pattern: "[competitor] alternative", "alternative to [competitor]", "looking for [product type]"
• Example: "twilio alternative", "alternative to agora", "looking for video SDK"

Category 2: Competitor Pain (intent score: 8) People frustrated with a specific competitor.

• Pattern: "[competitor] issues", "frustrated with [competitor]", "[competitor] doesn't support"
• Example: "twilio video quality issues", "frustrated with agora pricing", "vonage api unreliable"

Category 3: Problem Space Questions (intent score: 6) People trying to solve the exact problem the product addresses.

• Pattern: "how to [thing product does]", "best way to [problem]", "recommendations for [category]"
• Example: "how to add video calling to app", "best webrtc framework", "real-time communication SDK"

Category 4: Tool Comparison (intent score: 8) People actively comparing options — in buying mode.

• Pattern: "[competitor A] vs [competitor B]", "comparing [tools]", "which [product type] should I use"
• Example: "twilio vs agora", "comparing video APIs", "which webrtc platform"

Category 5: DIY / Built Own Solution (intent score: 9) People who built a custom solution — validated the need, would pay for a proper product.

• Pattern: "I built my own [thing]", "Show HN: [thing product replaces]", "custom [solution type]"
• Example: "I built my own video conferencing", "Show HN: open source video call", "custom webrtc server"

Category 6: Scaling Challenges (intent score: 7) People hitting limits that the product solves.

• Pattern: "[problem] at scale", "scaling [thing]", "[thing] breaks with many users"
• Example: "webrtc scaling issues", "video calls lagging with 50+ participants", "scaling real-time communication"

Category 7: Migration Intent (intent score: 9) People who have already decided to leave — looking for where to go.

• Pattern: "migrating from [competitor]", "moving away from [competitor]", "switching from [competitor]"
• Example: "migrating from twilio video", "moving away from agora", "switching video API providers"

Category 8: Budget / Pricing Pain (intent score: 7) Cost is the trigger — open to cheaper or better-value alternatives.

• Pattern: "[competitor] too expensive", "[competitor] pricing", "cheaper alternative to [competitor]"
• Example: "twilio too expensive", "agora pricing 2026", "cheaper video API"

Category 9: Feature Gap Complaints (intent score: 7) Needs something their current tool doesn't do — and the user's product does.

• Pattern: "does [competitor] support [feature]", "[competitor] missing [feature]", "wish [competitor] had"
• Example: "does twilio support recording", "agora missing breakout rooms", "wish vonage had better docs"

Step 3: Discover Relevant Subreddits

Do a web search to find subreddits where the user's ICP is active. Search for:

• "[product category] subreddit"
• "[technology] subreddit"
• "[competitor name] subreddit"

Common developer subreddits to consider (pick the relevant ones):

• r/programming, r/webdev, r/devops, r/selfhosted
• r/kubernetes, r/aws, r/googlecloud, r/azure
• r/node, r/python, r/golang, r/rust
• r/startups, r/SaaS, r/entrepreneur
• r/sysadmin, r/networking
• Technology-specific: r/VOIP, r/machinelearning, r/dataengineering, etc.

Select 5-10 subreddits most relevant to the user's space.

Step 4: Present Queries for Review

Present ALL generated queries to the user in a structured table:

Category                  | Queries
--------------------------|------------------------------------------
Alternative Seeking       | "twilio alternative", "agora alternative", ...
Competitor Pain           | "twilio issues", "frustrated with agora", ...
...                       | ...

Subreddits to scan: r/webdev, r/VOIP, r/programming, ...

Ask:

"Here are the search queries I've generated. Would you like to:

1. Run with these as-is
2. Add or remove specific queries
3. Add or remove subreddits

Estimated cost: HN is free. Reddit via Apify will cost approximately $[estimate based on query count x ~$0.05 per query]."

Wait for user approval before proceeding.

Step 5: Save Queries File

Once approved, save the queries as a JSON file:

cat > ${CLAUDE_SKILL_DIR}/../.tmp/community_queries.json << 'QUERIESEOF'
{
    "product": "Product Name",
    "queries": [
        {"category": "alternative_seeking", "query": "twilio alternative"},
        {"category": "alternative_seeking", "query": "agora alternative"},
        {"category": "competitor_pain", "query": "twilio video quality issues"}
    ],
    "subreddits": ["r/webdev", "r/VOIP", "r/programming"]
}
QUERIESEOF

Phase 3: Execute Scan

Step 6: Verify Environment

python3 -c "import requests; print('OK')"

Step 7: Run the Tool

python3 ${CLAUDE_SKILL_DIR}/scripts/community_signals.py \
    --queries ${CLAUDE_SKILL_DIR}/../.tmp/community_queries.json \
    --days 30 \
    --max-reddit-posts 50 \
    --max-reddit-comments 20 \
    --output ${CLAUDE_SKILL_DIR}/../.tmp/community_signals.csv

The tool will:

1. Search Hacker News (stories + comments) for all queries — free
2. Search Reddit via Apify for all queries + scan subreddits — pay per result
3. Filter to last 30 days
4. Deduplicate users across platforms
5. Score by intent strength, signal count, category diversity, and cross-platform presence
6. Fetch HN user profiles (karma, bio) — free
7. Export two CSV files: _users.csv and _signals.csv

Optional flags:

• --skip-reddit — only search HN (free, for testing)
• --skip-hn — only search Reddit
• --days 7 — narrower time window for very fresh signals

Phase 4: Analyze & Recommend

Step 9: Analyze the Results

Read the output CSV files and present a structured briefing:

9a. Overall Stats

• Total signals found (HN + Reddit)
• Unique users
• Split by platform (HN vs Reddit)
• Cross-platform matches (same username on both)

9b. Signal Category Breakdown

• How many signals per category
• Which categories produced the most results
• Which categories had the highest-engagement posts (upvotes, comments)

9c. Top Subreddits Discovered

• Which subreddits appeared most frequently
• This tells the user where their prospects hang out — valuable for community marketing, not just outreach

9d. Highest-Intent Users

• List top 15-20 users by composite score
• For each: username, platform, categories they appeared in, sample post/comment, engagement
• Flag cross-platform users prominently

9e. Common Themes

• What are people specifically asking for or complaining about?
• Any patterns in the pain points that the user's product addresses?
• Any surprising findings (e.g., a competitor getting mentioned negatively much more than others)?

Step 10: Recommend Next Steps

Based on findings + user's product context:

1. If strong signals found (>50 high-intent users):

   • Recommend enriching top users via SixtyFour
   • For HN users: use their HN bio/karma + username for enrichment context
   • For Reddit users: username is the only identifier — enrichment hit rates may be lower
   • Suggest starting with HN users (more likely to have real names in bio)

2. If cross-platform matches found:

   • These are highest priority — someone active on both HN and Reddit in your space is deeply engaged
   • Recommend enriching these first

3. If specific subreddits emerged as hotspots:

   • Recommend ongoing monitoring of those subreddits
   • Suggest the user consider community engagement (commenting, answering questions) in those subreddits

4. If "alternative seeking" or "migration intent" signals dominate:

   • These are the most time-sensitive leads — they're actively evaluating RIGHT NOW
   • Recommend immediate outreach

5. If "DIY / built own" signals found:

   • These are the highest-quality leads — they've validated the need
   • Recommend personalized outreach referencing their project

6. Always include:

   • Cost estimate for enrichment
   • Suggested outreach angle per signal category
   • Reminder that community forum users respond better to helpful engagement than cold outreach

Step 11: Ask for Go-Ahead

"Would you like me to:

1. Enrich the top [N] users via SixtyFour (estimated cost: $X)
2. Run a deeper scan on the hotspot subreddits
3. Export this data for manual review first
4. Combine these results with GitHub signals data (if available)"

Wait for user confirmation.

Output Schema

community_signals_users.csv — One row per unique user across all platforms

Column	Description
username	Forum username
platform	hackernews or reddit
composite_score	Overall lead score (intent + diversity + cross-platform)
intent_score	Sum of category-weighted intent scores
signal_count	Number of matching posts/comments
categories	Which signal categories they appeared in
platforms_active	Which platforms they were found on
subreddits	Reddit subreddits they posted in
hn_karma	HN karma score (HN users only)
hn_bio	HN profile bio (HN users only)
total_engagement	Sum of upvotes + comments across their signals
first_seen	Earliest matching post/comment
latest_seen	Most recent matching post/comment
sample_url	Link to one of their matching posts

community_signals_signals.csv — One row per matching post/comment

Column	Description
platform	hackernews or reddit
author	Username
category	Signal category code
category_label	Human-readable category name
content_type	story, comment, or post
title	Post/story title
text	Post/comment body (truncated)
subreddit	Reddit subreddit (if applicable)
score	Upvotes
num_comments	Comment count
created_at	Date posted
query_matched	Which search query found this
url	Permalink to the post/comment

Scoring System

Intent scores by category:

Category	Score per Signal
Alternative Seeking	9
DIY / Built Own	9
Migration Intent	9
Competitor Pain	8
Tool Comparison	8
Scaling Challenge	7
Budget / Pricing	7
Feature Gap	7
Problem Space	6

Composite score bonuses:

• +2 per unique category the user appeared in (diversity)
• +10 if user found on multiple platforms (cross-platform)
• +2 per signal (capped at +10)

Cost Estimates

Platform	Cost	Notes
Hacker News	Free	Algolia API, 10k req/hr
Reddit (Apify)	~$0.004/result + $0.04/run	Pay per result
Typical run (45 queries)	~$5-10 total	HN free + Reddit ~$5-10

Limitations

• Reddit comments: Can't search comments directly — finds posts first, then fetches comments on those posts. Some comment-only discussions may be missed.
• Reddit date filter: No native date range parameter in Apify actor. Filtering happens in post-processing using created_at timestamps.
• User identity: Forum usernames are pseudonymous. Enrichment hit rates will be lower than GitHub (where people often use real names). HN users are more identifiable (many put real names in bio).
• Rate limits: HN Algolia: 10k req/hr. Apify: depends on plan.