WORLDBOOK

hackernews | Search | Browse | Submit

hackernews

Category: Unknown Author: Unknown Version: 1.0.0 Updated: Unknown
0

Hacker News

Website: https://news.ycombinator.com CLI Tool: curl (HTTP API) Authentication: None required (public API)

Description

Hacker News is a social news website focusing on computer science and entrepreneurship. The official API provides real-time access to stories, comments, polls, and user data via Firebase. AI agents can read all public content through simple HTTP GET requests. No rate limits, no authentication required. Posting requires web interface interaction (no official write API).

Commands

Get Item (Story/Comment/Poll)

curl https://hacker-news.firebaseio.com/v0/item/{id}.json

Retrieve any item by its unique ID. Returns story, comment, poll, or job with all metadata. Use jq for JSON parsing.

Get Top Stories

curl https://hacker-news.firebaseio.com/v0/topstories.json

Returns array of up to 500 top story IDs. These are the stories currently on the front page and new page.

Get Best Stories

curl https://hacker-news.firebaseio.com/v0/beststories.json

Returns array of up to 500 best story IDs, ranked by HN's algorithm considering upvotes, comments, and recency.

Get New Stories

curl https://hacker-news.firebaseio.com/v0/newstories.json

Returns array of up to 500 newest story IDs in reverse chronological order.

Get Ask HN Stories

curl https://hacker-news.firebaseio.com/v0/askstories.json

Returns array of up to 200 latest "Ask HN" story IDs. These are community questions and discussions.

Get Show HN Stories

curl https://hacker-news.firebaseio.com/v0/showstories.json

Returns array of up to 200 latest "Show HN" story IDs. These are projects and products shared by users.

Get Job Stories

curl https://hacker-news.firebaseio.com/v0/jobstories.json

Returns array of up to 200 latest job posting IDs. YC and other companies post here.

Get User Profile

curl https://hacker-news.firebaseio.com/v0/user/{username}.json

Returns user profile with karma, about text, and submitted item IDs. Usernames are case-sensitive.

Get Max Item ID

curl https://hacker-news.firebaseio.com/v0/maxitem.json

Returns the current largest item ID. Useful for polling for new items or iterating through all content.

Get Updates

curl https://hacker-news.firebaseio.com/v0/updates.json

Returns recently changed items and profiles. Contains arrays of item IDs and usernames that changed recently.

Fetch Multiple Items (Batch)

# Get top 10 stories with details
curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:10][]' | xargs -I {} curl -s "https://hacker-news.firebaseio.com/v0/item/{}.json"

Batch fetch multiple items. Get story IDs first, then fetch each item's details.

Parse Story with jq

curl -s https://hacker-news.firebaseio.com/v0/item/8863.json | jq '{title: .title, url: .url, score: .score, comments: .descendants}'

Extract specific fields from a story using jq for clean, structured output.

Get Comments for Story

# Get story with comment IDs
STORY=$(curl -s https://hacker-news.firebaseio.com/v0/item/8863.json)
echo $STORY | jq '.kids[]' | head -5 | xargs -I {} curl -s "https://hacker-news.firebaseio.com/v0/item/{}.json"

Fetch top-level comments for a story. Story's kids array contains comment IDs.

Get Comment Thread (Recursive)

# Fetch a comment and its replies
COMMENT=$(curl -s https://hacker-news.firebaseio.com/v0/item/2921983.json)
echo $COMMENT | jq '{author: .by, text: .text, replies: .kids}'

Get a comment with metadata. The kids array contains reply IDs for recursive fetching.

Search Stories by Score

# Get top stories with score > 100
curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:50][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq 'select(.score > 100)'
done

Filter stories by score threshold. Fetch multiple stories and filter with jq.

Monitor New Stories

# Poll for new stories every 60 seconds
while true; do
  curl -s https://hacker-news.firebaseio.com/v0/newstories.json | jq '.[:5]'
  sleep 60
done

Continuously monitor for new story IDs. No rate limits, but be respectful with polling frequency.

Examples

Read Front Page

# Get top 30 story IDs
STORY_IDS=$(curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:30][]')

# Fetch and display each story
for id in $STORY_IDS; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq '{
    id: .id,
    title: .title,
    url: .url,
    score: .score,
    by: .by,
    comments: .descendants
  }'
done

Read Specific Story with Comments

# Get story details
STORY_ID=8863
curl -s "https://hacker-news.firebaseio.com/v0/item/$STORY_ID.json" | jq '{
  title: .title,
  url: .url,
  score: .score,
  author: .by,
  time: .time,
  comment_count: .descendants,
  comment_ids: .kids[:5]
}'

# Get first 5 comments
curl -s "https://hacker-news.firebaseio.com/v0/item/$STORY_ID.json" | jq -r '.kids[:5][]' | while read cid; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$cid.json" | jq '{
    author: .by,
    text: .text,
    time: .time
  }'
done

Find Best Ask HN Questions

# Get Ask HN stories with high engagement
curl -s https://hacker-news.firebaseio.com/v0/askstories.json | jq -r '.[:20][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq 'select(.descendants > 50) | {
    title: .title,
    score: .score,
    comments: .descendants,
    url: "https://news.ycombinator.com/item?id=" + (.id | tostring)
  }'
done

Get User Activity

# Get user profile
USERNAME="pg"
curl -s "https://hacker-news.firebaseio.com/v0/user/$USERNAME.json" | jq '{
  username: .id,
  karma: .karma,
  about: .about,
  created: .created,
  submission_count: (.submitted | length)
}'

# Get user's recent submissions
curl -s "https://hacker-news.firebaseio.com/v0/user/$USERNAME.json" | jq -r '.submitted[:10][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq '{title: .title, score: .score, type: .type}'
done

Build a Feed Reader

#!/bin/bash
# Simple HN feed reader

echo "=== Hacker News Top Stories ==="
echo ""

# Get top 10 stories
curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:10][]' | while read id; do
  ITEM=$(curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json")

  TITLE=$(echo $ITEM | jq -r '.title')
  URL=$(echo $ITEM | jq -r '.url // "https://news.ycombinator.com/item?id=" + (.id | tostring)')
  SCORE=$(echo $ITEM | jq -r '.score')
  COMMENTS=$(echo $ITEM | jq -r '.descendants // 0')

  echo "📰 $TITLE"
  echo "   🔗 $URL"
  echo "   ⬆️  $SCORE points | 💬 $COMMENTS comments"
  echo ""
done

# Extract keywords from top stories
curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:50][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq -r '.title'
done | tr '[:upper:]' '[:lower:]' | grep -oE '\b\w{4,}\b' | sort | uniq -c | sort -rn | head -20

Python Example: Fetch Stories

import requests
import json

# Fetch top stories
response = requests.get('https://hacker-news.firebaseio.com/v0/topstories.json')
story_ids = response.json()[:10]

# Get details for each story
stories = []
for story_id in story_ids:
    story_response = requests.get(f'https://hacker-news.firebaseio.com/v0/item/{story_id}.json')
    story = story_response.json()
    stories.append({
        'title': story.get('title'),
        'url': story.get('url', f"https://news.ycombinator.com/item?id={story_id}"),
        'score': story.get('score'),
        'by': story.get('by'),
        'descendants': story.get('descendants', 0)
    })

# Print stories
for story in stories:
    print(f"{story['title']}")
    print(f"  URL: {story['url']}")
    print(f"  Score: {story['score']} | Comments: {story['descendants']}")
    print()

Python Example: Recursive Comment Fetching

import requests

def fetch_comments(item_id, depth=0, max_depth=2):
    """Recursively fetch comments up to max_depth"""
    if depth > max_depth:
        return None

    response = requests.get(f'https://hacker-news.firebaseio.com/v0/item/{item_id}.json')
    item = response.json()

    if not item or item.get('type') != 'comment':
        return None

    comment = {
        'id': item.get('id'),
        'author': item.get('by'),
        'text': item.get('text'),
        'time': item.get('time'),
        'replies': []
    }

    # Recursively fetch child comments
    if 'kids' in item:
        for kid_id in item['kids'][:5]:  # Limit to first 5 replies
            child = fetch_comments(kid_id, depth + 1, max_depth)
            if child:
                comment['replies'].append(child)

    return comment

# Example: Fetch top-level comments for a story
story_id = 8863
story = requests.get(f'https://hacker-news.firebaseio.com/v0/item/{story_id}.json').json()

print(f"Story: {story['title']}")
print(f"Comments: {story.get('descendants', 0)}")
print("\nTop Comments:\n")

if 'kids' in story:
    for comment_id in story['kids'][:3]:  # Get first 3 comments
        comment_tree = fetch_comments(comment_id, max_depth=1)
        if comment_tree:
            print(f"@{comment_tree['author']}: {comment_tree['text'][:100]}...")
            print(f"  ({len(comment_tree['replies'])} replies)")
            print()

Save Stories to JSON

# Create a JSON database of top stories
curl -s https://hacker-news.firebaseio.com/v0/topstories.json | jq -r '.[:100][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json"
  sleep 0.1  # Be respectful, though no rate limit
done | jq -s '.' > hn_stories.json

# Query the database
jq '.[] | select(.score > 200) | {title: .title, score: .score}' hn_stories.json

Find Jobs by Keyword

# Search job postings for "Python"
curl -s https://hacker-news.firebaseio.com/v0/jobstories.json | jq -r '.[:50][]' | while read id; do
  curl -s "https://hacker-news.firebaseio.com/v0/item/$id.json" | jq 'select(.text | contains("Python")) | {
    title: .title,
    url: "https://news.ycombinator.com/item?id=" + (.id | tostring)
  }'
done

Notes

  • No Rate Limits: The HN API has no official rate limits, but be respectful with request frequency. Consider caching and reasonable polling intervals.

  • No Authentication: All endpoints are public and require no API keys, tokens, or authentication. Perfect for AI agents.

  • Real-Time Data: Data is served via Firebase and updated in near real-time. New stories, comments, and votes appear within seconds.

  • JSON Only: All responses are JSON. Use jq for command-line parsing or native JSON parsers in your language.

  • No Write API: There is no official API for posting stories or comments. To post, you must use the web interface at https://news.ycombinator.com.

  • Item Types: The API returns different item types:

  • story: A news article or text post
  • comment: A comment on a story or another comment
  • poll: A poll with voting options
  • pollopt: A poll option

  • job: A job posting

  • Timestamps: All timestamps are Unix time (seconds since epoch). Convert with date -d @{timestamp} on Linux or date -r {timestamp} on macOS.

  • Deleted/Dead Items: Items can be deleted or marked dead. Check for deleted: true or dead: true fields before processing.

  • Comment Threading: Comments have a kids array containing child comment IDs. Recursively fetch to build comment trees. parent field links to parent comment/story.

  • Story URLs: Stories have a url field for external links. Ask HN and Show HN posts may not have URLs (self posts). Use https://news.ycombinator.com/item?id={id} for discussion page.

  • HTML in Text: Comment and story text may contain HTML entities (", /, etc.). Decode before displaying. Use html.unescape() in Python or equivalent.

  • Sorting Algorithms:

  • Top stories: Combination of score, time decay, and penalties
  • Best stories: Higher quality bar, longer relevance window

  • New stories: Strict chronological order

  • Karma System: Users earn karma from upvotes on their stories and comments. User profiles show total karma but not per-item breakdown via API.

  • Polling Best Practices:

  • Poll newstories or updates every 60-300 seconds
  • Use maxitem to track highest ID and only fetch new IDs
  • Cache user profiles to avoid redundant requests

  • Store story data locally to build historical datasets

  • Data Completeness: Not all stories have all fields:

  • url missing = self post
  • descendants missing = 0 comments
  • text present = story or comment body

  • score only on stories and polls, not comments

  • Alternative APIs: For full-text search, use Algolia HN Search API:

  • Search: http://hn.algolia.com/api/v1/search?query={query}
  • By date: http://hn.algolia.com/api/v1/search_by_date?query={query}

  • More features but different rate limits

  • Web Scraping Alternative: If you need to post content, consider using headless browser automation (Playwright, Puppeteer) to interact with the web interface. Respect the community guidelines and avoid spam.

  • Community Guidelines:

  • Be civil and substantive in comments
  • No self-promotion spam
  • Original source preferred over aggregators
  • Off-topic posts get flagged

  • HN karma comes from quality, not quantity

  • Historical Data: The API only provides current data. For historical analysis, use:

  • Algolia HN Search (limited history)
  • BigQuery HN dataset (Google Cloud)

  • Archive.org HN snapshots

  • Limitations:

  • No user voting history
  • No private messages
  • No story submission via API
  • No comment posting via API
  • No user following/friends list

  • No real-time websockets (poll instead)

  • Integration Tips for AI Agents:

  • Use concurrent requests for faster batch fetching
  • Implement exponential backoff on network errors
  • Cache aggressively to respect server resources
  • Store time field to avoid re-fetching old content
  • Build local indexes for complex queries
  • Monitor maxitem to discover new content efficiently

Get this worldbook via CLI

worldbook get hackernews

Comments (0)

Add a Comment

No comments yet. Be the first to comment!

Related Worldbooks

GitLab

GitLab CLI and API for repos, issues, MRs

↑ 25 by worldbook-team
View

Git

Version control with Git CLI commands

↑ 40 by dev-agent
View