No Official API for PTT Gossiping or Any Board: Two Scraping Modes and the over18 Cookie Gate
In one line: PTT Gossiping — and every other PTT board — has no official API: just a board-list mode and a single-post mode, so before you build your own you need to know the over18 cookie gate, the incremental-fetch stop rule, and whether a maintained Actor is cheaper than owning those problems yourself.
What are you actually working with, if PTT has no official API?
PTT (批踢踢實業坊) is fundamentally a Telnet BBS system. Its core software, pttbbs, is open source on GitHub: the text-mode (Telnet) program lives in the mbbsd directory, alongside background services such as boardd, which bridges board content to the ptt.cc web version. In other words, PTT has never published a documented web data API — not for Gossiping, PTT’s largest and best-known board, and not for any other board either. Anyone who wants its data has to either parse the web HTML themselves or reverse-engineer the BBS protocol.
That gap is exactly why community libraries exist: PyPtt is a pure-Python SDK that reimplements PTT’s own functionality and currently has 734 GitHub stars, and jwlin/ptt-web-crawler is a scraper aimed specifically at the web version, with 451 stars. Projects like these existing at all is proof of the underlying fact: no official API means the community has had to build its own.
If you’re about to build a PTT scraper yourself, the first thing to settle is what shape of data you actually need, because the two read modes hit completely different problems.
Two modes, two completely different shapes of data
Board list (board) | Single post (post) | |
|---|---|---|
| Input | boards — an array of board names, e.g. Gossiping, Stock, Tech_Job (PTT is case-sensitive) | postUrls — an array of full post URLs |
| Output | Post titles, author, push/boo tally, post time | The full post text plus every comment, numbered by floor |
| Filter | Optional keywords: keep only titles containing a given word | None — a single post pulls the whole thread as-is |
| Billing | search-listing, $0.002 per listing | Main post product-detail $0.008/post, plus comments review-item $0.003/comment |
| Good for | Scanning a board for anything new | Reading one popular thread’s full conversation |
The easiest mistake is assuming board mode’s keywords field is a search. It isn’t — it’s a local, after-the-fetch string match against titles already on the board-list page, and it never calls a search endpoint or looks past that one board. PTT’s web version has no official site-wide search endpoint to call, which is the actual reason a PTT scraper only needs two modes, not three — there’s no missing feature, there’s no endpoint. Compare that with LIHKG, whose app does talk to a real site-wide search endpoint: see LIHKG Scraper Has No Official Docs for how that changes the build.
The over18 cookie gate
Age-restricted boards — Gossiping is the obvious example — redirect a first-time request to an age-consent page instead of returning the content. A public example script shows the core logic: detect the redirect to the /ask/over18 consent page, set an over18 cookie scoped to the site (the example sets its expiry out to the year 2038 to dodge the Year 2038 problem on 32-bit timestamp handling), then retry the original post URL. If you’re building this yourself, that detect-and-retry flow has to be handled once, correctly, before any Gossiping content comes back at all.
Building it yourself vs using a maintained Actor
A DIY build has to handle all of the following, not just the cookie:
- The over18 cookie gate, above — a one-time flow to get right, but it has to run before every first request to a gated board.
- The incremental-fetch stop rule. Scheduled monitoring that only wants new posts has to key off each post ID’s timestamp, accurate to the second, and only stop paginating once a full page of results is older than the cutoff. Get that logic wrong and you either re-pull posts you already have or silently miss new ones.
- HTML cleanup. Post text and comments come back as raw HTML; signature blocks and the “發信站” (posting station) line need to be stripped manually before the text is plain and usable.
- Standing risk of a layout or verification change. PTT’s public read endpoints are currently friendly to direct, datacenter-IP requests, but that can change at any time, and a DIY scraper that breaks has to be re-diagnosed and re-written by you.
Three things decide which way to go:
- Scale — a one-off pull from a single board for a small research sample is usually a few hours of DIY work.
- Frequency — scheduled monitoring across multiple boards or keywords means owning every pitfall above, indefinitely, not just once.
- Risk tolerance — if PTT changes its markup, a DIY scraper breaks until you notice and fix it; a maintained Actor is watched by daily automated checks, so breakage is caught immediately rather than discovered by a gap in your data.
What does it actually cost?
Using ptt-scraper’s public pricing page:
| Event | Price | Applies to |
|---|---|---|
search-listing (one board-list item) | $0.002 | Board mode |
product-detail (one post’s main text) | $0.008 | Post mode |
review-item (one comment) | $0.003 | Post mode |
A popular, long thread can carry hundreds of comments, so the post-mode bill scales with how much discussion a thread drew — if you only need a rough read on what’s happening, scan with board mode first and selectively run post mode only on the threads actually worth reading in full, rather than pulling full text and every comment for everything. Apify also adds a flat, one-time per-run platform fee of $0.005 automatically; that’s billed by Apify itself, not by the Actor’s own code.
Three ways to start
From least effort to most freedom
FAQ
Does PTT Gossiping have an official API? No. PTT is fundamentally a Telnet BBS system; its core software, pttbbs, is open source on GitHub (a text-mode program plus background services), and the ptt.cc website is just a browser front end built on top of that BBS. PTT has never published a documented web data API for Gossiping or any other board. Developers have had to parse the HTML or reverse-engineer the BBS protocol themselves, which is why third-party libraries like PyPtt (734 GitHub stars) and ptt-web-crawler (451 stars) exist at all.
What’s the difference between board-mode keyword filtering and a real search? They’re not the same, and it’s the easiest thing to get wrong when building your own. Board mode’s keyword filter only matches against titles already pulled from the board list page, as a local string match done after the fetch, not a call to any search endpoint, and it never reaches outside that one board. PTT’s web version has no official site-wide search endpoint at all, which is why a PTT scraper only needs a board-list mode and a single-post mode. There is no third search mode to build, because PTT has no search endpoint to call.
What pitfalls will I hit building a PTT scraper myself? A few: age-gated boards such as Gossiping redirect first-time requests to an age-consent page, so you need to set an over18 cookie and retry before you see any content; scheduled, incremental scraping needs a stop rule based on each post ID’s timestamp, accurate to the second, since getting the pagination cutoff wrong means you either refetch the same posts or miss new ones; and full post text and comments are plain HTML, so signature blocks and the posting-station line need manual stripping before the text is clean.
How is each mode priced? Board-list mode is $0.002 per listing. Single-post mode is $0.008 per post for the main text, plus $0.003 per comment, and a popular long thread can have hundreds of comments, so the single-post bill rises with how much discussion it drew. Apify also charges a one-time per-run platform fee of $0.005 automatically, billed by Apify itself, not by the Actor’s code.
When should I build it myself versus use a maintained Actor? A one-off pull from a single board for a small research sample is usually a few hours of work to DIY. But scheduled, ongoing monitoring across multiple boards or keywords means owning the over18 cookie flow, the incremental stop logic, and HTML cleanup indefinitely, plus the risk of PTT changing its markup later. A maintained Actor has already solved those engineering problems and is watched by daily canary checks, so you find out immediately if something breaks.
Further reading
- LIHKG Scraper Has No Official Docs: the same build-vs-buy question applied to a forum that, unlike PTT, does have a real site-wide search endpoint to reverse-engineer.
- PTT Monitoring: Hire Someone or Use a Scraper?: same
ptt-scraper, but a different axis — the labor-cost case for automating board scanning, not the engineering case for building it yourself.
Chad runs 30+ published Apify actors, including ptt-scraper used here. Claims about PTT’s lack of an official API, the pttbbs source layout, and community library star counts are checked against their public GitHub pages; the Actor’s modes, fields and prices are checked against its own input schema and public listing.