Web scraping via JavaScript runtime heap snapshots (adriancooney.ie)
recommended by 1 user | 2022 | 354 points by adriancooney | web development | original thread |
One user recommended this thread. What they said:
jaymzcampbell Jun 15, 2023 | on: How to crawl a quarter billion webpages in 40 hours (2012) | context

On the topic of JSON and whatnot that reminded me of this awesome post[1] about looking within heap snapshots for entire data structures where they may have a lot of nicely structured data within it not readily available from more direct URL calls.

[1] https://www.adriancooney.ie/blog/web-scraping-via-javascript...

(Previous comments about it from 2022 here: https://news.ycombinator.com/item?id=31205139)

About | Topics | Past | Comments | Recent | Hindsight | Data | Keyboard shortcuts
Hacker News for Hacker News
Not affiliated with Y Combinator or Hacker News. Comments are by their HN authors. Data: open-index/hacker-news (ODC-BY) and the HN API, as of November 3, 2026.