A crawl is only a snapshot
Most crawlers give you one snapshot. When you come back, you crawl again and compare two spreadsheets by hand, guessing at what moved in between.
Code solved this long ago with diffs. Websites deserve the same. In Silkra, every refresh is a new version and the change log is the diff: what changed, and whether it affects how pages are fetched and indexed.