web scraping - Python Scrapy - parse URL content for most recent updated date

Question

I have a webcrawler/scraper that's written in Python using the scrapy framework. I've been trying to use the "last-modified" date to identify the most recent update for each page - but I also collect each HTML file for the pages that are scraped. Is there a more accurate method for collecting the date each page was most recently updated?

Answer 1

等待大神答复