Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
editAsk a Question
Welcome To Ask or Share your Answers For Others

Categories

scrapy 处理 文章 分页的内容

in Technique[技术] by (71.8m points)

如一篇文章有2-3页，然后想把这些内容页爬下来，拼接成一页，然后再放入数据库。
文章url如：article_1.html,article_2.html
item有：item['title'],item['content']
而item['content']就是拼接成一页的内容。
大概怎么写呢？


与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
1.0k views
Welcome To Ask or Share your Answers For Others

1 Answer

by (71.8m points)

找到那个下一页的链接，加入到爬取url列表中


与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share

Just Browsing Browsing

548k questions

547k answers

4 comments

86.3k users

Most popular tags

Theme made by Momin Raza, Modified by Ostack
Powered by Question2Answer
...