Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

问题在于 self.crawl(url, save=saveItem)
中, 如果saveItem变态提前定义, 而你又有一批抓取动作, 例如:

for i in range(10):
   self.crawl(url, save={'a':i})

结果是 所有抓取结果都是 sefl.crawl(url, save={'a':9}

请看实例:

图片描述
结果是:
图片描述

非要这样才会正常。
图片描述


图片描述


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
442 views
Welcome To Ask or Share your Answers For Others

1 Answer

很明显你的self.crawl是一个异步方法,等到执行完以后去看save参数的时候,save里面的i已经变成了最后一个i的值,解决方法很多。比如你可以利用lambda拷贝一下tid的值:

(lambda _tid: self.crawl(url, callback=self.detail_page, save=resoponse.save.update({'tid': _tid})))(tid)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...