本文整理汇总了Python中scrapy.settings.Settings.overrides['SPIDER_MODULES']方法的典型用法代码示例。如果您正苦于以下问题：Python Settings.overrides['SPIDER_MODULES']方法的具体用法？Python Settings.overrides['SPIDER_MODULES']怎么用？Python Settings.overrides['SPIDER_MODULES']使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类scrapy.settings.Settings的用法示例。
在下文中一共展示了Settings.overrides['SPIDER_MODULES']方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: parse
# 需要导入模块: from scrapy.settings import Settings [as 别名]
# 或者: from scrapy.settings.Settings import overrides['SPIDER_MODULES'] [as 别名]
def parse(self, response):
"""Accesses the page at start_urls and parses each event link,
makes a new request to the event link and uses parse_concert
to render into JSON.
"""
for href in response.css('a.event_link::attr(href)'):
full_url = response.urljoin(href.extract())
yield Request(full_url, callback=self.parse_concert)
def parse_concert(self, response):
"""Parses an event page,
extracting data from the HTML response
using CSS
"""
yield {
'title': response.css('.event-information h1::text').extract()[0],
'location': response.css('.venue-details h2::text').extract()[0],
'time': response.css('.venue-details h4::text').extract()[0],
}
timestamp = '{:%Y%m%dT%H%M%S}'.format(datetime.now())
settings = Settings()
settings.overrides['SPIDER_MODULES'] = 'WeGoTicketsSpider'
settings.overrides['FEED_FORMAT'] = 'json'
settings.overrides['FEED_URI'] = timestamp + 'events.json'
process = CrawlerProcess(settings)
process.crawl(WeGoTicketsSpider)
process.start()