本文整理汇总了Python中scrapy.log.DEBUG属性的典型用法代码示例。如果您正苦于以下问题：Python log.DEBUG属性的具体用法？Python log.DEBUG怎么用？Python log.DEBUG使用的例子？那么, 这里精选的属性代码示例或许可以为您提供帮助。您也可以进一步了解该属性所在类scrapy.log的用法示例。
在下文中一共展示了log.DEBUG属性的5个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: process_item
# 需要导入模块: from scrapy import log [as 别名]
# 或者: from scrapy.log import DEBUG [as 别名]
def process_item(self, item, spider):
"""
Scrape edilen her girdiyi veritabanina ekle. Bu method sayfa process edildikten, icerisindeki
bilgiler cekildikten ve Item objesi olusturulduktan sonra her seferinde cagriliyor.
:param item: Parse edilmis nesne
:type item: Scrapy item
:param spider: Su anda calisan, spiders/ dizini altinda belirtilen spiderlardan herhangi biri
:type spider: Scrapy spider
:return: Gonderilen Item
:rtype: Scrapy item
"""
log.msg('[%s] PROCESSING ITEM [item no: %s, baslik: %s]' %
(spider.name, item['girdi_id'], item['baslik']),
level=log.DEBUG)
girdi = Girdi(**item)
try:
session.add(girdi)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
return item 示例2: request_seen
# 需要导入模块: from scrapy import log [as 别名]
# 或者: from scrapy.log import DEBUG [as 别名]
def request_seen(self, request):
is_seen = is_request_seen(request)
if not is_seen:
log.msg('New URL: %s. Adding it to seen database' % request.url, log.DEBUG)
seen = Seen(fingerprint=request_fingerprint(request),
url=request.url,
last_crawl_time=datetime.now())
try:
session.add(seen)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
else:
log.msg('[seen] "%s" is seen. Skipping.' % request.url, log.INFO)
return is_seen 示例3: debug
# 需要导入模块: from scrapy import log [as 别名]
# 或者: from scrapy.log import DEBUG [as 别名]
def debug(msg):
log.msg(str(msg), level=log.DEBUG) 示例4: process_spider_output
# 需要导入模块: from scrapy import log [as 别名]
# 或者: from scrapy.log import DEBUG [as 别名]
def process_spider_output(self, response, result, spider):
for x in result:
if isinstance(x, Request) and hasattr(spider, 'disallow_urls'):
if self.should_follow(x, spider):
yield x
else:
log.msg("Filtered URL %s: " % (x.url),
level=log.DEBUG, spider=spider)
else:
yield x 示例5: dropped
# 需要导入模块: from scrapy import log [as 别名]
# 或者: from scrapy.log import DEBUG [as 别名]
def dropped(self, item, exception, response, spider):
return {
'level': log.DEBUG,
'msg': logformatter.DROPPEDMSG,
'args': {
'exception': exception,
'item': item,
}
}