本文整理匯總了Python中scrapy.log.DEBUG屬性的典型用法代碼示例。如果您正苦於以下問題：Python log.DEBUG屬性的具體用法？Python log.DEBUG怎麽用？Python log.DEBUG使用的例子？那麽, 這裏精選的屬性代碼示例或許可以為您提供幫助。您也可以進一步了解該屬性所在類scrapy.log的用法示例。
在下文中一共展示了log.DEBUG屬性的5個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Python代碼示例。
示例1: process_item
# 需要導入模塊: from scrapy import log [as 別名]
# 或者: from scrapy.log import DEBUG [as 別名]
def process_item(self, item, spider):
"""
Scrape edilen her girdiyi veritabanina ekle. Bu method sayfa process edildikten, icerisindeki
bilgiler cekildikten ve Item objesi olusturulduktan sonra her seferinde cagriliyor.
:param item: Parse edilmis nesne
:type item: Scrapy item
:param spider: Su anda calisan, spiders/ dizini altinda belirtilen spiderlardan herhangi biri
:type spider: Scrapy spider
:return: Gonderilen Item
:rtype: Scrapy item
"""
log.msg('[%s] PROCESSING ITEM [item no: %s, baslik: %s]' %
(spider.name, item['girdi_id'], item['baslik']),
level=log.DEBUG)
girdi = Girdi(**item)
try:
session.add(girdi)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
return item 示例2: request_seen
# 需要導入模塊: from scrapy import log [as 別名]
# 或者: from scrapy.log import DEBUG [as 別名]
def request_seen(self, request):
is_seen = is_request_seen(request)
if not is_seen:
log.msg('New URL: %s. Adding it to seen database' % request.url, log.DEBUG)
seen = Seen(fingerprint=request_fingerprint(request),
url=request.url,
last_crawl_time=datetime.now())
try:
session.add(seen)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
else:
log.msg('[seen] "%s" is seen. Skipping.' % request.url, log.INFO)
return is_seen 示例3: debug
# 需要導入模塊: from scrapy import log [as 別名]
# 或者: from scrapy.log import DEBUG [as 別名]
def debug(msg):
log.msg(str(msg), level=log.DEBUG) 示例4: process_spider_output
# 需要導入模塊: from scrapy import log [as 別名]
# 或者: from scrapy.log import DEBUG [as 別名]
def process_spider_output(self, response, result, spider):
for x in result:
if isinstance(x, Request) and hasattr(spider, 'disallow_urls'):
if self.should_follow(x, spider):
yield x
else:
log.msg("Filtered URL %s: " % (x.url),
level=log.DEBUG, spider=spider)
else:
yield x 示例5: dropped
# 需要導入模塊: from scrapy import log [as 別名]
# 或者: from scrapy.log import DEBUG [as 別名]
def dropped(self, item, exception, response, spider):
return {
'level': log.DEBUG,
'msg': logformatter.DROPPEDMSG,
'args': {
'exception': exception,
'item': item,
}
}