本文整理汇总了Python中readability.Document.options['debug']方法的典型用法代码示例。如果您正苦于以下问题：Python Document.options['debug']方法的具体用法？Python Document.options['debug']怎么用？Python Document.options['debug']使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类readability.Document的用法示例。
在下文中一共展示了Document.options['debug']方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: parseitem
# 需要导入模块: from readability import Document [as 别名]
# 或者: from readability.Document import options['debug'] [as 别名]
def parseitem(self,response):
':type response: Response'
if 'Please turn on JavaScript' in response.body:
body = response.body
body = re.sub('<p class="caption"[^<]+', '', body)
body = re.sub('<noscript>(.|\r|\n)*?</noscript>','',body)
response = response.replace(body=body)
sel = Selector(response)
item = NewsscraperItem()
### storing the name of URL and source in item dictionary
item['url']= response.url
item['source']= self.name
### extracting the time of scraping of data inside the item
item['dateScraped']= strftime("%Y-%m-%d %H:%M:%S", gmtime())
### checking for url category defined in allowed domain. If response.url contain the string then move in otherwise in else condition
try:
if 'www.bbc.co.uk' in response.url:
### extracting title
title = sel.xpath("//h1[starts-with(@class,'story')]/text()").extract()
if(title):
### extracting title from the page and checking different xpath for searching the title
item['title']=title[0].strip()
### extracting date from the page using xpath
d = sel.xpath("//span[@class='date']/text()").extract()[0].strip()
###string to datetime conversion
f = strptime(d,'%d %B %Y')
###formating date in a particular format defined in config file
item['date']= strftime(Config['dateformat'],f)
### extracting content from the page
x = sel.xpath("(//div[@class='story-body']//*[self::p or self::strong]/text()) |(//span[@class='cross-head']/text())|(//div[@class='story-body']/p/a/text())").extract()
if len(x) > 1:
st="\n"
p = st.join(x)
### using regular expression to remove continuous white spaces from the content and replace by single space
item['content']= re.sub(r"[ \t\n]+", " ",p)
else:
# Not able to extract article content using xpath. Move to backup approach and use readability
try:
html = sel.xpath("//div[@class = 'story-body']").extract()
doc = Document(html)
doc.options['debug'] = False
try:
logging.basicConfig(level=logging.CRITICAL)
htmlContent = doc.summary()
content = html2texthandler(htmlContent)
except Exception, e:
pass
finally:
logging.basicConfig(level=logging.INFO)
item['content']= re.sub(r"[ \t\n\"]", " ",content)
except:
return