本文整理汇总了Python中scraper.Scraper.scrap方法的典型用法代码示例。如果您正苦于以下问题：Python Scraper.scrap方法的具体用法？Python Scraper.scrap怎么用？Python Scraper.scrap使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类scraper.Scraper的用法示例。
在下文中一共展示了Scraper.scrap方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: main
# 需要导入模块: from scraper import Scraper [as 别名]
# 或者: from scraper.Scraper import scrap [as 别名]
def main(request):
username = request.POST['username']
password = request.POST['password']
scraper = Scraper(username, password)
data = scraper.scrap(True)
return HttpResponse(obj2json(data), mimetype='application/json')示例2: main
# 需要导入模块: from scraper import Scraper [as 别名]
# 或者: from scraper.Scraper import scrap [as 别名]
def main():
p = optparse.OptionParser()
p.add_option('--username', '-u')
p.add_option('--password', '-p')
options, arguments = p.parse_args()
if options.username and options.password:
scraper = Scraper(options.username, options.password.decode('utf-8'))
try:
student = scraper.scrap()
print '%s - %s' % (student.name, student.course)
for semester in student.semesters:
print '\nSemester: %s' % semester.period
for discipline in semester.disciplines:
print 'Discipline: %s' % discipline.name
for grade in discipline.grades:
print '%s - %s' % (grade.name.decode('utf-8'), grade.value)
except Exception, e:
print 'Could not fetch grades: %s' % e.message示例3: open
# 需要导入模块: from scraper import Scraper [as 别名]
# 或者: from scraper.Scraper import scrap [as 别名]
from scraper import Scraper
if __name__ == "__main__":
links = []
doc = open("URLs.csv", "r").read()
try:
for line in doc.split("\n"):
l = line.split(";")[-2]
links.append("https://www.sec.gov/Archives/" + l + "-index.htm")
except:
pass
errors = open("errors.txt", "w")
try:
for index, link in enumerate(links[:1000]):
# try:
print index, Scraper.scrap(link)
print
# except Exception as ex:
# errors.write(str(index) + ": " + str(link) + "\n")
finally:
errors.close()