本文整理汇总了Python中durus.connection.Connection.get_crawler方法的典型用法代码示例。如果您正苦于以下问题：Python Connection.get_crawler方法的具体用法？Python Connection.get_crawler怎么用？Python Connection.get_crawler使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类durus.connection.Connection的用法示例。
在下文中一共展示了Connection.get_crawler方法的2个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: main
# 需要导入模块: from durus.connection import Connection [as 别名]
# 或者: from durus.connection.Connection import get_crawler [as 别名]
def main():
from shutil import copyfile
from os.path import exists
def usage():
sys.stdout.write(
"Usage: python %s <existing_file> <new_file>\n" % sys.argv[0])
sys.stdout.write(" Creates a new py3k-compatible file ")
sys.stdout.write("from an existing FileStorage file.\n")
raise SystemExit
if len(sys.argv) != 3:
usage()
infile = sys.argv[1]
outfile = sys.argv[2]
if not exists(infile):
usage()
if exists(outfile):
if input('overwrite %r? [y/N] ' % outfile).strip().lower() != 'y':
raise SystemExit
copyfile(infile, outfile)
# monkey patch pickler class, must be done before importing durus stuff
patch_pickler()
from durus.__main__ import get_storage_class
from durus.connection import Connection
storage_class = get_storage_class(outfile)
storage = storage_class(outfile)
connection = Connection(storage)
print ("Converting %s for use with py3k." % outfile)
for j, x in enumerate(connection.get_crawler()):
x._p_note_change()
if j > 0 and j % 10000 == 0:
print(j)
connection.commit()
print(j)
connection.commit()
connection.pack()示例2: usage
# 需要导入模块: from durus.connection import Connection [as 别名]
# 或者: from durus.connection.Connection import get_crawler [as 别名]
from durus.connection import Connection
from durus.file_storage import FileStorage
from shutil import copyfile
def usage():
sys.stdout.write(
"Usage: python %s <existing_file> <new_file>\n" % sys.argv[0])
sys.stdout.write(" Creates a new py3k-compatible file ")
sys.stdout.write("from an existing FileStorage file.\n")
raise SystemExit
if len(sys.argv) != 3:
usage()
from os.path import exists
if not exists(sys.argv[1]):
usage()
if exists(sys.argv[2]):
usage()
copyfile(sys.argv[1], sys.argv[2])
storage = FileStorage(sys.argv[2])
connection = Connection(storage)
print ("Converting %s for use with py3k." % sys.argv[2])
for j, x in enumerate(connection.get_crawler()):
x._p_note_change()
if j > 0 and j % 10000 == 0:
print(j)
connection.commit()
print(j)
connection.commit()
connection.pack()