当前位置: 首页>>代码示例>>Python>>正文


Python Config.preconfilename方法代码示例

本文整理汇总了Python中config.config.Config.preconfilename方法的典型用法代码示例。如果您正苦于以下问题:Python Config.preconfilename方法的具体用法?Python Config.preconfilename怎么用?Python Config.preconfilename使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在config.config.Config的用法示例。


在下文中一共展示了Config.preconfilename方法的1个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Python代码示例。

示例1: __init__

# 需要导入模块: from config.config import Config [as 别名]
# 或者: from config.config.Config import preconfilename [as 别名]
class WgetLogValidator:
    def __init__(self):
        # regular expressions:
        # regexp denoting successful collection index/directory listing save:
        self.cindex      = re.compile(r'(index|listing).*saved')

        # regexp for timestamp string containing an 'http' hdf (or xml) target:
        self.timestamp   = r'--\d{4}-\d{2}-\d{2} *\d{2}:\d{2}:\d{2}--'
        self.target      = r'(http|ftp).*\.(hdf|nc).*$'
        self.regex_save_loc    = ".*('/.*/.*') saved.*"
        self.timestamp_and_target = re.compile( self.timestamp + ' *' + '(' + self.target + ')')    # save target as group

        self.regex_saved_file_loc = re.compile(r'=> (\'.*\')')

        # regexp denoting target was successfully downloaded (saved):
        self.saved       = re.compile(r'saved')

        # regexp denoting overall completion:
        self.finished    = re.compile(r'FINISHED')

        # regex for 'Remote file no newer than local file'
        self.regex_no_retrieve = re.compile(r'Remote file no newer than local file*not retrieving\.')

        # regex for 'failed: Network is unreachable'
        self.regex_unreachable_network = re.compile(r'Connecting.*failed: Network is unreachable')

        self.download_list = []
        self.saved_file_locs = []

        self.summary = 'Wget Log Summary (' + str(date.today()) + '):*************************************************************************\n\n'

        self.config = Config()

    def summary_str(self):
        return self.summary

    def downloaded_filenames(self):
        return self.download_list

    def saved_file_locs(self):
        return self.saved_file_locs

    def file_count(self):
        return len(self.download_list)

    def str_file_count(self):
        return "file download count = " + str(self.file_count())

    def validate_logs(self, logfiles):
        # write new download file paths to preconlist.txt
        precon = open(self.config.preconfilename(), 'w')

        for logfilename in logfiles:

            # logicals, re match objects for checking status of download requests:

            cindex_present  = False         # collection index presence
            downloading     = False         # True at start of a reqest, False when complete.
            err             = False         # True for incomplete downloads.
            tt_match        = None          # timestamp and target match object.
            tt_match_prev   = None          # prior match object for possible failed request
                                            # reporting.
            tt_match_this_line  = False     # timestamp and target match on this line.
            _finished_      = False         # 'FINISHED' all files in collection
            saved_match     = []            # 'saved' string match object
            errlist         = []            # list of failed download targets.
            should_finish   = False

            logfile = ''
            try:
                logfile = open(logfilename)
            except IOError as e:
                print e;

            # for individual file download checks, scan through the "noise" in the file,
            # looking for typical download start/end signatures, taking note of those that
            # are incomplete:

            for line in logfile:

                # timestamp and target request match?
                if self.timestamp_and_target.match(line):
                    should_finish = True

                #if timestamp_and_target.match(line) and not get_request.search(line):
                    tt_match_this_line=True
                    if tt_match:
                        tt_match_prev = tt_match
                    tt_match = self.timestamp_and_target.match(line)

                # "saved" string match?
                saved_match = self.saved.search(line)

                # check process status:
                if tt_match_this_line and not downloading:
                    # start of new request
                    downloading = True
                elif tt_match_this_line and downloading and \
                     tt_match.group(1)!=tt_match_prev.group(1):
                    # new download request without successful completion of prior one
#.........这里部分代码省略.........
开发者ID:munizas,项目名称:datafetchpipeline,代码行数:103,代码来源:wgetlogvalidator.py


注:本文中的config.config.Config.preconfilename方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。