当前位置: 首页>>代码示例>>Python>>正文


Python Comment.link_id方法代码示例

本文整理汇总了Python中models.Comment.link_id方法的典型用法代码示例。如果您正苦于以下问题:Python Comment.link_id方法的具体用法?Python Comment.link_id怎么用?Python Comment.link_id使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在models.Comment的用法示例。


在下文中一共展示了Comment.link_id方法的1个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Python代码示例。

示例1: _process_comments

# 需要导入模块: from models import Comment [as 别名]
# 或者: from models.Comment import link_id [as 别名]
    def _process_comments(self, url, next_newest_id, data, after):
        """Threaded comment parser"""
        
        comments = data['data']['children']
        new_comments = 0
        pages_parsed = 0
        
        for i, c in enumerate(comments):
            comment = c['data']

            if comment['id'] <= next_newest_id: # comments ids are generated sequentially
                # so we can safely compare them like this without having to cache a timestamp
                # and without risking problems if a comment is deleted
                self.logger.debug("%s: comment['id'] <= next_newest_id" % current_process().name)
                break
            
            # Create new Comment, stick in it in the database.
            # I feel like we should be able to loop through the JSON object and avoid writing out all these assignments
            # Would that even be a good idea?
            comment_entry = Comment()
            comment_entry.subreddit_id           = comment['subreddit_id']
            comment_entry.link_title             = comment['link_title']
            comment_entry.subreddit              = comment['subreddit']
            comment_entry.link_author            = comment['link_author']
            comment_entry.id                     = comment['id']
            comment_entry.gilded                 = comment['gilded']
            comment_entry.author                 = comment['author']
            comment_entry.parent_id              = comment['parent_id']
            comment_entry.body                   = comment['body']
            comment_entry.edited                 = comment['edited']
            comment_entry.author_flair_css_class = comment['author_flair_css_class']
            comment_entry.downs                  = comment['downs']
            comment_entry.body_html              = comment['body_html']
            comment_entry.link_id                = comment['link_id']
            comment_entry.score_hidden           = comment['score_hidden']
            comment_entry.name                   = comment['name']
            comment_entry.created                = datetime.fromtimestamp(comment['created'])
            comment_entry.author_flair_text      = comment['author_flair_text']
            comment_entry.created_utc            = datetime.fromtimestamp(comment['created_utc'])
            comment_entry.ups                    = comment['ups']
            comment_entry.distinguished          = comment['distinguished']
            
            # Append to current session. We'll commit at the end.
            session.add(comment_entry)

            # update our counter
            # For informational purposes. Maybe I'll have it propagate a log in the database,
            #  that would make it really easy to graph comment rate over time           
            new_comments = new_comments + 1
        else:
            # we didn't break out of the loop, therefore we never encountered next_newest_id
            # so, get the next page and keep going
            data, newest_id, after = self._parse_json(url, after)
            
            if pages_parsed <= 3:
                # we should never have to go back further than two additional pages
                # if we have, something is probably wrong            
                self.logger.debug("Getting a second page.")
                pages_parsed = pages_parsed + 1
                self._process_comments(url, next_newest_id, data, after)
            else:
                self.logger.debug("Attempted to parse more than three pages.")

        # Push new objects to database
        session.commit()
        
        self.logger.debug( "%s: %s %s" % (current_process().name, datetime.now(), new_comments) )
开发者ID:Dakta,项目名称:reddit-scraper,代码行数:69,代码来源:scrape_comments.py


注:本文中的models.Comment.link_id方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。