本文整理汇总了Python中luigi.util方法的典型用法代码示例。如果您正苦于以下问题:Python luigi.util方法的具体用法?Python luigi.util怎么用?Python luigi.util使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类luigi
的用法示例。
在下文中一共展示了luigi.util方法的1个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: requires
# 需要导入模块: import luigi [as 别名]
# 或者: from luigi import util [as 别名]
def requires(self):
yield self.clone(BuildLearnerProgramReport)
yield self.clone(BuildCohortProgramReportTask)
# Luigi has a great decorator, inherits, which "copies parameters (and nothing else) from one task class to another,
# and avoids direct pythonic inheritance". This is great in that it avoids the difficulties of sharing common
# parameters between dependent tasks. Read more here:
# https://luigi.readthedocs.io/en/stable/api/luigi.util.html#using-inherits-and-requires-to-ease-parameter-pain.
# However, we use a version of Luigi that has an older implementation of this decorator, which creates a
# new wrapper class that subclasses the wrapped class. Normally, this isn't a problem. But because we're calling
# super() in the run method, we run into a problem. Let's say that the subclassed class has name A. When we call
# the decorator on A, we are returned a wrapper version of A, which is assigned the name A. Therefore, in the MRO
# of class A, A appears twice. When super() in called in the run method, A is returned, leading to infinite recursion.
# In future versions of Luigi, this is fixed. This is a temporary workaround.
# Note that we do not use this decorator with any task that is directly downstream of ExportVerticaTableToS3Task
# due to issues with the "inheritance" of the sqoop_fields_terminated_by parameter, whose default value is
# u'\x01'; jobs running on EMR fail fails with the following error:
# "[Fatal Error] job.xml:209:161: Character reference "" is an invalid XML character."