Python DataFlowHook.start_template_dataflow方法代碼示例

本文整理匯總了Python中airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook.start_template_dataflow方法的典型用法代碼示例。如果您正苦於以下問題：Python DataFlowHook.start_template_dataflow方法的具體用法？Python DataFlowHook.start_template_dataflow怎麽用？Python DataFlowHook.start_template_dataflow使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook的用法示例。

在下文中一共展示了DataFlowHook.start_template_dataflow方法的4個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Python代碼示例。

示例1: execute

# 需要導入模塊: from airflow.contrib.hooks.gcp_dataflow_hook import DataFlowHook [as 別名]
# 或者: from airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook import start_template_dataflow [as 別名]
    def execute(self, context):
        hook = DataFlowHook(gcp_conn_id=self.gcp_conn_id,
                            delegate_to=self.delegate_to,
                            poll_sleep=self.poll_sleep)

        hook.start_template_dataflow(self.task_id, self.dataflow_default_options,
                                     self.parameters, self.template)

開發者ID:caseybrown89，項目名稱:airflow，代碼行數:9，代碼來源:dataflow_operator.py

示例2: DataFlowTemplateHookTest

# 需要導入模塊: from airflow.contrib.hooks.gcp_dataflow_hook import DataFlowHook [as 別名]
# 或者: from airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook import start_template_dataflow [as 別名]
class DataFlowTemplateHookTest(unittest.TestCase):

    def setUp(self):
        with mock.patch(BASE_STRING.format('GoogleCloudBaseHook.__init__'),
                        new=mock_init):
            self.dataflow_hook = DataFlowHook(gcp_conn_id='test')

    @mock.patch(DATAFLOW_STRING.format('DataFlowHook._start_template_dataflow'))
    def test_start_template_dataflow(self, internal_dataflow_mock):
        self.dataflow_hook.start_template_dataflow(
            task_id=TASK_ID, variables=DATAFLOW_OPTIONS_TEMPLATE, parameters=PARAMETERS,
            dataflow_template=TEMPLATE)
        internal_dataflow_mock.assert_called_once_with(
            mock.ANY, DATAFLOW_OPTIONS_TEMPLATE, PARAMETERS, TEMPLATE)

開發者ID:arihantsurana，項目名稱:incubator-airflow，代碼行數:16，代碼來源:test_gcp_dataflow_hook.py

示例3: DataFlowTemplateHookTest

# 需要導入模塊: from airflow.contrib.hooks.gcp_dataflow_hook import DataFlowHook [as 別名]
# 或者: from airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook import start_template_dataflow [as 別名]
class DataFlowTemplateHookTest(unittest.TestCase):

    def setUp(self):
        with mock.patch(BASE_STRING.format('GoogleCloudBaseHook.__init__'),
                        new=mock_init):
            self.dataflow_hook = DataFlowHook(gcp_conn_id='test')

    @mock.patch(DATAFLOW_STRING.format('DataFlowHook._start_template_dataflow'))
    def test_start_template_dataflow(self, internal_dataflow_mock):
        self.dataflow_hook.start_template_dataflow(
            job_name=JOB_NAME, variables=DATAFLOW_OPTIONS_TEMPLATE, parameters=PARAMETERS,
            dataflow_template=TEMPLATE)
        options_with_region = {'region': 'us-central1'}
        options_with_region.update(DATAFLOW_OPTIONS_TEMPLATE)
        internal_dataflow_mock.assert_called_once_with(
            mock.ANY, options_with_region, PARAMETERS, TEMPLATE)

    @mock.patch(DATAFLOW_STRING.format('_DataflowJob'))
    @mock.patch(DATAFLOW_STRING.format('DataFlowHook.get_conn'))
    def test_start_template_dataflow_with_runtime_env(self, mock_conn, mock_dataflowjob):
        dataflow_options_template = copy.deepcopy(DATAFLOW_OPTIONS_TEMPLATE)
        options_with_runtime_env = copy.deepcopy(RUNTIME_ENV)
        options_with_runtime_env.update(dataflow_options_template)

        dataflowjob_instance = mock_dataflowjob.return_value
        dataflowjob_instance.wait_for_done.return_value = None
        method = (mock_conn.return_value
                  .projects.return_value
                  .locations.return_value
                  .templates.return_value
                  .launch)

        self.dataflow_hook.start_template_dataflow(
            job_name=JOB_NAME,
            variables=options_with_runtime_env,
            parameters=PARAMETERS,
            dataflow_template=TEMPLATE
        )
        body = {"jobName": mock.ANY,
                "parameters": PARAMETERS,
                "environment": RUNTIME_ENV
                }
        method.assert_called_once_with(
            projectId=options_with_runtime_env['project'],
            location='us-central1',
            gcsPath=TEMPLATE,
            body=body,
        )

開發者ID:apache，項目名稱:incubator-airflow，代碼行數:50，代碼來源:test_gcp_dataflow_hook.py

示例4: DataFlowTemplateHookTest

# 需要導入模塊: from airflow.contrib.hooks.gcp_dataflow_hook import DataFlowHook [as 別名]
# 或者: from airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook import start_template_dataflow [as 別名]
class DataFlowTemplateHookTest(unittest.TestCase):

    def setUp(self):
        with mock.patch(BASE_STRING.format('GoogleCloudBaseHook.__init__'),
                        new=mock_init):
            self.dataflow_hook = DataFlowHook(gcp_conn_id='test')

    @mock.patch(DATAFLOW_STRING.format('DataFlowHook._start_template_dataflow'))
    def test_start_template_dataflow(self, internal_dataflow_mock):
        self.dataflow_hook.start_template_dataflow(
            job_name=JOB_NAME, variables=DATAFLOW_OPTIONS_TEMPLATE, parameters=PARAMETERS,
            dataflow_template=TEMPLATE)
        options_with_region = {'region': 'us-central1'}
        options_with_region.update(DATAFLOW_OPTIONS_TEMPLATE)
        internal_dataflow_mock.assert_called_once_with(
            mock.ANY, options_with_region, PARAMETERS, TEMPLATE)

開發者ID:Fokko，項目名稱:incubator-airflow，代碼行數:18，代碼來源:test_gcp_dataflow_hook.py

注：本文中的airflow.contrib.hooks.gcp_dataflow_hook.DataFlowHook.start_template_dataflow方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。