当前位置: 首页>>代码示例>>Java>>正文

Java PipelineOptionsFactory.create方法代码示例

本文整理汇总了Java中com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.create方法的典型用法代码示例。如果您正苦于以下问题：Java PipelineOptionsFactory.create方法的具体用法？Java PipelineOptionsFactory.create怎么用？Java PipelineOptionsFactory.create使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory的用法示例。


在下文中一共展示了PipelineOptionsFactory.create方法的14个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: testBundleSplitIsJustSource

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testBundleSplitIsJustSource() throws Exception {
  PipelineOptions options = PipelineOptionsFactory.create();
  List<GCSFilesSource> bundles = source.splitIntoBundles(0, null);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(0, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(1, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(100000, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(10, null);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:24，代码来源:GCSFilesSourceTest.java

示例2: testReaderGetCurrent

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testReaderGetCurrent() {
  String projectName = "sampleProject";
  String objectName = REPOSITORY + this.source.getDirDelimiter() + projectName;
  String fileContent = "sample file content";
  ByteArrayOutputStream[] out = new ByteArrayOutputStream[1];
  PipelineOptions options = PipelineOptionsFactory.create();

  setUpGetFilesPage(objectName);
  setUpGetFileContent(fileContent, out);

  try {
    BoundedReader<KV<List<String>, String>> reader = this.source.createReader(options);
    reader.start();
    KV<List<String>, String> value = reader.getCurrent();
    assertEquals(value.getKey().size(), 2);
    assertEquals(value.getKey().get(0), REPOSITORY);
    assertEquals(value.getKey().get(1), projectName);
    assertEquals(value.getValue(), fileContent);
  } catch (IOException e) {
    fail();
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:24，代码来源:GCSFilesSourceTest.java

示例3: testReaderAdvance

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testReaderAdvance() {
  String objectName = REPOSITORY + this.source.getDirDelimiter() + "sampleProject";
  PipelineOptions options = PipelineOptionsFactory.create();
  BoundedReader<KV<List<String>, String>> reader;

  try {
    setUpGetFilesPage(objectName, 0);
    reader = this.source.createReader(options);
    assertFalse(reader.start());

    setUpGetFilesPage(objectName, 1);
    reader = this.source.createReader(options);
    assertTrue(reader.start());
    assertFalse(reader.advance());

    setUpGetFilesPage(objectName, 2);
    reader = this.source.createReader(options);
    assertTrue(reader.start());
    assertTrue(reader.advance());
    assertFalse(reader.advance());
  } catch (IOException e) {
    fail();
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:26，代码来源:GCSFilesSourceTest.java

示例4: testBundleSplitIsJustSource

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testBundleSplitIsJustSource() throws Exception {
  PipelineOptions options = PipelineOptionsFactory.create();
  List<LiveProjectSource> bundles = source.splitIntoBundles(0, null);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(0, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(1, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(100000, options);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);

  bundles = source.splitIntoBundles(10, null);
  assertEquals(bundles.size(), 1);
  assertEquals(bundles.get(0), source);
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:24，代码来源:LiveProjectSourceTest.java

示例5: testAdvanceWithoutStart

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testAdvanceWithoutStart() {
  PipelineOptions options = PipelineOptionsFactory.create();
  LiveProjectReader reader;

  this.listProjectsResponse.setProjects(new ArrayList<Project>(0));
  this.listProjectsResponse.setNextPageToken(null);
  try {
    reader = (LiveProjectReader) this.source.createReader(options);
    assertFalse(reader.advance());
    assertNull(reader.getNextPageToken());
    assertTrue(reader.getProjects().isEmpty());
    reader.getCurrent();
  } catch (IOException e) {
    fail("IOException in reader.start");
  } catch (NoSuchElementException ignored) {
    // test passed.
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:20，代码来源:LiveProjectSourceTest.java

示例6: testReaderStart

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testReaderStart() {
  String objectName = REPOSITORY + this.source.getDirDelimiter() + "sampleProject";
  PipelineOptions options = PipelineOptionsFactory.create();
  setUpGetFilesPage(objectName);
  try {
    assertTrue(this.source.createReader(options).start());
  } catch (IOException e) {
    fail();
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:12，代码来源:GCSFilesSourceTest.java

示例7: testAdvanceWhenPageTokenNull

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testAdvanceWhenPageTokenNull() {
  String projectName = "sampleProjectName";
  String projectId = "sampleProjectId";
  String orgId = ORG;
  ResourceId resourceId = new ResourceId().setId(orgId);
  GCPProject gcpProject = new GCPProject(projectId, orgId, projectName);
  Project project =
      new Project()
          .setProjectId(projectId)
          .setParent(resourceId)
          .setName(projectName)
          .setLifecycleState("ACTIVE");
  List<Project> projects = Arrays.asList(project);
  PipelineOptions options = PipelineOptionsFactory.create();
  LiveProjectReader reader;

  this.listProjectsResponse.setProjects(projects);
  this.listProjectsResponse.setNextPageToken(null);
  try {
    reader = (LiveProjectReader) this.source.createReader(options);
    assertTrue(reader.start());
    assertEquals(reader.getNextPageToken(), null);
    assertEquals(reader.getCurrent(), gcpProject);
    assertFalse(reader.advance());
    reader.getCurrent();
    fail("No exception when reading from empty source");
  } catch (IOException e) {
    fail("IOException in reader.start");
  } catch (NoSuchElementException ignored) {
    // test passed.
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:34，代码来源:LiveProjectSourceTest.java

示例8: getLocalExecutionOptions

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
private PipelineOptions getLocalExecutionOptions() {
  return PipelineOptionsFactory.create();
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:4，代码来源:LiveStateCheckerApp.java

示例9: getLocalExecutionOptions

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
private static PipelineOptions getLocalExecutionOptions() {
  return PipelineOptionsFactory.create();
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:4，代码来源:LiveStateCheckerRunner.java

示例10: testProducesSortedKeys

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testProducesSortedKeys() throws Exception {
  PipelineOptions options = PipelineOptionsFactory.create();
  assertFalse(source.producesSortedKeys(options));
  assertFalse(source.producesSortedKeys(null));
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:7，代码来源:GCSFilesSourceTest.java

示例11: testAdvance

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testAdvance() {
  String projectName = "sampleProjectName";
  String projectId = "sampleProjectId";
  String orgId = "sampleOrgId";
  ResourceId resourceId = new ResourceId().setId(orgId);
  GCPProject gcpProject = new GCPProject(projectId, orgId, projectName);
  Project project =
      new Project()
          .setProjectId(projectId)
          .setParent(resourceId)
          .setName(projectName)
          .setLifecycleState("ACTIVE");
  List<Project> projects = new ArrayList<>();
  String nextPageToken = null;
  PipelineOptions options = PipelineOptionsFactory.create();
  LiveProjectReader reader;

  projects = Arrays.asList(project);
  nextPageToken = "samplePageToken";
  this.listProjectsResponse.setProjects(projects);
  this.listProjectsResponse.setNextPageToken(nextPageToken);

  try {
    reader = (LiveProjectReader) this.source.createReader(options);
    assertTrue(reader.start());
    assertEquals(reader.getNextPageToken(), nextPageToken);
    assertEquals(reader.getProjects().size(), 1);
    assertEquals(reader.getCurrent(), gcpProject);

    this.listProjectsResponse.setNextPageToken(null);
    assertTrue(reader.advance());
    assertEquals(reader.getProjects().size(), 1);
    assertEquals(reader.getCurrent(), gcpProject);
    assertFalse(reader.advance());
    assertEquals(reader.getProjects().size(), 0);

    projects = Arrays.asList(project, project);
    this.listProjectsResponse.setProjects(projects);
    reader = (LiveProjectReader) this.source.createReader(options);
    assertTrue(reader.start());
    assertEquals(reader.getProjects().size(), 2);
    assertEquals(reader.getCurrent(), gcpProject);
    assertTrue(reader.advance());
    assertEquals(reader.getProjects().size(), 1);
    assertEquals(reader.getCurrent(), gcpProject);

    projects = new ArrayList<>();
    this.listProjectsResponse.setProjects(projects);
    assertFalse(reader.advance());
    assertEquals(reader.getProjects().size(), 0);
    assertFalse(reader.advance());
    assertEquals(reader.getProjects().size(), 0);
  } catch (IOException e) {
    fail("IOException in reader.start");
  }
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:58，代码来源:LiveProjectSourceTest.java

示例12: testPipeline

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testPipeline() throws IOException {
  String editorRole = "roles/editor";
  String editorMember = "serviceAccount:[email protected]";
  String editorMemberLive = "serviceAccount:[email protected]";
  String ownerRole = "roles/owner";
  String ownerMember = "user:[email protected]";
  String fileContent = "[\n"
      + "      {\n"
      + "        \"role\": \"" + ownerRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + ownerMember + "\"\n"
      + "        ]\n"
      + "      },\n"
      + "      {\n"
      + "        \"role\": \"" + editorRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + editorMember + "\"\n"
      + "        ]\n"
      + "      }\n"
      + "    ]";
  String filePath = ORG_ID + DELIM + PROJECT_ID + DELIM + POLICY_FILE;
  String projectName = "sampleProjectName";
  String projectId = PROJECT_ID;
  String orgId = ORG_ID;
  ResourceId resourceId = new ResourceId().setId(orgId);
  Project project =
      new Project()
          .setProjectId(projectId)
          .setParent(resourceId)
          .setName(projectName)
          .setLifecycleState("ACTIVE");
  Binding editorBinding = new Binding()
      .setRole(editorRole)
      .setMembers(Arrays.asList(editorMemberLive));
  Binding ownerBinding = new Binding()
      .setRole(ownerRole)
      .setMembers(Arrays.asList(ownerMember));
  List<Binding> bindings = Arrays.asList(ownerBinding, editorBinding);
  Policy iamPolicy = new Policy().setBindings(bindings);
  PipelineOptions options = PipelineOptionsFactory.create();

  setUpGetFileContent(fileContent);
  setUpGetFilesPage(filePath);
  when(listProjects.execute())
      .thenReturn(this.listProjectsResponse
          .setNextPageToken("halting string")
          .setProjects(Arrays.asList(project)));
  when(this.getIamPolicy.execute()).thenReturn(iamPolicy);

  GCPProject.setProjectsApiStub(this.projectsObject);

  // setting up the output objects.
  GCPProject gcpProject = new GCPProject(projectId, orgId, projectName);
  PolicyBinding ownerPolicyBinding = new PolicyBinding(ownerRole, Arrays.asList(ownerMember));
  PolicyBinding editorPolicyBinding =
      new PolicyBinding(editorRole, Arrays.asList(editorMember));
  PolicyBinding editorPolicyBindingLive =
      new PolicyBinding(editorRole, Arrays.asList(editorMemberLive));
  GCPResourcePolicy checkedPolicy = new GCPResourcePolicy(
      gcpProject, Arrays.asList(ownerPolicyBinding, editorPolicyBinding));
  GCPResourcePolicy livePolicy = new GCPResourcePolicy(
      gcpProject, Arrays.asList(ownerPolicyBinding, editorPolicyBindingLive));
  Map<StateSource, GCPResourceState> outputMap = new HashMap<>(2);
  outputMap.put(StateSource.DESIRED, checkedPolicy);
  outputMap.put(StateSource.LIVE, livePolicy);

  new OnDemandLiveStateChecker(options, this.checkedSource)
  .appendAssertContains(new String[]{constructMessage(gcpProject, outputMap)})
  .run();
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:72，代码来源:OnDemandLiveStateCheckerTest.java

示例13: testPipeline

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testPipeline() throws IOException {
  String editorRole = "roles/editor";
  String editorMember = "serviceAccount:[email protected]";
  String editorMemberLive = "serviceAccount:[email protected]";
  String ownerRole = "roles/owner";
  String ownerMember = "user:[email protected]";
  String fileContent = "[\n"
      + "      {\n"
      + "        \"role\": \"" + ownerRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + ownerMember + "\"\n"
      + "        ]\n"
      + "      },\n"
      + "      {\n"
      + "        \"role\": \"" + editorRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + editorMember + "\"\n"
      + "        ]\n"
      + "      }\n"
      + "    ]";
  String filePath = ORG_ID + DELIM + PROJECT_ID + DELIM + POLICY_FILE;
  String projectName = "sampleProjectName";
  String projectId = PROJECT_ID;
  String orgId = ORG_ID;
  ResourceId resourceId = new ResourceId().setId(orgId);
  Project project =
      new Project()
          .setProjectId(projectId)
          .setParent(resourceId)
          .setName(projectName)
          .setLifecycleState("ACTIVE");
  Binding editorBinding = new Binding()
      .setRole(editorRole)
      .setMembers(Arrays.asList(editorMemberLive));
  Binding ownerBinding = new Binding()
      .setRole(ownerRole)
      .setMembers(Arrays.asList(ownerMember));
  List<Binding> bindings = Arrays.asList(ownerBinding, editorBinding);
  Policy iamPolicy = new Policy().setBindings(bindings);
  PipelineOptions options = PipelineOptionsFactory.create();

  setUpGetFileContent(fileContent);
  setUpGetFilesPage(filePath);
  when(listProjects.execute())
      .thenReturn(this.listProjectsResponse
          .setNextPageToken("halting string")
          .setProjects(Arrays.asList(project)));
  when(this.getIamPolicy.execute()).thenReturn(iamPolicy);

  GCPProject.setProjectsApiStub(this.projectsObject);

  // setting up the output objects.
  GCPProject gcpProject = new GCPProject(projectId, orgId, projectName);
  PolicyBinding ownerPolicyBinding = new PolicyBinding(ownerRole, Arrays.asList(ownerMember));
  PolicyBinding editorPolicyBinding =
      new PolicyBinding(editorRole, Arrays.asList(editorMember));
  PolicyBinding editorPolicyBindingLive =
      new PolicyBinding(editorRole, Arrays.asList(editorMemberLive));
  GCPResourcePolicy desiredPolicy = new GCPResourcePolicy(
      gcpProject, Arrays.asList(ownerPolicyBinding, editorPolicyBinding));
  GCPResourcePolicy livePolicy = new GCPResourcePolicy(
      gcpProject, Arrays.asList(ownerPolicyBinding, editorPolicyBindingLive));
  GCPResourcePolicyDiff diff = GCPResourcePolicyDiff.diff(desiredPolicy, livePolicy);
  MessageConstructor messageConstructor =
      new MessageConstructor(gcpProject, desiredPolicy, livePolicy, diff);

  new LiveStateChecker(options, this.checkedSource, ORG_ID)
      .build()
      .appendAssertContains(new String[]{messageConstructor.constructMessage()})
      .run();
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:73，代码来源:LiveStateCheckerTest.java

示例14: testUnmatchedStatesOutputIsCorrect

import com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory; //导入方法依赖的package包/类
@Test
public void testUnmatchedStatesOutputIsCorrect() throws IOException {
  // create the policy for the live project
  String editorRole = "roles/editor";
  String editorMember = "serviceAccount:[email protected]";
  String ownerRole = "roles/owner";
  String ownerMember = "user:[email protected]";
  String fileContent = "[\n"
      + "      {\n"
      + "        \"role\": \"" + ownerRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + ownerMember + "\"\n"
      + "        ]\n"
      + "      },\n"
      + "      {\n"
      + "        \"role\": \"" + editorRole + "\",\n"
      + "        \"members\": [\n"
      + "          \"" + editorMember + "\"\n"
      + "        ]\n"
      + "      }\n"
      + "    ]";
  String liveProjectName = "someLiveProjectName";
  String liveProjectId = "someLiveProjectId";
  String orgId = ORG_ID;
  ResourceId resourceId = new ResourceId().setId(orgId);
  Project liveProject =
      new Project()
          .setProjectId(liveProjectId)
          .setParent(resourceId)
          .setName(liveProjectName)
          .setLifecycleState("ACTIVE");
  Binding editorBinding = new Binding()
      .setRole(editorRole)
      .setMembers(Arrays.asList(editorMember));
  Binding ownerBinding = new Binding()
      .setRole(ownerRole)
      .setMembers(Arrays.asList(ownerMember));
  List<Binding> bindings = Arrays.asList(ownerBinding, editorBinding);
  Policy iamPolicy = new Policy().setBindings(bindings);
  // when calling projects().list(), return the live project
  when(listProjects.execute())
  .thenReturn(this.listProjectsResponse
      .setNextPageToken("halting string")
      .setProjects(Arrays.asList(liveProject)));
  when(this.getIamPolicy.execute()).thenReturn(iamPolicy);

  // mock out the desired policy
  String desiredProjectId = "someKnownGoodProject";
  String desiredPolicyPath = ORG_ID + DELIM + desiredProjectId + DELIM + POLICY_FILE;

  setUpGetFileContent(fileContent);
  setUpGetFilesPage(desiredPolicyPath);

  PipelineOptions options = PipelineOptionsFactory.create();

  LiveStateChecker liveStateChecker =
      new LiveStateChecker(options, this.checkedSource, ORG_ID)
        .build();

  String[] expectedOutput = new String[] {
      "DESIRED:someKnownGoodProject",
      "LIVE:someLiveProjectId"
  };

  DataflowAssert
      .that(liveStateChecker.getUnmatchedStatesOutput())
      .containsInAnyOrder(expectedOutput);

  liveStateChecker.run();
}
开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:71，代码来源:LiveStateCheckerTest.java


注：本文中的com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.create方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。