Java DoFnTester类代码示例

本文整理汇总了Java中com.google.cloud.dataflow.sdk.transforms.DoFnTester类的典型用法代码示例。如果您正苦于以下问题：Java DoFnTester类的具体用法？Java DoFnTester怎么用？Java DoFnTester使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

DoFnTester类属于com.google.cloud.dataflow.sdk.transforms包，在下文中一共展示了DoFnTester类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: testConvertJsonToReceiptDoFn

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testConvertJsonToReceiptDoFn() {
    ConvertJsonToReceiptDoFn dofn = new ConvertJsonToReceiptDoFn();
    DoFnTester<String, Receipt> fnTester = DoFnTester.of(dofn);
    List<String> input = new ArrayList<>();
    input.add("{\"salesDate\": \"2013-02-10T13:45:30+0900\"," +
            "\"storeCode\": \"010\"," +
            "\"registerID\": \"123\"," +
            "\"receiptNumber\": 1," +
            "\"receiptBranchNumber\": 2," +
            "\"productCode\": \"301\"," +
            "\"quantity\": 3," +
            "\"price\": 108 }");
    List<Receipt> receipts = fnTester.processBatch(input);
    Receipt receipt1 = receipts.get(0);
    Assert.assertThat("", receipt1.getPrice(), is(108));
    Assert.assertThat("", receipt1.getQuantity(), is(3));
    Assert.assertThat("", receipt1.getStoreCode(), is("010"));
    Assert.assertThat("", receipt1.getProductCode(), is("301"));
    Assert.assertThat("", receipt1.getRegisterID(), is("123"));
    Assert.assertThat("", receipt1.getReceiptNumber(), is(1));
    Assert.assertThat("", receipt1.getReceiptBranchNumber(), is(2));
}

开发者ID:topgate，项目名称:retail-demo，代码行数:24，代码来源:MainPipelineTest.java

示例2: doMutation_encodesKeysAndCounts

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void doMutation_encodesKeysAndCounts() {
  // Arrange
  DoFnTester<KV<String, Integer>, Mutation> tester = DoFnTester.of(LoadBooks.ENCODE_NGRAM);
  KV<String, Integer> input = KV.of("this is a test", 513);

  // Act
  List<Mutation> output = tester.processBatch(input);

  // Assert
  Put put = (Put) output.get(0);
  assertThat(put.getRow()).isEqualTo("this is a test".getBytes(StandardCharsets.UTF_8));
  Cell valueCell = put.get(LoadBooks.FAMILY, LoadBooks.COUNT_QUALIFIER).get(0);
  byte[] valueArray = valueCell.getValueArray();
  byte[] value =
      Arrays.copyOfRange(
          valueArray,
          valueCell.getValueOffset(),
          valueCell.getValueOffset() + valueCell.getValueLength());
  assertThat(value).isEqualTo(new byte[] {0, 0, 2, 1});
}

开发者ID:GoogleCloudPlatform，项目名称:cloud-bigtable-examples，代码行数:22，代码来源:LoadBooksTest.java

示例3: testCSVToMapLineCombineFn

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testCSVToMapLineCombineFn() {

    final String[] csv = new String[] {
            "000,Tokyo,120",
            "001,Osaka,100",
            "002,Kyoto,140"
    };
    final List<String> csvlist = Arrays.asList(csv);

    Pipeline p = TestPipeline.create();
    PCollection<String> maplines = p.apply(Create.of(csvlist)).setCoder(StringUtf8Coder.of());
    PCollectionView<Map<String,String>> mapview = maplines.apply(Combine.globally(new CSVToMapLineCombineFn()).asSingletonView());

    final String[] dummy = new String[] {
            "000",
            "001",
            "002"
    };
    List<String> dummylist = Arrays.asList(dummy);
    DoFnTester<String,String> fnTester = DoFnTester.of(new AAA(mapview));
    fnTester.setSideInputInGlobalWindow(mapview, csvlist);

    //dummylines.apply(ParDo.of(fnTester));
    List<String> results = fnTester.processBatch(dummylist);
    System.out.println(results);

    //p.apply()
}

开发者ID:topgate，项目名称:retail-demo，代码行数:30，代码来源:MainPipelineTest.java

示例4: testExtractWordsFn

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
/** Example test that tests a specific DoFn. */
@Test
public void testExtractWordsFn() {
  DoFnTester<String, String> extractWordsFn =
      DoFnTester.of(new ExtractWordsFn());

  Assert.assertThat(extractWordsFn.processBatch(" some  input  words "),
                    CoreMatchers.hasItems("some", "input", "words"));
  Assert.assertThat(extractWordsFn.processBatch(" "),
                    CoreMatchers.<String>hasItems());
  Assert.assertThat(extractWordsFn.processBatch(" some ", " input", " words"),
                    CoreMatchers.hasItems("some", "input", "words"));
}

开发者ID:sinmetal，项目名称:iron-hippo，代码行数:14，代码来源:WordCountTest.java

示例5: setUp

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() throws IOException {
  this.projectsApiObject = mock(Projects.class);
  GCPProject.setProjectsApiStub(this.projectsApiObject);

  this.getIamPolicy = mock(Projects.GetIamPolicy.class);
  this.tester = DoFnTester.of(new ExtractState());
  when(this.projectsApiObject.getIamPolicy(anyString(), any(GetIamPolicyRequest.class)))
      .thenReturn(this.getIamPolicy);
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:11，代码来源:ExtractStateTest.java

示例6: setUp

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() throws IOException {
  GCPProject.setProjectsApiStub(this.projectsApiObject);
  this.projectsApiObject = mock(Projects.class);
  this.getIamPolicy = mock(GetIamPolicy.class);
  this.tester = DoFnTester.of(new ExtractState());
  when(this.projectsApiObject.getIamPolicy(anyString(), any(GetIamPolicyRequest.class)))
      .thenReturn(this.getIamPolicy);
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:10，代码来源:ListServiceAccountsTest.java

示例7: testFilterStateNoMatchingResources

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMatchingResources() {
  GCPProject checkedProject = getSampleProject("_checked");
  GCPProject liveProject = getSampleProject("_live");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(checkedProject, 1);
  GCPResourceState livePolicy = getSampleGCPResourcePolicy(liveProject, 2);
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) checkedProject, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) liveProject, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  FilterOutMatchingState function = new FilterOutMatchingState(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);
  assertEquals(0, results.size());
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:29，代码来源:FilterOutMatchingStateTest.java

示例8: testFilterStateNoMismatches

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMismatches() {
  GCPProject project = getSampleProject("");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
  GCPResourceState livePolicy = checkedPolicy;
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  FilterOutMatchingState function = new FilterOutMatchingState(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);
  assertEquals(0, results.size());
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:28，代码来源:FilterOutMatchingStateTest.java

示例9: testFilterStateOneMismatch

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateOneMismatch() {
  GCPProject project = getSampleProject("");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
  GCPResourceState livePolicy = getSampleGCPResourcePolicy(project, 2);
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  FilterOutMatchingState function = new FilterOutMatchingState(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  Map<StateSource, GCPResourceState> outputMap = new HashMap<>(2);
  outputMap.put(StateSource.DESIRED, checkedPolicy);
  outputMap.put(StateSource.LIVE, livePolicy);
  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> expectedOutput =
      Arrays.asList(KV.of((GCPResource) project, outputMap));
  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);

  assertEquals(expectedOutput, results);
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:34，代码来源:FilterOutMatchingStateTest.java

示例10: testFilterStateNoMatchingResources

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMatchingResources() {
  GCPProject checkedProject = getSampleProject("_checked");
  GCPProject liveProject = getSampleProject("_live");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(checkedProject, 1);
  GCPResourceState livePolicy = getSampleGCPResourcePolicy(liveProject, 2);
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) checkedProject, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) liveProject, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);
  assertEquals(0, results.size());
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:29，代码来源:JoinKnownGoodAndLiveStatesTest.java

示例11: testFilterStateNoMismatches

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMismatches() {
  GCPProject project = getSampleProject("");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
  GCPResourceState livePolicy = checkedPolicy;
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);
  assertEquals(1, results.size());
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:28，代码来源:JoinKnownGoodAndLiveStatesTest.java

示例12: testFilterStateOneMismatch

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateOneMismatch() {
  GCPProject project = getSampleProject("");
  GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
  GCPResourceState livePolicy = getSampleGCPResourcePolicy(project, 2);
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
  List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
      Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));

  PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
      pipeline.apply(Create.of(sideInputList)).setCoder(
          KvCoder.of(SerializableCoder.of(GCPResource.class),
              KvCoder.of(SerializableCoder.of(StateSource.class),
                  SerializableCoder.of(GCPResourceState.class))));
  PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
      View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);

  JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
  DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
      KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
  tester.setSideInputInGlobalWindow(view, sideInputList);

  Map<StateSource, GCPResourceState> outputMap = new HashMap<>(2);
  outputMap.put(StateSource.DESIRED, checkedPolicy);
  outputMap.put(StateSource.LIVE, livePolicy);
  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> expectedOutput =
      Arrays.asList(KV.of((GCPResource) project, outputMap));
  List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
      tester.processBatch(mainInputList);

  assertEquals(expectedOutput, results);
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:34，代码来源:JoinKnownGoodAndLiveStatesTest.java

示例13: GroupIntsTest

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test(dataProvider = "refAndInts")
public void GroupIntsTest(Integer[] integers, Set<KV<ReferenceShard, Integer>> s) {
    DoFnTester<Integer, KV<ReferenceShard, Integer>> fnTester = DoFnTester.of(new GroupIntsDoFn());
    List<KV<ReferenceShard, Integer>> kvs = fnTester.processBatch(integers);
    for (KV<ReferenceShard, Integer> r : kvs) {
        Assert.assertTrue(s.contains(r));
    }
}

开发者ID:broadinstitute，项目名称:gatk-dataflow，代码行数:9，代码来源:SampleUnitTest.java

示例14: testReadFromFileFn

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testReadFromFileFn(){
    List<SimpleInterval> intervals = Arrays.asList(new SimpleInterval("chr7:1-202"), new SimpleInterval("chr8:2-202"));
    DoFn<File, GATKRead> readfn = new DataflowUtils.LoadReadsFromFileFn(intervals, ValidationStringency.SILENT);

    File inputFile = new File(getToolTestDataDir(), "example_reads.bam");
    List<GATKRead> expected = getReadsFromFile(intervals, inputFile, false);

    DoFnTester<File, GATKRead> tester = DoFnTester.of(readfn);
    List<GATKRead> output = tester.processBatch(inputFile);

    Assert.assertTrue(ReadUtils.readListsAreEqualIgnoreUUID(output, expected), "Actual reads do not match expected reads");
}

开发者ID:broadinstitute，项目名称:gatk-dataflow，代码行数:14，代码来源:DataflowUtilsUnitTest.java

示例15: setUp

import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
  this.tester = DoFnTester.of(new FileToState());
}

开发者ID:GoogleCloudPlatform，项目名称:policyscanner，代码行数:5，代码来源:FileToStateTest.java

注：本文中的com.google.cloud.dataflow.sdk.transforms.DoFnTester类示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。