本文整理汇总了Java中com.google.cloud.dataflow.sdk.values.KV.of方法的典型用法代码示例。如果您正苦于以下问题:Java KV.of方法的具体用法?Java KV.of怎么用?Java KV.of使用的例子?那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类com.google.cloud.dataflow.sdk.values.KV
的用法示例。
在下文中一共展示了KV.of方法的5个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: getCurrent
import com.google.cloud.dataflow.sdk.values.KV; //导入方法依赖的package包/类
/**
* Get the next file in queue.
* @return A Key-Value pair where the key is a list of strings representing the path of
* the file and the value is a string representing the content of the file.
* @throws NoSuchElementException If the file can't be read from the GCS API.
*/
@Override
public KV<List<String>, String> getCurrent() throws NoSuchElementException {
String filePath = this.currentFiles.get(0);
String fileContent = null;
try {
fileContent = this.source.getFileContent(filePath);
} catch (IOException ioe) {
throw new NoSuchElementException(
"Object " + filePath + " not found in bucket " + this.source.bucket);
} catch (GeneralSecurityException gse) {
throw new NoSuchElementException(
"Cannot access object "
+ filePath
+ " in bucket "
+ this.source.bucket
+ " due to security reasons");
}
List<String> splitPath = Arrays.asList(filePath.split(this.source.getDirDelimiter()));
return KV.of(splitPath, fileContent);
}
示例2: apply
import com.google.cloud.dataflow.sdk.values.KV; //导入方法依赖的package包/类
@Override
public KV<String, WorkflowArgs> apply(Iterable<KV<String, WorkflowArgs>> input) {
String key = null;
WorkflowArgs retval = null;
// Merge arguments
for (KV<String, WorkflowArgs> kv : input) {
// Modify a copy
WorkflowArgs wa = new WorkflowArgs(kv.getValue());
// First time, nothing to merge
if (retval == null) {
key = kv.getKey();
retval = wa;
// Find differences and merge
} else {
retval.gatherArgs(wa);
}
}
return KV.of(key, retval);
}
示例3: doMutation_encodesKeysAndCounts
import com.google.cloud.dataflow.sdk.values.KV; //导入方法依赖的package包/类
@Test
public void doMutation_encodesKeysAndCounts() {
// Arrange
DoFnTester<KV<String, Integer>, Mutation> tester = DoFnTester.of(LoadBooks.ENCODE_NGRAM);
KV<String, Integer> input = KV.of("this is a test", 513);
// Act
List<Mutation> output = tester.processBatch(input);
// Assert
Put put = (Put) output.get(0);
assertThat(put.getRow()).isEqualTo("this is a test".getBytes(StandardCharsets.UTF_8));
Cell valueCell = put.get(LoadBooks.FAMILY, LoadBooks.COUNT_QUALIFIER).get(0);
byte[] valueArray = valueCell.getValueArray();
byte[] value =
Arrays.copyOfRange(
valueArray,
valueCell.getValueOffset(),
valueCell.getValueOffset() + valueCell.getValueLength());
assertThat(value).isEqualTo(new byte[] {0, 0, 2, 1});
}
示例4: apply
import com.google.cloud.dataflow.sdk.values.KV; //导入方法依赖的package包/类
@Override
public KV<LatLon, TableRow> apply(TableRow t) {
float lat = Float.parseFloat(t.get("latitude").toString());
float lon = Float.parseFloat(t.get("longitude").toString());
final float PRECISION = 0.005f; // very approximately 500m
float roundedLat = (float) Math.floor(lat / PRECISION) * PRECISION + PRECISION / 2;
float roundedLon = (float) Math.floor(lon / PRECISION) * PRECISION + PRECISION / 2;
LatLon key = new LatLon(roundedLat, roundedLon);
return KV.of(key, t);
}
示例5: processElement
import com.google.cloud.dataflow.sdk.values.KV; //导入方法依赖的package包/类
@Override
public void processElement(
DoFn<KV<String, TSAggValueProto>, KV<String, WorkDataPoint>>.ProcessContext c)
throws Exception {
// Extract only the needed data to do the work and create Proto
TSProto openFX = c.element().getValue().getOpenState();
TSProto closeFX = c.element().getValue().getCloseState();
double closeOverOpen = closeFX.getAskPrice() / openFX.getAskPrice();
double logRtn = Math.log(closeOverOpen);
String key = c.element().getKey();
WorkDataPoint data =
WorkDataPoint.newBuilder().setKey(key).setTime(closeFX.getTime())
.setValue(logRtn).build();
String partition = c.window().maxTimestamp().toString();
KV<String, WorkDataPoint> bars = KV.of(partition, data);
c.outputWithTimestamp(bars, c.timestamp());
c.sideOutput(tag, 1);
}