本文整理汇总了Scala中org.apache.spark.util.sketch.BloomFilter类的典型用法代码示例。如果您正苦于以下问题：Scala BloomFilter类的具体用法？Scala BloomFilter怎么用？Scala BloomFilter使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
在下文中一共展示了BloomFilter类的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Scala代码示例。
示例1: TreeBloom
//设置package包名称以及导入依赖的类
import org.apache.spark.util.sketch.BloomFilter
import org.apache.spark.sql._
import org.apache.spark.sql.catalyst.InternalRow
import Main.sc
object TreeBloom {
// Implements bloom filter using treeAggregate instead of aggregate
// See https://issues.apache.org/jira/browse/SPARK-21039
def bloomFilter(singleCol: DataFrame, expectedNumItems:Long, fpp:Double): BloomFilter = {
val zero = BloomFilter.create(expectedNumItems, fpp)
sc.setJobGroup("bloomFilter", "Bloom filter creation")
singleCol.queryExecution.toRdd.treeAggregate(zero)(
(filter: BloomFilter, row: InternalRow) => {
filter.putLong(row.getInt(0))
filter
},
(filter1, filter2) => filter1.mergeInPlace(filter2)
)
}
}