本文整理汇总了Python中pyspark.mllib.linalg.DenseVector.parse方法的典型用法代码示例。如果您正苦于以下问题：Python DenseVector.parse方法的具体用法？Python DenseVector.parse怎么用？Python DenseVector.parse使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类pyspark.mllib.linalg.DenseVector的用法示例。
在下文中一共展示了DenseVector.parse方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: cat2Num
# 需要导入模块: from pyspark.mllib.linalg import DenseVector [as 别名]
# 或者: from pyspark.mllib.linalg.DenseVector import parse [as 别名]
def cat2Num(self, df, indices):
'''sbaronia - extract the categorical data and make df out of it
so oneHotEncoding can be run on them'''
protocol_ind0 = df.select(df.id,df.rawFeatures[indices[0]].alias("features0")).cache()
protocol_ind1 = df.select(df.id,df.rawFeatures[indices[1]].alias("features1")).cache()
ind0_enc = self.oneHotEncoding(protocol_ind0,"features0").cache()
ind1_enc = self.oneHotEncoding(protocol_ind1,"features1").cache()
'''sbaronia - add those hot encoded features columns to original df'''
int_join_1 = df.join(ind0_enc, ind0_enc.id == df.id, 'inner').drop(ind0_enc.id).cache()
int_join_2 = int_join_1.join(ind1_enc, int_join_1.id == ind1_enc.id, 'inner').drop(int_join_1.id).cache()
'''sbaronia - now create a new column features which has
converted vector form and drop rest columns'''
comb_udf = udf(replaceCat2Num,StringType())
int_join_2 = int_join_2.select(int_join_2.id,int_join_2.rawFeatures, \
comb_udf(int_join_2.rawFeatures, \
int_join_2.num_features0, \
int_join_2.num_features1).alias("features")).cache()
'''sbaronia - convert list of numerical features to DenseVector
so they can be used in KMeans'''
dense_udf = udf(lambda line: DenseVector.parse(line), VectorUDT())
feat = int_join_2.select(int_join_2.id,int_join_2.rawFeatures,dense_udf(int_join_2.features).alias("features")).cache()
return feat