当前位置: 首页>>代码示例>>Java>>正文


Java PigStats.result方法代码示例

本文整理汇总了Java中org.apache.pig.tools.pigstats.PigStats.result方法的典型用法代码示例。如果您正苦于以下问题:Java PigStats.result方法的具体用法?Java PigStats.result怎么用?Java PigStats.result使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在org.apache.pig.tools.pigstats.PigStats的用法示例。


在下文中一共展示了PigStats.result方法的2个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: testTFileRecordReader

import org.apache.pig.tools.pigstats.PigStats; //导入方法依赖的package包/类
@Test
public void testTFileRecordReader() throws Exception {
    PrintWriter w = new PrintWriter(new FileWriter("1.txt"));
    for (int i = 0; i < 30; i++) {
        w.println("1\tthis is a test for compression of temp files");
    }
    w.close();
    
    Util.copyFromLocalToCluster(cluster, "1.txt", "1.txt");
    
    PrintWriter w1 = new PrintWriter(new FileWriter("tfile.pig"));
    w1.println("A = load '1.txt' as (a0:int, a1:chararray);");
    w1.println("B = group A by a0;");
    w1.println("store B into 'tfile' using org.apache.pig.impl.io.TFileStorage();");
    w1.close();
    
    PrintWriter w2 = new PrintWriter(new FileWriter("tfile2.pig"));
    w2.println("A = load 'tfile' using org.apache.pig.impl.io.TFileStorage() as (a:int, b:bag{(b0:int, b1:chararray)});");
    w2.println("B = foreach A generate flatten($1);");
    w2.println("store B into '2.txt';");
    w2.close();
    
    try {
        String[] args = { "-Dpig.tmpfilecompression.codec=gz",
                "-Dtfile.io.chunk.size=100", "tfile.pig" };
        PigStats stats = PigRunner.run(args, null);
 
        assertTrue(stats.isSuccessful());
 
        String[] args2 = { "-Dpig.tmpfilecompression.codec=gz",
                "-Dtfile.io.chunk.size=100", "tfile2.pig" };
        PigStats stats2 = PigRunner.run(args2, null);

        assertTrue(stats2.isSuccessful());
        
        OutputStats os = stats2.result("B");
        Iterator<Tuple> iter = os.iterator();
        int count = 0;
        String expected = "(1,this is a test for compression of temp files)";
        while (iter.hasNext()) {
            count++;
            assertEquals(expected, iter.next().toString());
        }
        assertEquals(30, count);
        
    } finally {
        new File("tfile.pig").delete(); 
        new File("tfile2.pig").delete(); 
        new File("1.txt").delete(); 
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:52,代码来源:TestTmpFileCompression.java

示例2: testTFileRecordReader

import org.apache.pig.tools.pigstats.PigStats; //导入方法依赖的package包/类
@Test
public void testTFileRecordReader() throws Exception {
    PrintWriter w = new PrintWriter(new FileWriter("1.txt"));
    for (int i = 0; i < 30; i++) {
        w.println("1\tthis is a test for compression of temp files");
    }
    w.close();

    Util.copyFromLocalToCluster(cluster, "1.txt", "1.txt");

    PrintWriter w1 = new PrintWriter(new FileWriter("tfile.pig"));
    w1.println("A = load '1.txt' as (a0:int, a1:chararray);");
    w1.println("B = group A by a0;");
    w1.println("store B into 'tfile' using org.apache.pig.impl.io.TFileStorage();");
    w1.close();

    PrintWriter w2 = new PrintWriter(new FileWriter("tfile2.pig"));
    w2.println("A = load 'tfile' using org.apache.pig.impl.io.TFileStorage() as (a:int, b:bag{(b0:int, b1:chararray)});");
    w2.println("B = foreach A generate flatten($1);");
    w2.println("store B into '2.txt';");
    w2.close();

    try {
        String[] args = { "-Dpig.tmpfilecompression.codec=gz",
                "-Dtfile.io.chunk.size=100", "tfile.pig" };
        PigStats stats = PigRunner.run(args, null);

        assertTrue(stats.isSuccessful());

        String[] args2 = { "-Dpig.tmpfilecompression.codec=gz",
                "-Dtfile.io.chunk.size=100", "tfile2.pig" };
        PigStats stats2 = PigRunner.run(args2, null);

        assertTrue(stats2.isSuccessful());

        OutputStats os = stats2.result("B");
        Iterator<Tuple> iter = os.iterator();
        int count = 0;
        String expected = "(1,this is a test for compression of temp files)";
        while (iter.hasNext()) {
            count++;
            assertEquals(expected, iter.next().toString());
        }
        assertEquals(30, count);

    } finally {
        new File("tfile.pig").delete();
        new File("tfile2.pig").delete();
        new File("1.txt").delete();
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:52,代码来源:TestTmpFileCompression.java


注:本文中的org.apache.pig.tools.pigstats.PigStats.result方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。