当前位置: 首页>>代码示例>>Java>>正文


Java OperatorPlan类代码示例

本文整理汇总了Java中org.apache.pig.impl.plan.OperatorPlan的典型用法代码示例。如果您正苦于以下问题:Java OperatorPlan类的具体用法?Java OperatorPlan怎么用?Java OperatorPlan使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。


OperatorPlan类属于org.apache.pig.impl.plan包,在下文中一共展示了OperatorPlan类的15个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: match

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
public Map<OperatorKey, OperatorKey> match(OperatorPlan<Operator> plan1,
                                           OperatorPlan<Operator> plan2,
                                           StringBuilder messages) {
    // Find plan1.OperatorSet - plan2.OperatorSet
    int diff1 = diffKeys(plan1, plan2, messages, "plan2") ;

    // Find plan2.OperatorSet - plan1.OperatorSet
    int diff2 = diffKeys(plan2, plan1, messages, "plan1") ;

    // If there is a problem, just finish here
    if ( (diff1 != 0) || (diff2 != 0) ) {
        return null ;
    }

    // if no problem, we just return exact matching
    Iterator<Operator> iter = plan1.getKeys().values().iterator() ;
    Map<OperatorKey, OperatorKey> outputMap
                            = new HashMap<OperatorKey, OperatorKey>() ;
    while(iter.hasNext()) {
        Operator op = iter.next() ;
        outputMap.put(op.getOperatorKey(), op.getOperatorKey()) ;
    }

    return outputMap;
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:26,代码来源:ExactKeyMatcher.java

示例2: match

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
public Map<OperatorKey, OperatorKey> match(OperatorPlan plan1,
                                           OperatorPlan plan2,
                                           StringBuilder messages) {

    List<OperatorKey> keyList1 = getSortedKeyList(plan1) ;
    List<OperatorKey> keyList2 = getSortedKeyList(plan2) ;

    // This matching logic only works when both plans have
    // the same number of operators
    if (keyList1.size() != keyList2.size()) {
        messages.append("Two plans have different size") ;
        return null ;
    }

    // Populating the output map
    Map<OperatorKey, OperatorKey> outputMap
                            = new HashMap<OperatorKey, OperatorKey>() ;
    for(int i=0; i< keyList1.size() ; i++) {
        outputMap.put(keyList1.get(i), keyList2.get(i)) ;
    }

    return outputMap ;
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:24,代码来源:IncreasingKeyMatcher.java

示例3: match

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
/**
 * Return true if the given plan has nodes that match the pattern 
 * represented by this class
 * If a match is found, the PatterNodes in the plan will return non 
 * null node for getMatch(). 
 * @param inpPlan - input plan to match
 * @return true if match is found
 */
public boolean match(OperatorPlan<? extends Operator<?>> inpPlan){
    reset();

    PatternPlan pattern = this;
    currentPlan = inpPlan;

    if(pattern.size() == 0){
        return true;
    }

    PatternNode ptNode = (PatternNode) pattern.getSinks().get(0);
    //try matching the pattern with the plan, starting with ptNode
    Iterator it = currentPlan.iterator();
    while(it.hasNext()){
        Operator<?> plOp = (Operator<?>) it.next();
        if(match(ptNode, plOp)){
            if(this.size() != ptNodesVisited.size()){
                //BUG
                throw new RuntimeException("invalid size of pattern nodes visited");
            }

            return true;
        }

    }
    return false;
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:36,代码来源:PatternPlan.java

示例4: showPlanOperators

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
private <T extends OperatorPlan<? extends Operator<?>>> 
void showPlanOperators(T p) {
    System.out.println("Operators:");

    ArrayList<Operator<?>> ops = new ArrayList<Operator<?>>(p.getKeys()
            .values());
    Collections.sort(ops);
    for (Operator<?> op : ops) {
        System.out.println("    op: " + op.name());
    }
    System.out.println();
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:13,代码来源:TestMultiQueryCompiler.java

示例5: diffKeys

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
/***
 * Report plan1.OperatorSet - plan2.OperatorSet
 *
 * @param plan1
 * @param plan2
 * @param messages where the report messages go. null if no messages needed
 * @param plan2Name the name that is used to refer to plan2 in messages
 * @return
 */

private int diffKeys(OperatorPlan<Operator> plan1,
                     OperatorPlan<Operator> plan2,
                     StringBuilder messages,
                     String plan2Name) {
    int count = 0 ;

    // prepare
    Map<OperatorKey, Operator> keyList = plan1.getKeys() ;
    Iterator<OperatorKey> iter = keyList.keySet().iterator() ;

    // go through the list of vertices of the first plan
    while(iter.hasNext()) {

        OperatorKey key = iter.next() ;

        // if the same key doesn't exist in the second plan
        // we've got a problem
        if (plan2.getOperator(key) == null) {
            Operator op1 = plan1.getOperator(key) ;
            if (messages != null) {
                messages.append(op1.getClass().getSimpleName()) ;
                appendOpKey(op1.getOperatorKey(), messages) ;
                messages.append(" doesn't exist") ;
                if (plan2Name != null) {
                    messages.append(" in ") ;
                    messages.append(plan2Name) ;
                    messages.append("\n") ;
                }
            }
            // increment diff counter
            count++ ;
        }
    }

    return count ;
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:47,代码来源:ExactKeyMatcher.java

示例6: getSortedKeyList

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
private List<OperatorKey> getSortedKeyList(OperatorPlan plan) {

        List<OperatorKey> keyList
                        = new ArrayList<OperatorKey>(plan.getKeys().keySet()) ;
        Collections.sort(keyList); 
        return keyList ;
    }
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:8,代码来源:IncreasingKeyMatcher.java

示例7: initialPlanNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
@Override
public void initialPlanNotification(String scriptId, OperatorPlan<?> plan) {
    synchronized (listeners) {
        for (PigProgressNotificationListener listener : listeners) {
            try {
                listener.initialPlanNotification(scriptId, plan);
            } catch (NoSuchMethodError e) {
                LOG.warn("PigProgressNotificationListener implementation doesn't "
                       + "implement initialPlanNotification(..) method: "
                       + listener.getClass().getName(), e);
            }
        }
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:15,代码来源:SyncProgressNotificationAdaptor.java

示例8: emitInitialPlanNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
public void emitInitialPlanNotification(OperatorPlan<?> plan) {
    for (PigProgressNotificationListener listener: listeners) {
        try {
            listener.initialPlanNotification(id, plan);
        } catch (NoSuchMethodError e) {
            LOG.warn("PigProgressNotificationListener implementation doesn't "
                   + "implement initialPlanNotification(..) method: "
                   + listener.getClass().getName(), e);
        }
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:12,代码来源:ScriptState.java

示例9: showPlanOperators

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
private <T extends OperatorPlan<? extends Operator<?>>>
void showPlanOperators(T p) {
    System.out.println("Operators:");

    ArrayList<Operator<?>> ops = new ArrayList<Operator<?>>(p.getKeys()
            .values());
    Collections.sort(ops);
    for (Operator<?> op : ops) {
        System.out.println("    op: " + op.name());
    }
    System.out.println();
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:13,代码来源:TestMultiQueryCompiler.java

示例10: dagLaunchNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
public void dagLaunchNotification(String dagName, OperatorPlan<?> dagPlan, int numVerticesToLaunch)  {
    for (PigTezProgressNotificationListener listener: tezListeners) {
        listener.dagLaunchNotification(id, dagName, dagPlan, numVerticesToLaunch);
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:6,代码来源:TezScriptState.java

示例11: initialPlanNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
@Override
public void initialPlanNotification(String id, OperatorPlan<?> plan) {
    System.out.println("id: " + id + " planNodes: " + plan.getKeys().size());
    assertNotNull(plan);
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:6,代码来源:TestPigRunner.java

示例12: merge

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
/**
 * The merge of a list of map plans
 * @param <O>
 * @param <E>
 * @param finPlan - Final Plan into which the list of plans is merged
 * @param plans - list of map plans to be merged
 * @throws PlanException
 */
private <O extends Operator, E extends OperatorPlan<O>> void merge(
        E finPlan, List<E> plans) throws PlanException {
    for (E e : plans) {
        finPlan.merge(e);
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:15,代码来源:MRCompiler.java

示例13: merge

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
/**
 * The merge of a list of plans into a single plan
 * @param <O>
 * @param <E>
 * @param finPlan - Final Plan into which the list of plans is merged
 * @param plans - list of plans to be merged
 * @throws PlanException
 */
private <O extends Operator<?>, E extends OperatorPlan<O>> void merge(
        E finPlan, List<E> plans) throws PlanException {
    for (E e : plans) {
        finPlan.merge(e);
    }
}
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:15,代码来源:TezCompiler.java

示例14: initialPlanNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
@Override
public void initialPlanNotification(String scriptId, OperatorPlan<?> plan) {

}
 
开发者ID:apache,项目名称:zeppelin,代码行数:5,代码来源:PigScriptListener.java

示例15: initialPlanNotification

import org.apache.pig.impl.plan.OperatorPlan; //导入依赖的package包/类
/**
 * Invoked before any Hadoop jobs (or a Tez DAG) are run with the plan that is to be executed.
 *
 * @param scriptId the unique id of the script
 * @param plan the OperatorPlan that is to be executed
 */
public void initialPlanNotification(String scriptId, OperatorPlan<?> plan);
 
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:8,代码来源:PigProgressNotificationListener.java


注:本文中的org.apache.pig.impl.plan.OperatorPlan类示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。