当前位置: 首页>>代码示例>>Java>>正文


Java TransactionAttempt.getTransactionId方法代码示例

本文整理汇总了Java中storm.trident.topology.TransactionAttempt.getTransactionId方法的典型用法代码示例。如果您正苦于以下问题:Java TransactionAttempt.getTransactionId方法的具体用法?Java TransactionAttempt.getTransactionId怎么用?Java TransactionAttempt.getTransactionId使用的例子?那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在storm.trident.topology.TransactionAttempt的用法示例。


在下文中一共展示了TransactionAttempt.getTransactionId方法的13个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: execute

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
    TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);

    if(tuple.getSourceStreamId().equals(MasterBatchCoordinator.SUCCESS_STREAM_ID)) {
        _state.cleanupBefore(attempt.getTransactionId());
        _coord.success(attempt.getTransactionId());
    } else if (tuple.getSourceStreamId().equals(MasterBatchCoordinator.COMMIT_STREAM_ID)) { 
    	// Do nothing.
    } else {
        long txid = attempt.getTransactionId();
        Object prevMeta = _state.getPreviousState(txid);
        Object meta = _coord.initializeTransaction(txid, prevMeta, _state.getState(txid));
        _state.overrideState(txid, meta);
        collector.emit(MasterBatchCoordinator.BATCH_STREAM_ID, new Values(attempt, meta));
    }
            
}
 
开发者ID:songtk,项目名称:learn_jstorm,代码行数:19,代码来源:TridentSpoutCoordinator.java

示例2: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * Emit a batch for the specified transaction attempt and metadata for the transaction. The metadata
    * was created by the Coordinator in the initializeTranaction method. This method must always emit
    * the same batch of tuples across all tasks for the same transaction id.
    * 
    * emitBatch will keep getting called repeatedly. Each invocation to emitBatch will be done in a new trxn.
    * emitBatch is internally invoked by Storm repeatedly, like a infinite while loop.Each iteration of the
    * loop is a trxn.Trxn Ids are auto-generated.
    * 
    */

@Override
public void emitBatch(TransactionAttempt trxnAttempt, Object coordinatorMeta,
		TridentCollector collector) {
	// TODO Auto-generated method stub
	String curThreadName = Thread.currentThread().getName();
	long trxnId = trxnAttempt.getTransactionId();
	for (int i = 0; i < BATCH_SIZE; i++) {
          List<Object> events = new ArrayList<Object>();
          
          String uid = UUID.randomUUID().toString();
          int field1 = rand.nextInt(1);//possible values - 0
          int field2 = rand.nextInt(1);//possible values - 0
          
          TupleDS tupleDs = new TupleDS(uid, uid, field1, field2);
          
          events.add(tupleDs);
          collector.emit(events);
  		  CommonUtil.logMessage(logger, curThreadName, "emitBatch: emitting tuple for trxnId = %d with uid = %s, field1 = %d, field2 = %d",  trxnId, uid, field1, field2);

	}
	
}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:34,代码来源:SpoutEmitter.java

示例3: execute

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
    TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);

    if (tuple.getSourceStreamId().equals(MasterBatchCoordinator.SUCCESS_STREAM_ID)) {
        _state.cleanupBefore(attempt.getTransactionId());
        _coord.success(attempt.getTransactionId());
    } else {
        long txid = attempt.getTransactionId();
        Object prevMeta = _state.getPreviousState(txid);
        Object meta = _coord.initializeTransaction(txid, prevMeta, _state.getState(txid));
        _state.overrideState(txid, meta);
        collector.emit(MasterBatchCoordinator.BATCH_STREAM_ID, new Values(attempt, meta));
    }

}
 
开发者ID:kkllwww007,项目名称:jstrom,代码行数:17,代码来源:TridentSpoutCoordinator.java

示例4: success

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * This attempt committed successfully, so all state for this commit and before can be safely cleaned up.
    * In case of batch replay scenario in Opaque Transactional State Management, this method will get invoked
    * (irrespective of thrown FailedException for a trxnId). This is because as per design of Opaque 
    * Transactional State Management, the same data isn't tied to a partuclar trxnId.
    * The same failed data can be transmitted via another trxnId.
    */

@Override
public void success(TransactionAttempt trxnAttempt) {
	// TODO Auto-generated method stub
	String curThreadName = Thread.currentThread().getName();
	try {
			throw new Exception("success: throwing intentional Exception - check code flow");
	} catch (Exception e) {
		// TODO: handle exception
		e.printStackTrace();
	}
	long trxnId = trxnAttempt.getTransactionId();
	
	CommonUtil.logMessage(logger, curThreadName, "success: entered for trxnId = %d", trxnId);
	//successfulTransactions.incrementAndGet();
	//Util.logMessage(logger, curThreadName, "success: exiting with updated succesfulTrxns = %d", successfulTransactions.get());

}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:26,代码来源:RandomPhraseEmitter.java

示例5: commit

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void commit(TransactionAttempt attempt) {
    // this code here handles a case where a previous commit failed, and the partitions
    // changed since the last commit. This clears out any state for the removed partitions
    // for this txid.
    // we make sure only a single task ever does this. we're also guaranteed that
    // it's impossible for there to be another writer to the directory for that partition
    // because only a single commit can be happening at once. this is because in order for 
    // another attempt of the batch to commit, the batch phase must have succeeded in between.
    // hence, all tasks for the prior commit must have finished committing (whether successfully or not)
    if(_changedMeta && _index==0) {
        Set<String> validIds = new HashSet<String>();
        for(ISpoutPartition p: (List<ISpoutPartition>) _emitter.getOrderedPartitions(_savedCoordinatorMeta)) {
            validIds.add(p.getId());
        }
        for(String existingPartition: _state.list("")) {
            if(!validIds.contains(existingPartition)) {
                RotatingTransactionalState s = new RotatingTransactionalState(_state, existingPartition);
                s.removeState(attempt.getTransactionId());
            }
        }
        _changedMeta = false;
    }            
    
    Long txid = attempt.getTransactionId();
    Map<String, Object> metas = _cachedMetas.remove(txid);
    for(String partitionId: metas.keySet()) {
        Object meta = metas.get(partitionId);
        _partitionStates.get(partitionId).rotatingState.overrideState(txid, meta);
    }
}
 
开发者ID:greeenSY,项目名称:Tstream,代码行数:32,代码来源:OpaquePartitionedTridentSpoutExecutor.java

示例6: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void emitBatch(TransactionAttempt tx, Object coordinatorMeta, TridentCollector collector) {
    long txid = tx.getTransactionId();

    long now = System.currentTimeMillis();
    if (now - lastRotate > rotateTime) {
        Map<Long, List<Object>> failed = idsMap.rotate();
        for (Long id : failed.keySet()) {
            // TODO: this isn't right... it's not in the map anymore
            fail(id);
        }
        lastRotate = now;
    }

    if (idsMap.containsKey(txid)) {
        fail(txid);
    }

    _collector.reset(collector);
    if (!prepared) {
        _spout.open(_conf, _context, new SpoutOutputCollector(_collector));
        prepared = true;
    }
    for (int i = 0; i < _maxBatchSize; i++) {
        _spout.nextTuple();
        if (_collector.numEmitted < i) {
            break;
        }
    }
    idsMap.put(txid, _collector.ids);

}
 
开发者ID:kkllwww007,项目名称:jstrom,代码行数:33,代码来源:RichSpoutBatchExecutor.java

示例7: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void emitBatch(TransactionAttempt tx, Object coordinatorMeta, TridentCollector collector) {
    long txid = tx.getTransactionId();
    
    long now = System.currentTimeMillis();
    if(now - lastRotate > rotateTime) {
        Map<Long, List<Object>> failed = idsMap.rotate();
        for(Long id: failed.keySet()) {
            //TODO: this isn't right... it's not in the map anymore
            fail(id);
        }
        lastRotate = now;
    }
    
    if(idsMap.containsKey(txid)) {
        fail(txid);
    }
    
    _collector.reset(collector);
    if(!prepared) {
        _spout.open(_conf, _context, new SpoutOutputCollector(_collector));
        prepared = true;
    }
    for(int i=0; i<_maxBatchSize; i++) {
        _spout.nextTuple();
        if(_collector.numEmitted < i) {
            break;
        }
    }
    idsMap.put(txid, _collector.ids);

}
 
开发者ID:songtk,项目名称:learn_jstorm,代码行数:33,代码来源:RichSpoutBatchExecutor.java

示例8: success

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * This attempt committed successfully, so all state for this commit and before can be safely cleaned up.
    */

@Override
public void success(TransactionAttempt trxnAttempt) {
	// TODO Auto-generated method stub
	long trxnId = trxnAttempt.getTransactionId();
	String curThreadName = Thread.currentThread().getName();
	CommonUtil.logMessage(logger, curThreadName, "success: entered for trxnId = %d", trxnId);
	//successfulTransactions.incrementAndGet();
	//Util.logMessage(logger, curThreadName, "success: exiting with updated succesfulTrxns = %d", successfulTransactions.get());

}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:15,代码来源:SpoutEmitter.java

示例9: commit

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
@Override
public void commit(TransactionAttempt attempt) {
    // this code here handles a case where a previous commit failed, and the partitions
    // changed since the last commit. This clears out any state for the removed partitions
    // for this txid.
    // we make sure only a single task ever does this. we're also guaranteed that
    // it's impossible for there to be another writer to the directory for that partition
    // because only a single commit can be happening at once. this is because in order for
    // another attempt of the batch to commit, the batch phase must have succeeded in between.
    // hence, all tasks for the prior commit must have finished committing (whether successfully or not)
    if (_changedMeta && _index == 0) {
        Set<String> validIds = new HashSet<String>();
        for (ISpoutPartition p : (List<ISpoutPartition>) _emitter.getOrderedPartitions(_savedCoordinatorMeta)) {
            validIds.add(p.getId());
        }
        for (String existingPartition : _state.list("")) {
            if (!validIds.contains(existingPartition)) {
                RotatingTransactionalState s = new RotatingTransactionalState(_state, existingPartition);
                s.removeState(attempt.getTransactionId());
            }
        }
        _changedMeta = false;
    }

    Long txid = attempt.getTransactionId();
    Map<String, Object> metas = _cachedMetas.remove(txid);
    for (String partitionId : metas.keySet()) {
        Object meta = metas.get(partitionId);
        _partitionStates.get(partitionId).rotatingState.overrideState(txid, meta);
    }
}
 
开发者ID:kkllwww007,项目名称:jstrom,代码行数:32,代码来源:OpaquePartitionedTridentSpoutExecutor.java

示例10: success

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * This attempt committed successfully, so all state for this commit and before can be safely cleaned up.
    * In case of Transactional State Management, 
    * success will NOT be called when a batch is explicitly failed - by throwing FailedException.
    */

@Override
public void success(TransactionAttempt trxnAttempt) {
	// TODO Auto-generated method stub
	/**
	 * 	at bharati.binita.storm.trident.eg6.RandomPhraseEmitter.success(RandomPhraseEmitter.java:150)
		at storm.trident.spout.TridentSpoutExecutor.execute(TridentSpoutExecutor.java:79)
		at storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:333)
		at backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631)
		at backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399)
		at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
		at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
		at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
		at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
	 */
	try {
		throw new Exception("success: throwing intentional Exception - check code flow");
	} catch (Exception e) {
		// TODO: handle exception
		e.printStackTrace();
	}
	long trxnId = trxnAttempt.getTransactionId();
	String curThreadName = Thread.currentThread().getName();
	CommonUtil.logMessage(logger, curThreadName, "success: entered for trxnId = %d", trxnId);
	//successfulTransactions.incrementAndGet();
	//Util.logMessage(logger, curThreadName, "success: exiting with updated succesfulTrxns = %d", successfulTransactions.get());

}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:34,代码来源:RandomPhraseEmitter.java

示例11: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * Emit a batch for the specified transaction attempt and metadata for the transaction. The metadata
    * was created by the Coordinator in the initializeTranaction method. This method must always emit
    * the same batch of tuples across all tasks for the same transaction id - Transactional Spout feature..
    * 
    * emitBatch will keep getting called repeatedly. Each invocation to emitBatch will be done in a new trxn.
    * emitBatch is internally invoked by Storm repeatedly, like a infinite while loop.Each iteration of the
    * loop is a trxn.Trxn Ids are auto-generated.
    * 
    * In case of batch replay, same failed phrase will keep getting emitted over and over for the same trxnId.
    * 
    */

@Override
public void emitBatch(TransactionAttempt trxnAttempt, String coordinatorMeta,
		TridentCollector collector) {
	// TODO Auto-generated method stub
	
	String curThreadName = Thread.currentThread().getName();
	/* During batch replay scenario, failed trxId will be repeated.
	 * i.e you will see the same trxId more than once. */
	long trxnId = trxnAttempt.getTransactionId();
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: entered with latestIndex = %d",  latestIndex);


	CommonUtil.logMessage(logger, curThreadName, "emitBatch: entered with trxnId = %d",  trxnId);
			
	
	/**
	 * Store the trxId mapping to data. Used for replay during batch (trxn) failure scenario.  - start
	 */
	List<String> redisData = redisOperations.lrange(trxnId+"");
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: redisData = %s",  redisData);
	
	String randomPhrase = null;
	
	if(redisData != null && redisData.size() > 0)
	{
		//Implies this trxn is replayed!!
		CommonUtil.logMessage(logger, curThreadName, "emitBatch: trxn = %s is replayed !!",  trxnId);
		randomPhrase = redisData.get(0);
		
	}
	/**
	 *  2014-11-28 21:07:16 b.s.util [ERROR] Async loop died!
              java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 498
           2014-11-28 21:07:16 b.s.util [INFO] Halting process: ("Worker died")
           014-11-28 21:13:43 b.s.d.worker [INFO] Launching worker for   trident-eg5-2-1417188174 on 223733d0-502e-4bfd-a074-ca39e7d5c130:6703 with  id e4c3658f-881b-493b-8304-65bc01ddf18a and conf {"dev.zookeeper.path"  "/tmp/dev-storm-zookeeper".......................

	 */
	else if(latestIndex < 498)//To stop java.lang.ArrayIndexOutOfBoundsException , and to prevent worker from dying and restarting.
	{
		CommonUtil.logMessage(logger, curThreadName, "emitBatch: trxn = %s is a fresh one.",  trxnId);
		
		randomPhrase = sentences[latestIndex];
		
		latestIndex++;
		redisOperations.rpush(trxnId+"", Util.filterPhrase(randomPhrase));
		
	}
	/**
	 * Store the trxId mapping to data. Used for replay during batch (trxn) failure scenario.  - end
	 */
	
    
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: randomPhrase = %s",  randomPhrase);
			
	List<Object> randomPhraseList = new ArrayList<>();
	randomPhraseList.add(randomPhrase);
	
	collector.emit(randomPhraseList);
		
}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:74,代码来源:RandomPhraseEmitter.java

示例12: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * Emit a batch for the specified transaction attempt and metadata for the transaction. The metadata
    * was created by the Coordinator in the initializeTranaction method. Unlike transactional spout,
    * an opaque transactional spout cannot guarantee that the batch of tuples for a txid remains constant..
    * 
    * emitBatch will keep getting called repeatedly. Each invocation to emitBatch will be done in a new trxn.
    * emitBatch is internally invoked by Storm repeatedly, like a infinite while loop.Each iteration of the
    * loop is a trxn.Trxn Ids are auto-generated.
    * 
    */

@Override
public void emitBatch(TransactionAttempt trxnAttempt, String coordinatorMeta,
		TridentCollector collector) {
	// TODO Auto-generated method stub
	
	String curThreadName = Thread.currentThread().getName();
	/* During batch replay scenario, failed trxId will be repeated. i.e you will see the same trxId more than once.
	 * But, success method for the failed trxn gets invoked immediately, so , basically, you can NOT use the same trxn id
	 * more than once in Opaque Transactional Spout. If you want to replay older data, use a new trxn id.
	 * 
	 * Consequently, and also in contrast to Transactional Spouts, Opaque Transactional Spouts do not require every batch data to be persisted (in order to replay).
	 * Only the failed batch data should be persisted, unlike Transactional Spouts.In Transactional Spouts, even the successful trxns have to be persisted beforehand;
	 * for replay in case of subsequent failure during further processing.
	 *
	 */
	long trxnId = trxnAttempt.getTransactionId();
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: entered with latestIndex = %d",  latestIndex);


	CommonUtil.logMessage(logger, curThreadName, "emitBatch: entered with trxnId = %d",  trxnId);
	
	//Check if any previous trxn has failed.If yes, replay the failed trxn's data first. Check RedisStoreIBackingMap:multiPut.
	List<String> prevFailedData = redisOperations.lrange("replayPhrase", 0, 1);
		CommonUtil.logMessage(logger, curThreadName, "emitBatch: prevFailedData = %s",  prevFailedData);
		
	String randomPhrase = null;
		
	if(prevFailedData != null && prevFailedData.size() > 0)//Implies a earlier trxn had failed, need to replay that data.
	{
		randomPhrase = prevFailedData.get(0);
		//Now, remove the failedData from Redis.
		redisOperations.lpop("replayPhrase", 1);
	}
	//No previously failed data
	/**
	 *  2014-11-28 21:07:16 b.s.util [ERROR] Async loop died!
              java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 498
           2014-11-28 21:07:16 b.s.util [INFO] Halting process: ("Worker died")
           014-11-28 21:13:43 b.s.d.worker [INFO] Launching worker for   trident-eg5-2-1417188174 on 223733d0-502e-4bfd-a074-ca39e7d5c130:6703 with  id e4c3658f-881b-493b-8304-65bc01ddf18a and conf {"dev.zookeeper.path"  "/tmp/dev-storm-zookeeper".......................

	 */
	else if(latestIndex < 498)//To stop java.lang.ArrayIndexOutOfBoundsException , and to prevent worker from dying and restarting.
	{
		CommonUtil.logMessage(logger, curThreadName, "emitBatch: trxn = %s is a fresh one.",  trxnId);
		
		randomPhrase = sentences[latestIndex];
		
		latestIndex++;
		
		
	}
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: randomPhrase = %s, trxnId = %s", randomPhrase, trxnId);
	
	List<Object> randomPhraseList = new ArrayList<>();
	randomPhraseList.add(randomPhrase);
	
	collector.emit(randomPhraseList);

		
}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:72,代码来源:RandomPhraseEmitter.java

示例13: emitBatch

import storm.trident.topology.TransactionAttempt; //导入方法依赖的package包/类
/**
    * Emit a batch for the specified transaction attempt and metadata for the transaction. The metadata
    * was created by the Coordinator in the initializeTranaction method. This method must always emit
    * the same batch of tuples across all tasks for the same transaction id.
    * 
    * emitBatch will keep getting called repeatedly. Each invocation to emitBatch will be done in a new trxn.
    * emitBatch is internally invoked by Storm repeatedly, like a infinite while loop.Each iteration of the
    * loop is a trxn.Trxn Ids are auto-generated.
    * 
    */

@Override
public void emitBatch(TransactionAttempt trxnAttempt, Object coordinatorMeta,
		TridentCollector collector) {
	// TODO Auto-generated method stub
	
	String curThreadName = Thread.currentThread().getName();
	long trxnId = trxnAttempt.getTransactionId();


	CommonUtil.logMessage(logger, curThreadName, "emitBatch: entered with trxnId = %d",  trxnId);
			
	String randomPhrase = sentences[rand.nextInt(sentences.length)];
	CommonUtil.logMessage(logger, curThreadName, "emitBatch: randomPhrase = %s",  randomPhrase);
	

	StringTokenizer st = new StringTokenizer(randomPhrase);
	int count = 0;
	
	while(st.hasMoreTokens() && count < MAX_BATCH_SIZE)//content of a batch
	{
		List<Object> randomWordList = new ArrayList<Object>();//list size is always 1; restriction of collector.emit api itself; arg has to be a List.
		String randomWord = st.nextToken();
		randomWordList.add(randomWord);
		
		CommonUtil.logMessage(logger, curThreadName, "emitBatch: trxnId = %d, word = %s",  trxnId, randomWord);
		collector.emit(randomWordList);
		
		count++;
	}
	
	
	
}
 
开发者ID:BinitaBharati,项目名称:storm-trident-example,代码行数:45,代码来源:RandomWordEmitter.java


注:本文中的storm.trident.topology.TransactionAttempt.getTransactionId方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。