當前位置: 首頁>>代碼示例>>Java>>正文


Java Serializer類代碼示例

本文整理匯總了Java中org.apache.hadoop.io.serializer.Serializer的典型用法代碼示例。如果您正苦於以下問題:Java Serializer類的具體用法?Java Serializer怎麽用?Java Serializer使用的例子?那麽, 這裏精選的類代碼示例或許可以為您提供幫助。


Serializer類屬於org.apache.hadoop.io.serializer包,在下文中一共展示了Serializer類的10個代碼示例,這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚,您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: copy

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
/**
 * Make a copy of the writable object using serialization to a buffer
 * @param src the object to copy from
 * @param dst the object to copy into, which is destroyed
 * @return dst param (the copy)
 * @throws IOException
 */
@SuppressWarnings("unchecked")
public static <T> T copy(Configuration conf, 
                              T src, T dst) throws IOException {
  CopyInCopyOutBuffer buffer = CLONE_BUFFERS.get();
  buffer.outBuffer.reset();
  SerializationFactory factory = getFactory(conf);
  Class<T> cls = (Class<T>) src.getClass();
  Serializer<T> serializer = factory.getSerializer(cls);
  serializer.open(buffer.outBuffer);
  serializer.serialize(src);
  buffer.moveData();
  Deserializer<T> deserializer = factory.getDeserializer(cls);
  deserializer.open(buffer.inBuffer);
  dst = deserializer.deserialize(dst);
  return dst;
}
 
開發者ID:nucypher,項目名稱:hadoop-oss,代碼行數:24,代碼來源:ReflectionUtils.java

示例2: copy

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
/**
 * Make a copy of the writable object using serialization to a buffer
 * @param dst the object to copy from
 * @param src the object to copy into, which is destroyed
 * @throws IOException
 */
@SuppressWarnings("unchecked")
public static <T> T copy(Configuration conf, 
                              T src, T dst) throws IOException {
  CopyInCopyOutBuffer buffer = cloneBuffers.get();
  buffer.outBuffer.reset();
  SerializationFactory factory = getFactory(conf);
  Class<T> cls = (Class<T>) src.getClass();
  Serializer<T> serializer = factory.getSerializer(cls);
  serializer.open(buffer.outBuffer);
  serializer.serialize(src);
  buffer.moveData();
  Deserializer<T> deserializer = factory.getDeserializer(cls);
  deserializer.open(buffer.inBuffer);
  dst = deserializer.deserialize(dst);
  return dst;
}
 
開發者ID:spafka,項目名稱:spark_deep,代碼行數:23,代碼來源:ReflectionUtils.java

示例3: KeyValueWriter

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
public KeyValueWriter(Configuration conf, OutputStream output,
                      Class<K> kyClass, Class<V> valClass
                     ) throws IOException {
  keyClass = kyClass;
  valueClass = valClass;
  dataBuffer = new DataOutputBuffer();
  SerializationFactory serializationFactory
                                         = new SerializationFactory(conf);
  keySerializer
              = (Serializer<K>)serializationFactory.getSerializer(keyClass);
  keySerializer.open(dataBuffer);
  valueSerializer
            = (Serializer<V>)serializationFactory.getSerializer(valueClass);
  valueSerializer.open(dataBuffer);
  outputStream = new DataOutputStream(output);
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:17,代碼來源:TestMerge.java

示例4: serDeser

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
private <K> K serDeser(K conf) throws Exception {
  SerializationFactory factory = new SerializationFactory(CONF);
  Serializer<K> serializer =
    factory.getSerializer(GenericsUtil.getClass(conf));
  Deserializer<K> deserializer =
    factory.getDeserializer(GenericsUtil.getClass(conf));

  DataOutputBuffer out = new DataOutputBuffer();
  serializer.open(out);
  serializer.serialize(conf);
  serializer.close();

  DataInputBuffer in = new DataInputBuffer();
  in.reset(out.getData(), out.getLength());
  deserializer.open(in);
  K after = deserializer.deserialize(null);
  deserializer.close();
  return after;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:20,代碼來源:TestWritableJobConf.java

示例5: makeCopyForPassByValue

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
private <E> E makeCopyForPassByValue(Serialization<E> serialization,
                                      E obj) throws IOException {
  Serializer<E> ser =
    serialization.getSerializer(GenericsUtil.getClass(obj));
  Deserializer<E> deser =
    serialization.getDeserializer(GenericsUtil.getClass(obj));

  DataOutputBuffer dof = threadLocalDataOutputBuffer.get();

  dof.reset();
  ser.open(dof);
  ser.serialize(obj);
  ser.close();
  obj = ReflectionUtils.newInstance(GenericsUtil.getClass(obj),
                                    getChainJobConf());
  ByteArrayInputStream bais =
    new ByteArrayInputStream(dof.getData(), 0, dof.getLength());
  deser.open(bais);
  deser.deserialize(obj);
  deser.close();
  return obj;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:23,代碼來源:Chain.java

示例6: writeFirstKeyValueBytes

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
/**
 * This method is called to write the record that was most recently
 * served (before a call to the mark). Since the framework reads one
 * record in advance, to get this record, we serialize the current key
 * and value
 * @param out
 * @throws IOException
 */
private void writeFirstKeyValueBytes(DataOutputStream out) 
throws IOException {
  assert (getCurrentKey() != null && getCurrentValue() != null);
  WritableUtils.writeVInt(out, currentKeyLength);
  WritableUtils.writeVInt(out, currentValueLength);
  Serializer<KEYIN> keySerializer = 
    serializationFactory.getSerializer(keyClass);
  keySerializer.open(out);
  keySerializer.serialize(getCurrentKey());

  Serializer<VALUEIN> valueSerializer = 
    serializationFactory.getSerializer(valueClass);
  valueSerializer.open(out);
  valueSerializer.serialize(getCurrentValue());
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:24,代碼來源:ReduceContextImpl.java

示例7: copy

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
/**
 * Make a copy of the writable object using serialization to a buffer
 * @param src the object to copy from
 * @param dst the object to copy into, which is destroyed
 * @return dst param (the copy)
 * @throws IOException
 */
@SuppressWarnings("unchecked")
public static <T> T copy(Configuration conf, 
                              T src, T dst) throws IOException {
  CopyInCopyOutBuffer buffer = cloneBuffers.get();
  buffer.outBuffer.reset();
  SerializationFactory factory = getFactory(conf);
  Class<T> cls = (Class<T>) src.getClass();
  Serializer<T> serializer = factory.getSerializer(cls);
  serializer.open(buffer.outBuffer);
  serializer.serialize(src);
  buffer.moveData();
  Deserializer<T> deserializer = factory.getDeserializer(cls);
  deserializer.open(buffer.inBuffer);
  dst = deserializer.deserialize(dst);
  return dst;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:24,代碼來源:ReflectionUtils.java

示例8: configureSerializer

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
@SuppressWarnings({ "rawtypes", "unchecked" })
void configureSerializer(String confKey, Configuration conf) {
  Class clientInputSerializerClass = conf.getClass(confKey, null);
  if (clientInputSerializerClass != null) {
    LOG.info("Using custom serializer: " + clientInputSerializerClass.getName());
    clientInputSerializer = 
        (Serializer) ReflectionUtils.newInstance(clientInputSerializerClass, conf);

    try {
      clientInputSerializer.open(clientOut_);
    } catch (IOException e) {
      LOG.error("Could not open serializer", e);
      throw new RuntimeException(e);
    }
  } else {
    LOG.info("Not using a custom serializer");
  }
}
 
開發者ID:rhli,項目名稱:hadoop-EAR,代碼行數:19,代碼來源:PipeMapRed.java

示例9: cloneObj

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
private <T> T cloneObj(T t) throws IOException
{
  Serializer<T> keySerializer;
  Class<T> keyClass;
  PipedInputStream pis = new PipedInputStream();
  PipedOutputStream pos = new PipedOutputStream(pis);
  keyClass = (Class<T>)t.getClass();
  keySerializer = serializationFactory.getSerializer(keyClass);
  keySerializer.open(pos);
  keySerializer.serialize(t);
  Deserializer<T> keyDesiralizer = serializationFactory.getDeserializer(keyClass);
  keyDesiralizer.open(pis);
  T clonedArg0 = keyDesiralizer.deserialize(null);
  pos.close();
  pis.close();
  keySerializer.close();
  keyDesiralizer.close();
  return clonedArg0;

}
 
開發者ID:apache,項目名稱:apex-malhar,代碼行數:21,代碼來源:OutputCollectorImpl.java

示例10: testSerializer

import org.apache.hadoop.io.serializer.Serializer; //導入依賴的package包/類
@Test
public void testSerializer() throws Exception {
   WebPage originalWebPage = new WebPage(new URL("http://www.jboss.org"), "opensource", 10L);

   ByteArrayOutputStream baos = new ByteArrayOutputStream();

   JBossMarshallerSerialization<WebPage> marshallerSerialization = new JBossMarshallerSerialization<>();
   Serializer<WebPage> serializer = marshallerSerialization.getSerializer(WebPage.class);
   serializer.open(baos);
   serializer.serialize(originalWebPage);
   serializer.close();

   Deserializer<WebPage> deserializer = marshallerSerialization.getDeserializer(WebPage.class);
   deserializer.open(new ByteArrayInputStream(baos.toByteArray()));
   WebPage deserializedWebPage = deserializer.deserialize(null);
   deserializer.close();

   assertEquals(deserializedWebPage, originalWebPage);
}
 
開發者ID:infinispan,項目名稱:infinispan-hadoop,代碼行數:20,代碼來源:SerializerTest.java


注:本文中的org.apache.hadoop.io.serializer.Serializer類示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台,相關代碼片段篩選自各路編程大神貢獻的開源項目,源碼版權歸原作者所有,傳播和使用請參考對應項目的License;未經允許,請勿轉載。