當前位置: 首頁>>代碼示例>>Java>>正文

Java TableTunnel.setEndpoint方法代碼示例

本文整理匯總了Java中com.aliyun.odps.tunnel.TableTunnel.setEndpoint方法的典型用法代碼示例。如果您正苦於以下問題：Java TableTunnel.setEndpoint方法的具體用法？Java TableTunnel.setEndpoint怎麽用？Java TableTunnel.setEndpoint使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類com.aliyun.odps.tunnel.TableTunnel的用法示例。


在下文中一共展示了TableTunnel.setEndpoint方法的6個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: MaxComputeDataTransferUDTF

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
public MaxComputeDataTransferUDTF() throws Exception {
    odpsConfig.init("odps.conf");

    Account account = new AliyunAccount(odpsConfig.getAccessId(), odpsConfig.getAccessKey());
    odps = new Odps(account);
    odps.setEndpoint(odpsConfig.getOdpsEndPoint());
    odps.setDefaultProject(odpsConfig.getProjectName());
    tunnel = new TableTunnel(odps);
    tunnel.setEndpoint(odpsConfig.getTunnelEndPoint());
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:11，代碼來源:MaxComputeDataTransferUDTF.java

示例2: MaxComputeDataTransferUDTFMultiPart

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
public MaxComputeDataTransferUDTFMultiPart() throws Exception {
    odpsConfig.init("odps.conf");

    Account account = new AliyunAccount(odpsConfig.getAccessId(), odpsConfig.getAccessKey());
    odps = new Odps(account);
    odps.setEndpoint(odpsConfig.getOdpsEndPoint());
    odps.setDefaultProject(odpsConfig.getProjectName());
    tunnel = new TableTunnel(odps);
    tunnel.setEndpoint(odpsConfig.getTunnelEndPoint());
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:11，代碼來源:MaxComputeDataTransferUDTFMultiPart.java

示例3: buildTunnelWriter

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
private OdpsWriter buildTunnelWriter(String project, String tableName,
                                     String tunnelEndPoint, int retryCount, String sessionId) {
  TableTunnel tunnel = new TableTunnel(odps);
  if (StringUtils.isNotEmpty(tunnelEndPoint)) {
    tunnel.setEndpoint(tunnelEndPoint);
  }
  return new OdpsTunnelWriter(tunnel, project, tableName, retryCount, sessionId, useCompress);
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:9，代碼來源:OdpsUploadProcessor.java

示例4: init

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
public boolean init(StepMetaInterface smi, StepDataInterface sdi) {
    if (super.init(smi, sdi)) {
        meta = (OdpsInputMeta) smi;
        data = (OdpsInputData) sdi;

        Account account = new AliyunAccount(environmentSubstitute(meta.getAccessId()),
            environmentSubstitute(meta.getAccessKey()));
        Odps odps = new Odps(account);
        odps.setEndpoint(environmentSubstitute(meta.getEndpoint()));
        odps.setDefaultProject(environmentSubstitute(meta.getProjectName()));
        odps.setUserAgent("Maxcompute-Kettle-Plugin-2.0.0");

        TableTunnel tableTunnel = new TableTunnel(odps);
        String tunnelEndpoint = environmentSubstitute(meta.getTunnelEndpoint());
        if (!StringUtils.isEmpty(tunnelEndpoint)) {
            tableTunnel.setEndpoint(tunnelEndpoint);
        }
        DownloadSession downloadSession = null;
        try {
            if (meta.getPartition() != null && !meta.getPartition().trim().equals("")) {
                PartitionSpec partitionSpec =
                    new PartitionSpec(environmentSubstitute(meta.getPartition()));
                downloadSession = tableTunnel
                    .createDownloadSession(environmentSubstitute(meta.getProjectName()),
                        environmentSubstitute(meta.getTableName()), partitionSpec);
            } else {
                downloadSession = tableTunnel
                    .createDownloadSession(environmentSubstitute(meta.getProjectName()),
                        environmentSubstitute(meta.getTableName()));
            }

            schema = downloadSession.getSchema();
            initOdpsFieldPosMap(schema);

            long count = downloadSession.getRecordCount();
            logBasic("count is: " + count);
            data.tunnelRecordReader = downloadSession.openRecordReader(0L, count);
            return true;
        } catch (TunnelException e) {
            logError(e.getMessage(), e);
        } catch (Exception ex) {
            logError(ex.getMessage(), ex);
        }
    }
    return false;
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:47，代碼來源:OdpsInput.java

示例5: init

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
public boolean init(StepMetaInterface smi, StepDataInterface sdi) {
    if (super.init(smi, sdi)) {
        meta = (OdpsOutputMeta) smi;
        data = (OdpsOutputData) sdi;

        Account account = new AliyunAccount(environmentSubstitute(meta.getAccessId()),
            environmentSubstitute(meta.getAccessKey()));
        Odps odps = new Odps(account);
        odps.setEndpoint(environmentSubstitute(meta.getEndpoint()));
        odps.setDefaultProject(environmentSubstitute(meta.getProjectName()));
        odps.setUserAgent("Maxcompute-Kettle-Plugin-2.0.0");

        TableTunnel tableTunnel = new TableTunnel(odps);
        String tunnelEndpoint = environmentSubstitute(meta.getTunnelEndpoint());
        if (!StringUtils.isEmpty(tunnelEndpoint)) {
            tableTunnel.setEndpoint(tunnelEndpoint);
        }
        try {
            MaxcomputeUtil.dealTruncate(odps,
                odps.tables().get(environmentSubstitute(meta.getTableName())),
                environmentSubstitute(meta.getPartition()), meta.isTruncate());

            if (meta.getPartition() != null && !meta.getPartition().trim().equals("")) {
                PartitionSpec partitionSpec =
                    new PartitionSpec(environmentSubstitute(meta.getPartition()));
                data.uploadSession = tableTunnel
                    .createUploadSession(environmentSubstitute(meta.getProjectName()),
                        environmentSubstitute(meta.getTableName()), partitionSpec);
            } else {
                data.uploadSession = tableTunnel
                    .createUploadSession(environmentSubstitute(meta.getProjectName()),
                        environmentSubstitute(meta.getTableName()));
            }

            schema = data.uploadSession.getSchema();
            initOdpsFieldPosMap(schema);
            initOdpsColumn2StreamFieldMap();

            data.recordWriter = data.uploadSession.openBufferedWriter();

            return true;
        } catch (TunnelException e) {
            logError(e.getMessage(), e);
        } catch (Exception ex) {
            logError(ex.getMessage(), ex);
        }
    }
    return false;
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:50，代碼來源:OdpsOutput.java

示例6: setConf

import com.aliyun.odps.tunnel.TableTunnel; //導入方法依賴的package包/類
@Override
public void setConf(Configuration configuration) {
  this.conf = configuration;
  rowDOList = new LinkedList<OdpsRowDO>();

  inputDateFormat = conf.get(OdpsConstants.DATE_FORMAT);
  retryCount = conf.getInt(OdpsConstants.RETRY_COUNT,
          OdpsConstants.DEFAULT_RETRY_COUNT);
  batchSize = conf.getInt(OdpsConstants.BATCH_SIZE,
          OdpsConstants.DEFAULT_BATCH_SIZE);
  useCompress = conf.getBoolean(OdpsConstants.USE_COMPRESS_IN_UPLOAD, false);

  String project = conf.get(OdpsConstants.PROJECT);
  String endpoint = conf.get(OdpsConstants.ENDPOINT);
  String tableName = conf.get(OdpsConstants.TABLE_NAME);
  String tunnelEndPoint = OdpsUtil.getTunnelEndPoint(conf.get(OdpsConstants.TUNNEL_ENDPOINT));

  odps = new Odps(new AliyunAccount(conf.get(OdpsConstants.ACCESS_ID),
          conf.get(OdpsConstants.ACCESS_KEY)));
  odps.setUserAgent(OdpsUtil.getUserAgent());
  odpsTable = buildOdpsTable(odps, project, endpoint, tableName);

  partitionKeys = strToArray(conf.get(OdpsConstants.PARTITION_KEY));
  partitionValues = strToArray(conf.get(OdpsConstants.PARTITION_VALUE));
  if (partitionKeys != null) {
    partitionMap = buildPartitionMap();
  }

  List<String> inputColumnNames = Arrays.asList(
          conf.getStrings(OdpsConstants.INPUT_COL_NAMES));
  odpsRecordBuilder = new OdpsRecordBuilder(odpsTable,
          inputDateFormat, inputColumnNames);
  try {
    if (conf.getBoolean(OdpsConstants.ODPS_DISABLE_DYNAMIC_PARTITIONS, false)) {
      String partition = getPartitionSpec(partitionKeys, partitionValues, Maps.newHashMap());
      TableTunnel.UploadSession uploadSession = null;
      TableTunnel tunnel = new TableTunnel(odps);
      if (StringUtils.isNotEmpty(tunnelEndPoint)) {
        tunnel.setEndpoint(tunnelEndPoint);
      }
      if (partition == null) {
        uploadSession = tunnel.createUploadSession(project, tableName);
      } else {
        uploadSession =
            tunnel.createUploadSession(project, tableName, new PartitionSpec(partition));
      }
      odpsWriter =
          buildTunnelWriter(project, tableName, tunnelEndPoint, retryCount, uploadSession);
    } else {
      odpsWriter =
          buildTunnelWriter(project, tableName, tunnelEndPoint, retryCount, new String(""));
    }
  } catch (Exception e) {
    throw new RuntimeException(e);
  }
}
開發者ID:aliyun，項目名稱:aliyun-maxcompute-data-collectors，代碼行數:57，代碼來源:OdpsUploadProcessor.java


注：本文中的com.aliyun.odps.tunnel.TableTunnel.setEndpoint方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。