当前位置: 首页>>代码示例>>Java>>正文


Java ObjectListing.isTruncated方法代码示例

本文整理汇总了Java中com.amazonaws.services.s3.model.ObjectListing.isTruncated方法的典型用法代码示例。如果您正苦于以下问题:Java ObjectListing.isTruncated方法的具体用法?Java ObjectListing.isTruncated怎么用?Java ObjectListing.isTruncated使用的例子?那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在com.amazonaws.services.s3.model.ObjectListing的用法示例。


在下文中一共展示了ObjectListing.isTruncated方法的15个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: listDirectChildren

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
public List<String> listDirectChildren(URI parent) {
    S3RegionalResource s3RegionalResource = new S3RegionalResource(parent);
    String bucketName = s3RegionalResource.getBucketName();
    String s3BucketKey = s3RegionalResource.getKey();
    configureClient(s3RegionalResource);

    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
        .withBucketName(bucketName)
        .withPrefix(s3BucketKey)
        .withMaxKeys(1000)
        .withDelimiter("/");
    ObjectListing objectListing = amazonS3Client.listObjects(listObjectsRequest);
    ImmutableList.Builder<String> builder = ImmutableList.builder();
    builder.addAll(resourceResolver.resolveResourceNames(objectListing));

    while (objectListing.isTruncated()) {
        objectListing = amazonS3Client.listNextBatchOfObjects(objectListing);
        builder.addAll(resourceResolver.resolveResourceNames(objectListing));
    }
    return builder.build();
}
 
开发者ID:lxxlxx888,项目名称:Reer,代码行数:22,代码来源:S3Client.java

示例2: deleteAll

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
public void deleteAll(String bucket) {
    logger.info("delete all from bucket, bucket={}", bucket);

    ObjectListing listing = s3.listObjects(new ListObjectsRequest().withBucketName(bucket));

    while (listing != null) {
        List<DeleteObjectsRequest.KeyVersion> keys = new ArrayList<>(listing.getObjectSummaries().size());
        for (S3ObjectSummary summary : listing.getObjectSummaries()) {
            String key = summary.getKey();
            logger.info("add key to deletion batch, key={}", key);
            keys.add(new DeleteObjectsRequest.KeyVersion(key));
        }
        if (!keys.isEmpty()) {
            logger.info("delete key batch");
            s3.deleteObjects(new DeleteObjectsRequest(bucket).withKeys(keys));
        }
        if (!listing.isTruncated()) return;

        listing = s3.listNextBatchOfObjects(listing);
    }
}
 
开发者ID:neowu,项目名称:cmn-project,代码行数:22,代码来源:S3.java

示例3: listUris

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public List<URI> listUris(URI uri, Predicate<URI> uriPredicate) throws IOException {
    String bucketName = uri.getHost();
    if (client == null) {
        client = clientBuilder.client(uri);
    }
    String prefix = uri.getPath().length() > 1 ? uri.getPath().substring(1) : "";
    List<URI> uris = new ArrayList<>();
    ObjectListing list = client.listObjects(bucketName, prefix);
    addKeyUris(uris, list, uri, uriPredicate);
    while (list.isTruncated()) {
        list = client.listNextBatchOfObjects(list);
        addKeyUris(uris, list, uri, uriPredicate);
    }

    return uris;
}
 
开发者ID:baidu,项目名称:Elasticsearch,代码行数:18,代码来源:S3FileInput.java

示例4: listNextBatchOfObjects

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public ObjectListing listNextBatchOfObjects(ListNextBatchOfObjectsRequest listNextBatchOfObjectsRequest)
        throws SdkClientException, AmazonServiceException {
    listNextBatchOfObjectsRequest = beforeClientExecution(listNextBatchOfObjectsRequest);
    rejectNull(listNextBatchOfObjectsRequest,
            "The request object parameter must be specified when listing the next batch of objects in a bucket");
    ObjectListing previousObjectListing = listNextBatchOfObjectsRequest.getPreviousObjectListing();

    if (!previousObjectListing.isTruncated()) {
        ObjectListing emptyListing = new ObjectListing();
        emptyListing.setBucketName(previousObjectListing.getBucketName());
        emptyListing.setDelimiter(previousObjectListing.getDelimiter());
        emptyListing.setMarker(previousObjectListing.getNextMarker());
        emptyListing.setMaxKeys(previousObjectListing.getMaxKeys());
        emptyListing.setPrefix(previousObjectListing.getPrefix());
        emptyListing.setEncodingType(previousObjectListing.getEncodingType());
        emptyListing.setTruncated(false);

        return emptyListing;
    }
    return listObjects(listNextBatchOfObjectsRequest.toListObjectsRequest());
}
 
开发者ID:IBM,项目名称:ibm-cos-sdk-java,代码行数:23,代码来源:AmazonS3Client.java

示例5: scanBucket

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
public void scanBucket() {
    ListObjectsRequest listReq = new ListObjectsRequest()
        .withPrefix(opt.prefix())
        .withBucketName(opt.bucket());

    Logger.Info("Scanning S3 bucket %s %s", opt.bucket(), opt.prefix());
    ObjectListing listing = s3.listObjects(listReq);
    boolean ok = processObjects(listing.getObjectSummaries());

    while (ok && listing.isTruncated()) {
        listing = s3.listNextBatchOfObjects(listing);
        ok = processObjects(listing.getObjectSummaries());
    }

    Logger.Info("Completed scan, added %s images to the processing queue.", numSeen.get());
}
 
开发者ID:jhy,项目名称:RekognitionS3Batch,代码行数:17,代码来源:Scanner.java

示例6: list

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
@SuppressWarnings("unused")
public List<AwsFileMiniModel> list(String prefix) {

	AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider());
	List<AwsFileMiniModel> files = new ArrayList();

	ListObjectsRequest listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withPrefix(prefix);
	ObjectListing objectListing;

	do {
		objectListing = s3client.listObjects(listObjectsRequest);

		for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
			System.out.println(" - " + objectSummary.getKey() + "  " + "(size = " + objectSummary.getSize() + ")"
					+ " (date = " + objectSummary.getLastModified() + ")");
			files.add(new AwsFileMiniModel(objectSummary.getKey(), objectSummary.getLastModified()));
		}
		listObjectsRequest.setMarker(objectListing.getNextMarker());
	} while (objectListing.isTruncated());

	return files;

}
 
开发者ID:ismartonline,项目名称:ismartonline,代码行数:25,代码来源:AwsFileManager.java

示例7: listObjects

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
public static List<S3ObjectSummary> listObjects(String bucket, String prefix, AmazonS3 s3) {
  List<S3ObjectSummary> objects = new ArrayList<>();
  ObjectListing listing;

  try {
    if (prefix == null) {
      listing = s3.listObjects(bucket);
    } else {
      listing = s3.listObjects(bucket, prefix);
    }

    objects.addAll(listing.getObjectSummaries());
    while (listing.isTruncated()) {
      listing = s3.listNextBatchOfObjects(listing);
      objects.addAll(listing.getObjectSummaries());
    }
  } catch (AmazonS3Exception e) {
   log.warn("listObjects for bucket '{}' prefix '{}' returned error code: {}", bucket, prefix, e.getStatusCode());
  }

  return objects;
}
 
开发者ID:confluentinc,项目名称:kafka-connect-storage-cloud,代码行数:23,代码来源:TestWithMockedS3.java

示例8: deleteByTenant

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public void deleteByTenant(final String tenant) {
    final String folder = sanitizeTenant(tenant);

    LOG.info("Deleting S3 object folder (tenant) from bucket {} and key {}", s3Properties.getBucketName(), folder);

    // Delete artifacts
    ObjectListing objects = amazonS3.listObjects(s3Properties.getBucketName(), folder + "/");
    do {
        for (final S3ObjectSummary objectSummary : objects.getObjectSummaries()) {
            amazonS3.deleteObject(s3Properties.getBucketName(), objectSummary.getKey());
        }
        objects = amazonS3.listNextBatchOfObjects(objects);
    } while (objects.isTruncated());

}
 
开发者ID:eclipse,项目名称:hawkbit-extensions,代码行数:17,代码来源:S3Repository.java

示例9: getAllSummaries

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
/**
 * Method returns list of all <b>S3ObjectSummary</b> objects, subject to
 * <i>req</i> parameters. Multiple S3 calls will be performed if there are
 * more than 1000 elements there
 * 
 * @param req
 *            - ListObjectRequest to be used.
 * @return List of S3ObjectSummary from bucket,
 */
public List<S3ObjectSummary> getAllSummaries(ListObjectsRequest req) {
    List<S3ObjectSummary> result = new ArrayList<>();
    String marker = null;
    ListObjectsRequest req2 = (ListObjectsRequest) req.clone();
    ObjectListing listing;
    do {
        listing = client.listObjects(req2.withMarker(marker));
        marker = listing.getNextMarker();
        result.addAll(listing.getObjectSummaries());
    } while (listing.isTruncated());

    return result;
}
 
开发者ID:aol,项目名称:micro-server,代码行数:23,代码来源:S3Utils.java

示例10: list

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public List<NoteInfo> list(AuthenticationInfo subject) throws IOException {
  List<NoteInfo> infos = new LinkedList<>();
  NoteInfo info;
  try {
    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
            .withBucketName(bucketName)
            .withPrefix(user + "/" + "notebook");
    ObjectListing objectListing;
    do {
      objectListing = s3client.listObjects(listObjectsRequest);
      for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
        if (objectSummary.getKey().endsWith("note.json")) {
          info = getNoteInfo(objectSummary.getKey());
          if (info != null) {
            infos.add(info);
          }
        }
      }
      listObjectsRequest.setMarker(objectListing.getNextMarker());
    } while (objectListing.isTruncated());
  } catch (AmazonClientException ace) {
    throw new IOException("Unable to list objects in S3: " + ace, ace);
  }
  return infos;
}
 
开发者ID:apache,项目名称:zeppelin,代码行数:27,代码来源:S3NotebookRepo.java

示例11: remove

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public void remove(String noteId, AuthenticationInfo subject) throws IOException {
  String key = user + "/" + "notebook" + "/" + noteId;
  final ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
      .withBucketName(bucketName).withPrefix(key);

  try {
    ObjectListing objects = s3client.listObjects(listObjectsRequest);
    do {
      for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) {
        s3client.deleteObject(bucketName, objectSummary.getKey());
      }
      objects = s3client.listNextBatchOfObjects(objects);
    } while (objects.isTruncated());
  }
  catch (AmazonClientException ace) {
    throw new IOException("Unable to remove note in S3: " + ace, ace);
  }
}
 
开发者ID:apache,项目名称:zeppelin,代码行数:20,代码来源:S3NotebookRepo.java

示例12: afterSpiStopped

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
/**
 * @throws Exception If error.
 */
@Override protected void afterSpiStopped() throws Exception {
    AWSCredentials cred = new BasicAWSCredentials(IgniteS3TestSuite.getAccessKey(),
        IgniteS3TestSuite.getSecretKey());

    AmazonS3 s3 = new AmazonS3Client(cred);

    String bucketName = S3CheckpointSpi.BUCKET_NAME_PREFIX + "unit-test-bucket";

    try {
        ObjectListing list = s3.listObjects(bucketName);

        while (true) {
            for (S3ObjectSummary sum : list.getObjectSummaries())
                s3.deleteObject(bucketName, sum.getKey());

            if (list.isTruncated())
                list = s3.listNextBatchOfObjects(list);
            else
                break;
        }
    }
    catch (AmazonClientException e) {
        throw new IgniteSpiException("Failed to read checkpoint bucket: " + bucketName, e);
    }
}
 
开发者ID:apache,项目名称:ignite,代码行数:29,代码来源:S3CheckpointSpiSelfTest.java

示例13: doExecute

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
protected void doExecute(Parameters config, Parameters output) {
  String bucketName = this.readAsString(config, "aws.s3.bucketName", null, false, "aws.s3.bucketName not present in config");
  String prefix = this.readAsString(config, "aws.s3.list.prefix", null, false, "aws.s3.list.prefix not present");
  
  AmazonS3 client = new AmazonS3Client(AwsUtils.getCredentialProviderC(config), AwsUtils.getClientConfig(config));
  client.setRegion(AwsUtils.getRegion(config));
  
  ListObjectsRequest listObjectsRequest =
      new ListObjectsRequest().withBucketName(bucketName).withPrefix(prefix);

  List<S3ObjectSummary> summarys = new ArrayList<S3ObjectSummary>();
  ObjectListing objectListing;
  do {
    objectListing = client.listObjects(listObjectsRequest);
    summarys.addAll(objectListing.getObjectSummaries());
    String marker = objectListing.getNextMarker();
    listObjectsRequest.setMarker(marker);
    
    Detective.info("Reading from S3...current marker:" + marker + " Continue..." );
  } while (objectListing.isTruncated());
  
  output.put("s3ObjectSummaries", summarys);
}
 
开发者ID:detectiveframework,项目名称:detective,代码行数:25,代码来源:S3ListTask.java

示例14: findProgressivelyWithPartialMatch

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
/**
 * Searches for matching keys progressively. This means that instead of retrieving all keys given a prefix, it goes
 * down one level at a time and filters out all non-matching results. This avoids a lot of unused requests results.
 * WARNING: This method does not truncate results. Therefore all matching resources will be returned regardless of
 * the truncation.
 */
private void findProgressivelyWithPartialMatch(String bucketName, Set<Resource> resources, String prefix, String keyPattern) {
    ListObjectsRequest listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withDelimiter("/").withPrefix(prefix);
    ObjectListing objectListing = null;

    do {
        if (objectListing == null) {
            objectListing = this.amazonS3.listObjects(listObjectsRequest);
        } else {
            objectListing = this.amazonS3.listNextBatchOfObjects(objectListing);
        }

        Set<Resource> newResources = getResourcesFromObjectSummaries(bucketName, keyPattern, objectListing.getObjectSummaries());
        if (!newResources.isEmpty()) {
            resources.addAll(newResources);
        }

        for (String commonPrefix : objectListing.getCommonPrefixes()) {
            if (isKeyPathMatchesPartially(keyPattern, commonPrefix)) {
                findPathMatchingKeyInBucket(bucketName, resources, commonPrefix, keyPattern);
            }
        }
    } while (objectListing.isTruncated());
}
 
开发者ID:spring-cloud,项目名称:spring-cloud-aws,代码行数:30,代码来源:PathMatchingSimpleStorageResourcePatternResolver.java

示例15: deleteCollection

import com.amazonaws.services.s3.model.ObjectListing; //导入方法依赖的package包/类
@Override
public void deleteCollection(Map<String, Object> attributes, String collectionName) 
   throws IOException 
{
   if (s3Client==null) {
      throw new IOException("S3 Client is not available due to configuration error.");
   }
   String bucketName = getBucketName(attributes);
   if (bucketName==null) {
      throw new IOException("S3 Bucket not configured for "+attributes.get("app.database"));
   }
   try {
      ObjectListing listing = null;
      do {
         listing = listing==null ? s3Client.listObjects(bucketName, collectionName+"/") : s3Client.listNextBatchOfObjects(listing);
         for (S3ObjectSummary summary : listing.getObjectSummaries()) {
            s3Client.deleteObject(bucketName, summary.getKey());
         }
      } while (listing.isTruncated());
   } catch (AmazonClientException ex) {
      throw new IOException("Cannot delete collection objects due to S3 error.",ex);
   }
}
 
开发者ID:alexmilowski,项目名称:xproclet,代码行数:24,代码来源:S3MediaStorage.java


注:本文中的com.amazonaws.services.s3.model.ObjectListing.isTruncated方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。