当前位置: 首页>>代码示例>>Java>>正文


Java FileStatus.getPath方法代码示例

本文整理汇总了Java中org.apache.flink.core.fs.FileStatus.getPath方法的典型用法代码示例。如果您正苦于以下问题:Java FileStatus.getPath方法的具体用法?Java FileStatus.getPath怎么用?Java FileStatus.getPath使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在org.apache.flink.core.fs.FileStatus的用法示例。


在下文中一共展示了FileStatus.getPath方法的4个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: listNewFiles

import org.apache.flink.core.fs.FileStatus; //导入方法依赖的package包/类
private List<String> listNewFiles(FileSystem fileSystem) throws IOException {
	List<String> files = new ArrayList<String>();

	FileStatus[] statuses = fileSystem.listStatus(new Path(path));

	if (statuses == null) {
		LOG.warn("Path does not exist: {}", path);
	} else {
		for (FileStatus status : statuses) {
			Path filePath = status.getPath();
			String fileName = filePath.getName();
			long modificationTime = status.getModificationTime();

			if (!isFiltered(fileName, modificationTime)) {
				files.add(filePath.toString());
				modificationTimes.put(fileName, modificationTime);
			}
		}
	}

	return files;
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:23,代码来源:FileMonitoringFunction.java

示例2: getCopyTasks

import org.apache.flink.core.fs.FileStatus; //导入方法依赖的package包/类
private static void getCopyTasks(Path p, String rel, List<FileCopyTask> tasks) throws IOException {
	FileStatus[] res = p.getFileSystem().listStatus(p);
	if (res == null) {
		return;
	}
	for (FileStatus fs : res) {
		if (fs.isDir()) {
			getCopyTasks(fs.getPath(), rel + fs.getPath().getName() + "/", tasks);
		} else {
			Path cp = fs.getPath();
			tasks.add(new FileCopyTask(cp, rel + cp.getName()));
		}
	}
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:15,代码来源:DistCp.java

示例3: materializeSnapshot

import org.apache.flink.core.fs.FileStatus; //导入方法依赖的package包/类
KeyedStateHandle materializeSnapshot() throws Exception {

			stateBackend.cancelStreamRegistry.registerCloseable(closeableRegistry);

			// write meta data
			metaStateHandle = materializeMetaData();

			// write state data
			Preconditions.checkState(backupFileSystem.exists(backupPath));

			FileStatus[] fileStatuses = backupFileSystem.listStatus(backupPath);
			if (fileStatuses != null) {
				for (FileStatus fileStatus : fileStatuses) {
					final Path filePath = fileStatus.getPath();
					final String fileName = filePath.getName();
					final StateHandleID stateHandleID = new StateHandleID(fileName);

					if (fileName.endsWith(SST_FILE_SUFFIX)) {
						final boolean existsAlready =
							baseSstFiles != null && baseSstFiles.contains(stateHandleID);

						if (existsAlready) {
							// we introduce a placeholder state handle, that is replaced with the
							// original from the shared state registry (created from a previous checkpoint)
							sstFiles.put(
								stateHandleID,
								new PlaceholderStreamStateHandle());
						} else {
							sstFiles.put(stateHandleID, materializeStateData(filePath));
						}
					} else {
						StreamStateHandle fileHandle = materializeStateData(filePath);
						miscFiles.put(stateHandleID, fileHandle);
					}
				}
			}

			synchronized (stateBackend.materializedSstFiles) {
				stateBackend.materializedSstFiles.put(checkpointId, sstFiles.keySet());
			}

			return new IncrementalKeyedStateHandle(
				stateBackend.backendUID,
				stateBackend.keyGroupRange,
				checkpointId,
				sstFiles,
				miscFiles,
				metaStateHandle);
		}
 
开发者ID:axbaretto,项目名称:flink,代码行数:50,代码来源:RocksDBKeyedStateBackend.java

示例4: listEligibleFiles

import org.apache.flink.core.fs.FileStatus; //导入方法依赖的package包/类
/**
 * Returns the paths of the files not yet processed.
 * @param fileSystem The filesystem where the monitored directory resides.
 */
private Map<Path, FileStatus> listEligibleFiles(FileSystem fileSystem, Path path) throws IOException {

	final FileStatus[] statuses;
	try {
		statuses = fileSystem.listStatus(path);
	} catch (IOException e) {
		// we may run into an IOException if files are moved while listing their status
		// delay the check for eligible files in this case
		return Collections.emptyMap();
	}

	if (statuses == null) {
		LOG.warn("Path does not exist: {}", path);
		return Collections.emptyMap();
	} else {
		Map<Path, FileStatus> files = new HashMap<>();
		// handle the new files
		for (FileStatus status : statuses) {
			if (!status.isDir()) {
				Path filePath = status.getPath();
				long modificationTime = status.getModificationTime();
				if (!shouldIgnore(filePath, modificationTime)) {
					files.put(filePath, status);
				}
			} else if (format.getNestedFileEnumeration() && format.acceptFile(status)){
				files.putAll(listEligibleFiles(fileSystem, status.getPath()));
			}
		}
		return files;
	}
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:36,代码来源:ContinuousFileMonitoringFunction.java


注:本文中的org.apache.flink.core.fs.FileStatus.getPath方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。