當前位置: 首頁>>代碼示例>>Java>>正文

Java FileSystem.getHomeDirectory方法代碼示例

本文整理匯總了Java中org.apache.hadoop.fs.FileSystem.getHomeDirectory方法的典型用法代碼示例。如果您正苦於以下問題：Java FileSystem.getHomeDirectory方法的具體用法？Java FileSystem.getHomeDirectory怎麽用？Java FileSystem.getHomeDirectory使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類org.apache.hadoop.fs.FileSystem的用法示例。


在下文中一共展示了FileSystem.getHomeDirectory方法的11個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: copyLocalFileToDfs

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
public static Path copyLocalFileToDfs(FileSystem fs, String appId,
    Path srcPath, String dstFileName) throws IOException {
  Path dstPath = new Path(fs.getHomeDirectory(),
      Constants.DEFAULT_APP_NAME + Path.SEPARATOR + appId + Path.SEPARATOR + dstFileName);
  LOG.info("Copying " + srcPath + " to " + dstPath);
  fs.copyFromLocalFile(srcPath, dstPath);
  return dstPath;
}
開發者ID:Intel-bigdata，項目名稱:TensorFlowOnYARN，代碼行數:9，代碼來源:Utils.java

示例2: getFilesCount

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
public int getFilesCount(String storeBaseDir, String tableName) {
  int filesCount = 0;
  try {
    FileSystem fs = FileSystem.get(conf);
    Path storeBasePath = new Path(fs.getHomeDirectory(), storeBaseDir);
    Path tablePath = new Path(storeBasePath, tableName);
    if (fs.exists(tablePath)) {
      RemoteIterator<LocatedFileStatus> locatedFileStatusRemoteIterator =
          fs.listFiles(tablePath, false);
      while (locatedFileStatusRemoteIterator.hasNext()) {
        filesCount++;
        LocatedFileStatus next = locatedFileStatusRemoteIterator.next();
        System.out.println("File name is " + next.getPath());
      }
    }
  } catch (IOException e) {
    e.printStackTrace();
  }
  return filesCount;
}
開發者ID:ampool，項目名稱:monarch，代碼行數:21，代碼來源:HDFSQuasiService.java

示例3: getORCRecords

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
public List<OrcStruct> getORCRecords(String storeBaseDir, String tableName) throws IOException {
  List<OrcStruct> orcrecords = new ArrayList<>();
  try {
    FileSystem fs = FileSystem.get(conf);
    Path storeBasePath = new Path(fs.getHomeDirectory(), storeBaseDir);
    Path tablePath = new Path(storeBasePath, tableName);
    if (fs.exists(tablePath)) {
      RemoteIterator<LocatedFileStatus> locatedFileStatusRemoteIterator =
          fs.listFiles(tablePath, false);
      while (locatedFileStatusRemoteIterator.hasNext()) {
        LocatedFileStatus next = locatedFileStatusRemoteIterator.next();
        final org.apache.hadoop.hive.ql.io.orc.Reader fis =
            OrcFile.createReader(next.getPath(), OrcFile.readerOptions(conf));
        RecordReader rows = fis.rows();
        while (rows.hasNext()) {
          orcrecords.add((OrcStruct) rows.next(null));
        }
        System.out.println("File name is " + next.getPath());
      }
    }
  } catch (IOException e) {
    e.printStackTrace();
  }
  return orcrecords;
}
開發者ID:ampool，項目名稱:monarch，代碼行數:26，代碼來源:HDFSQuasiService.java

示例4: testHomeDirectory

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
/**
 * Tests get/set working directory in DFS.
 */
@Test(timeout=30000)
public void testHomeDirectory() throws IOException {
  final String[] homeBases = new String[] {"/home", "/home/user"};
  Configuration conf = new HdfsConfiguration();
  for (final String homeBase : homeBases) {
    conf.set(DFSConfigKeys.DFS_USER_HOME_DIR_PREFIX_KEY, homeBase);
    MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).build();
    FileSystem fileSys = cluster.getFileSystem();
    try {    
      // test home directory
      Path home = 
          fileSys.makeQualified(
              new Path(homeBase + "/" + getUserName(fileSys))); 
      Path fsHome = fileSys.getHomeDirectory();
      assertEquals(home, fsHome);
    } finally {
      fileSys.close();
      cluster.shutdown();
    }
  }
}
開發者ID:naver，項目名稱:hadoop，代碼行數:25，代碼來源:TestLocalDFS.java

示例5: addToLocalResources

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
private void addToLocalResources(FileSystem fs, String fileSrcPath,
																 String fileDstPath, String appId, Map<String, LocalResource> localResources,
																 String resources) throws IOException {
	String suffix =
			"prkeyrotation" + "/" + appId + "/" + fileDstPath;
	Path dst =
			new Path(fs.getHomeDirectory(), suffix);
	if (fileSrcPath == null) {
		FSDataOutputStream ostream = null;
		try {
			ostream = FileSystem
					.create(fs, dst, new FsPermission((short) 0710));
			ostream.writeUTF(resources);
		} finally {
			IOUtils.closeQuietly(ostream);
		}
	} else {
		fs.copyFromLocalFile(new Path(fileSrcPath), dst);
	}
	FileStatus scFileStatus = fs.getFileStatus(dst);
	LocalResource scRsrc =
			LocalResource.newInstance(
					ConverterUtils.getYarnUrlFromPath(dst),
					LocalResourceType.FILE, LocalResourceVisibility.APPLICATION,
					scFileStatus.getLen(), scFileStatus.getModificationTime());
	localResources.put(fileDstPath, scRsrc);
}
開發者ID:nucypher，項目名稱:hadoop-oss，代碼行數:28，代碼來源:Client.java

示例6: getHomeDirectory

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
/**
 * 此方法用於獲取文件係統的HomeDirectory
 *
 * @param fileSystemInfo
 *            文件係統信息
 * @return 文件係統的HomeDirectory
 */
public static Path getHomeDirectory(FileSystemInfo fileSystemInfo) {
	FileSystem fs = getFileSystem(fileSystemInfo);
	try {
		return fs.getHomeDirectory();
	} finally {
		closeFileSystem(fs);
	}
}
開發者ID:zhangjunfang，項目名稱:alluxio，代碼行數:16，代碼來源:HdfsAndAlluxioUtils_update.java

示例7: checkFileExistsSecured

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
public boolean checkFileExistsSecured(final String user, final String keytab, String storeBaseDir,
    String tableName) {
  try {
    UserGroupInformation.setConfiguration(conf);
    UserGroupInformation.loginUserFromKeytab(user, keytab);
    FileSystem fs = FileSystem.get(conf);
    Path storeBasePath = new Path(fs.getHomeDirectory(), storeBaseDir);
    Path tablePath = new Path(storeBasePath, tableName);
    return fs.exists(tablePath);
  } catch (IOException e) {
    e.printStackTrace();
  }
  return false;
}
開發者ID:ampool，項目名稱:monarch，代碼行數:15，代碼來源:HDFSQuasiService.java

示例8: checkFileExists

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
public boolean checkFileExists(String storeBaseDir, String tableName) {
  try {
    FileSystem fs = FileSystem.get(conf);
    Path storeBasePath = new Path(fs.getHomeDirectory(), storeBaseDir);
    Path tablePath = new Path(storeBasePath, tableName);
    return fs.exists(tablePath);
  } catch (IOException e) {
    e.printStackTrace();
  }
  return false;
}
開發者ID:ampool，項目名稱:monarch，代碼行數:12，代碼來源:HDFSQuasiService.java

示例9: addToLocalResources

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
private void addToLocalResources(FileSystem fs, String fileSrcPath,
    String fileDstPath, String appId, Map<String, LocalResource> localResources,
    String resources) throws IOException {
  String suffix =
      appName + "/" + appId + "/" + fileDstPath;
  Path dst =
      new Path(fs.getHomeDirectory(), suffix);
  if (fileSrcPath == null) {
    FSDataOutputStream ostream = null;
    try {
      ostream = FileSystem
          .create(fs, dst, new FsPermission((short) 0710));
      ostream.writeUTF(resources);
    } finally {
      IOUtils.closeQuietly(ostream);
    }
  } else {
    fs.copyFromLocalFile(new Path(fileSrcPath), dst);
  }
  FileStatus scFileStatus = fs.getFileStatus(dst);
  LocalResource scRsrc =
      LocalResource.newInstance(
          ConverterUtils.getYarnUrlFromURI(dst.toUri()),
          LocalResourceType.FILE, LocalResourceVisibility.APPLICATION,
          scFileStatus.getLen(), scFileStatus.getModificationTime());
  localResources.put(fileDstPath, scRsrc);
}
開發者ID:naver，項目名稱:hadoop，代碼行數:28，代碼來源:Client.java

示例10: testWorkingDirectory

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
/**
 * Tests get/set working directory in DFS.
 */
@Test
public void testWorkingDirectory() throws IOException {
  Configuration conf = new HdfsConfiguration();
  MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).build();
  FileSystem fileSys = cluster.getFileSystem();
  try {
    Path orig_path = fileSys.getWorkingDirectory();
    assertTrue(orig_path.isAbsolute());
    Path file1 = new Path("somewhat/random.txt");
    writeFile(fileSys, file1);
    assertTrue(fileSys.exists(new Path(orig_path, file1.toString())));
    fileSys.delete(file1, true);
    Path subdir1 = new Path("/somewhere");
    fileSys.setWorkingDirectory(subdir1);
    writeFile(fileSys, file1);
    cleanupFile(fileSys, new Path(subdir1, file1.toString()));
    Path subdir2 = new Path("else");
    fileSys.setWorkingDirectory(subdir2);
    writeFile(fileSys, file1);
    readFile(fileSys, file1);
    cleanupFile(fileSys, new Path(new Path(subdir1, subdir2.toString()),
                                  file1.toString()));

    // test home directory
    Path home = 
      fileSys.makeQualified(
          new Path(DFSConfigKeys.DFS_USER_HOME_DIR_PREFIX_DEFAULT
              + "/" + getUserName(fileSys))); 
    Path fsHome = fileSys.getHomeDirectory();
    assertEquals(home, fsHome);

  } finally {
    fileSys.close();
    cluster.shutdown();
  }
}
開發者ID:naver，項目名稱:hadoop，代碼行數:40，代碼來源:TestLocalDFS.java

示例11: execute

import org.apache.hadoop.fs.FileSystem; //導入方法依賴的package包/類
/**
 * Executes the filesystem operation.
 *
 * @param fs filesystem instance to use.
 *
 * @return a JSON object with the user home directory.
 *
 * @throws IOException thrown if an IO error occured.
 */
@Override
@SuppressWarnings("unchecked")
public JSONObject execute(FileSystem fs) throws IOException {
  Path homeDir = fs.getHomeDirectory();
  JSONObject json = new JSONObject();
  json.put(HttpFSFileSystem.HOME_DIR_JSON, homeDir.toUri().getPath());
  return json;
}
開發者ID:naver，項目名稱:hadoop，代碼行數:18，代碼來源:FSOperations.java


注：本文中的org.apache.hadoop.fs.FileSystem.getHomeDirectory方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。