Java Crawler類代碼示例

本文整理匯總了Java中org.jbake.app.Crawler類的典型用法代碼示例。如果您正苦於以下問題：Java Crawler類的具體用法？Java Crawler怎麽用？Java Crawler使用的例子？那麽, 這裏精選的類代碼示例或許可以為您提供幫助。

Crawler類屬於org.jbake.app包，在下文中一共展示了Crawler類的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: testRenderFileWorksWhenPathHasDotInButFileDoesNot

import org.jbake.app.Crawler; //導入依賴的package包/類
/**
 * See issue #300
 * 
 * @throws Exception
 */
@Test
   public void testRenderFileWorksWhenPathHasDotInButFileDoesNot() throws Exception {
	final String FOLDER = "real.path";
	final String FILENAME = "about";
	config.setProperty(Keys.OUTPUT_EXTENSION, "");
	Renderer renderer = new Renderer(db, outputPath, folder.newFolder("templates"), config, renderingEngine);
	
	Map<String, Object> content = new HashMap<String, Object>();
	content.put(Crawler.Attributes.TYPE, "page");
	content.put(Crawler.Attributes.URI, "/" + FOLDER + "/" + FILENAME);
	content.put(Crawler.Attributes.STATUS, "published");
	
	renderer.render(content);
	
	File outputFile = new File(outputPath.getAbsolutePath() + File.separatorChar + FOLDER + File.separatorChar + FILENAME);
	assertThat(outputFile).isFile();
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:23，代碼來源:RendererTest.java

示例2: renderPost

import org.jbake.app.Crawler; //導入依賴的package包/類
@Test
public void renderPost() throws Exception {
    // setup
    String filename = "second-post.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content"
            + File.separator + "blog" + File.separator + "2013" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("post")) {
        assertThat(output).contains(string);
    }
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:20，代碼來源:AbstractTemplateEngineRenderingTest.java

示例3: renderPage

import org.jbake.app.Crawler; //導入依賴的package包/類
@Test
public void renderPage() throws Exception {
    // setup
    String filename = "about.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("page")) {
        assertThat(output).contains(string);
    }
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:19，代碼來源:AbstractTemplateEngineRenderingTest.java

示例4: testRenderFileWorksWhenPathHasDotInButFileDoesNot

import org.jbake.app.Crawler; //導入依賴的package包/類
/**
 * See issue #300
 * 
 * @throws Exception
 */
@Test
   public void testRenderFileWorksWhenPathHasDotInButFileDoesNot() throws Exception {
	String FOLDER = "real.path";

	final String FILENAME = "about";
	config.setProperty(Keys.OUTPUT_EXTENSION, "");
	Renderer renderer = new Renderer(db, outputPath, folder.newFolder("templates"), config, renderingEngine);
	
	Map<String, Object> content = new HashMap<String, Object>();
	content.put(Crawler.Attributes.TYPE, "page");
	content.put(Crawler.Attributes.URI, "/" + FOLDER + "/" + FILENAME);
	content.put(Crawler.Attributes.STATUS, "published");
	
	renderer.render(content);
	
	File outputFile = new File(outputPath.getAbsolutePath() + File.separatorChar + FOLDER + File.separatorChar + FILENAME);
	assertThat(outputFile).isFile();
}

開發者ID:jbake-org，項目名稱:jbake，代碼行數:24，代碼來源:RendererTest.java

示例5: processHeader

import org.jbake.app.Crawler; //導入依賴的package包/類
/**
 * Process the header of the file.
 * @param config 
 *
 * @param contents Contents of file
 * @param content
 */
private void processHeader(Configuration config, List<String> contents, final Map<String, Object> content) {
    for (String line : contents) {
        if (line.equals(HEADER_SEPARATOR)) {
            break;
        }

        if (line.isEmpty()) {
        	continue;
        }
        
        String[] parts = line.split("=",2);
        if (parts.length != 2) {
            continue;
        }

        String key = parts[0].trim();
        String value = parts[1].trim();

        if (key.equalsIgnoreCase(Crawler.Attributes.DATE)) {
            DateFormat df = new SimpleDateFormat(config.getString(Keys.DATE_FORMAT));
            Date date = null;
            try {
                date = df.parse(value);
                content.put(key, date);
            } catch (ParseException e) {
                e.printStackTrace();
            }
        } else if (key.equalsIgnoreCase(Crawler.Attributes.TAGS)) {
            content.put(key, getTags(value));
        } else if (isJson(value)) {
            content.put(key, JSONValue.parse(value));
        } else {
            content.put(key, value);
        }
    }
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:44，代碼來源:MarkupEngine.java

示例6: get

import org.jbake.app.Crawler; //導入依賴的package包/類
@Override
public DocumentList get(ContentStore db, Map model, String key) {
    String tag = null;
    if (model.get(Crawler.Attributes.TAG) != null) {
        tag = model.get(Crawler.Attributes.TAG).toString();
    }
    // fetch the tagged documents from db
    return db.getPublishedDocumentsByTag(tag);
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:10，代碼來源:TaggedDocumentsExtractor.java

示例7: get

import org.jbake.app.Crawler; //導入依賴的package包/類
@Override
public DocumentList get(ContentStore db, Map model, String key) {
    String tag = null;
    if (model.get(Crawler.Attributes.TAG) != null) {
        tag = model.get(Crawler.Attributes.TAG).toString();
    }
    // fetch the tag posts from db
    return db.getPublishedPostsByTag(tag);
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:10，代碼來源:TagPostsExtractor.java

示例8: renderCustomTypePaper

import org.jbake.app.Crawler; //導入依賴的package包/類
@Test
public void renderCustomTypePaper() throws Exception {
    // setup
    config.setProperty("template.paper.file", "paper." + templateExtension);
    DocumentTypes.addDocumentType("paper");
    DBUtil.updateSchema(db);

    Crawler crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    Parser parser = new Parser(config, sourceFolder.getPath());
    Renderer renderer = new Renderer(db, destinationFolder, templateFolder, config);
    String filename = "published-paper.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + "papers" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile);
    for (String string : getOutputStrings("paper")) {
        assertThat(output).contains(string);
    }

}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:28，代碼來源:GroovyMarkupTemplateEngineRenderingTest.java

示例9: renderCustomTypePaper

import org.jbake.app.Crawler; //導入依賴的package包/類
@Test
public void renderCustomTypePaper() throws Exception {
    // setup
    config.setProperty("template.paper.file", "paper." + templateExtension);
    DocumentTypes.addDocumentType("paper");
    DBUtil.updateSchema(db);

    Crawler crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    Parser parser = new Parser(config, sourceFolder.getPath());
    Renderer renderer = new Renderer(db, destinationFolder, templateFolder, config);
    String filename = "published-paper.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + "papers" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("paper")) {
        assertThat(output).contains(string);
    }

}

開發者ID:jbake-org，項目名稱:jbake，代碼行數:28，代碼來源:GroovyMarkupTemplateEngineRenderingTest.java

示例10: getBody

import org.jbake.app.Crawler; //導入依賴的package包/類
public String getBody() {
    return contents.get(Crawler.Attributes.BODY).toString();
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:4，代碼來源:ParserContext.java

示例11: setBody

import org.jbake.app.Crawler; //導入依賴的package包/類
public void setBody(String str) {
    contents.put(Crawler.Attributes.BODY, str);
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:4，代碼來源:ParserContext.java

示例12: setup

import org.jbake.app.Crawler; //導入依賴的package包/類
@Before
public void setup() throws Exception {
    currentLocale = Locale.getDefault();
    Locale.setDefault(Locale.ENGLISH);

    ModelExtractorsDocumentTypeListener listener = new ModelExtractorsDocumentTypeListener();
    DocumentTypes.addListener(listener);

    URL sourceUrl = this.getClass().getResource("/");

    sourceFolder = new File(sourceUrl.getFile());
    if (!sourceFolder.exists()) {
        throw new Exception("Cannot find sample data structure!");
    }

    destinationFolder = folder.getRoot();

    templateFolder = new File(sourceFolder, templateDir);
    if (!templateFolder.exists()) {
        throw new Exception("Cannot find template folder!");
    }

    config = ConfigUtil.load(new File(this.getClass().getResource("/").getFile()));
    Iterator<String> keys = config.getKeys();
    while (keys.hasNext()) {
        String key = keys.next();
        if (key.startsWith("template") && key.endsWith(".file")) {
            String old = (String) config.getProperty(key);
            config.setProperty(key, old.substring(0, old.length() - 4) + "." + templateExtension);
        }
    }
    Assert.assertEquals(".html", config.getString(ConfigUtil.Keys.OUTPUT_EXTENSION));
    db = DBUtil.createDataStore("memory", "documents"+System.currentTimeMillis());

    crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    parser = new Parser(config, sourceFolder.getPath());
    renderer = new Renderer(db, destinationFolder, templateFolder, config);

    setupExpectedOutputStrings();
}

開發者ID:ghaseminya，項目名稱:jbake-rtl-jalaali，代碼行數:42，代碼來源:AbstractTemplateEngineRenderingTest.java

示例13: processHeader

import org.jbake.app.Crawler; //導入依賴的package包/類
/**
 * Process the header of the file.
 * @param config 
 *
 * @param contents Contents of file
 * @param content
 */
private void processHeader(Configuration config, List<String> contents, final Map<String, Object> content) {
    for (String line : contents) {
        if (line.equals(HEADER_SEPARATOR)) {
            break;
        }

        if (line.isEmpty()) {
        	continue;
        }
        
        String[] parts = line.split("=",2);
        if (parts.length != 2) {
            continue;
        }

        String utf8BOM = "\uFEFF";
        String key;
        if (parts[0].contains(utf8BOM)) {
        	key = parts[0].trim().replace(utf8BOM, "");
        } else {
        	key = parts[0].trim();
        }
        String value = parts[1].trim();

        if (key.equalsIgnoreCase(Crawler.Attributes.DATE)) {
            DateFormat df = new SimpleDateFormat(config.getString(Keys.DATE_FORMAT));
            Date date = null;
            try {
                date = df.parse(value);
                content.put(key, date);
            } catch (ParseException e) {
                e.printStackTrace();
            }
        } else if (key.equalsIgnoreCase(Crawler.Attributes.TAGS)) {
            content.put(key, getTags(value));
        } else if (isJson(value)) {
            content.put(key, JSONValue.parse(value));
        } else {
            content.put(key, value);
        }
    }
}

開發者ID:jbake-org，項目名稱:jbake，代碼行數:50，代碼來源:MarkupEngine.java

示例14: withDate

import org.jbake.app.Crawler; //導入依賴的package包/類
public FakeDocumentBuilder withDate(Date date) {
	fileModel.put(Crawler.Attributes.DATE, date);
	hasDate = true;
	return this;
}

開發者ID:jbake-org，項目名稱:jbake，代碼行數:6，代碼來源:FakeDocumentBuilder.java

示例15: withCurrentDate

import org.jbake.app.Crawler; //導入依賴的package包/類
private FakeDocumentBuilder withCurrentDate() {
    fileModel.put(Crawler.Attributes.DATE, new Date() );
    return this;
}

開發者ID:jbake-org，項目名稱:jbake，代碼行數:5，代碼來源:FakeDocumentBuilder.java

注：本文中的org.jbake.app.Crawler類示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。