当前位置: 首页>>代码示例>>Java>>正文

Java Crawler类代码示例

本文整理汇总了Java中org.jbake.app.Crawler类的典型用法代码示例。如果您正苦于以下问题：Java Crawler类的具体用法？Java Crawler怎么用？Java Crawler使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。


Crawler类属于org.jbake.app包，在下文中一共展示了Crawler类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: testRenderFileWorksWhenPathHasDotInButFileDoesNot

import org.jbake.app.Crawler; //导入依赖的package包/类
/**
 * See issue #300
 * 
 * @throws Exception
 */
@Test
   public void testRenderFileWorksWhenPathHasDotInButFileDoesNot() throws Exception {
	final String FOLDER = "real.path";
	final String FILENAME = "about";
	config.setProperty(Keys.OUTPUT_EXTENSION, "");
	Renderer renderer = new Renderer(db, outputPath, folder.newFolder("templates"), config, renderingEngine);
	
	Map<String, Object> content = new HashMap<String, Object>();
	content.put(Crawler.Attributes.TYPE, "page");
	content.put(Crawler.Attributes.URI, "/" + FOLDER + "/" + FILENAME);
	content.put(Crawler.Attributes.STATUS, "published");
	
	renderer.render(content);
	
	File outputFile = new File(outputPath.getAbsolutePath() + File.separatorChar + FOLDER + File.separatorChar + FILENAME);
	assertThat(outputFile).isFile();
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:23，代码来源:RendererTest.java

示例2: renderPost

import org.jbake.app.Crawler; //导入依赖的package包/类
@Test
public void renderPost() throws Exception {
    // setup
    String filename = "second-post.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content"
            + File.separator + "blog" + File.separator + "2013" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("post")) {
        assertThat(output).contains(string);
    }
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:20，代码来源:AbstractTemplateEngineRenderingTest.java

示例3: renderPage

import org.jbake.app.Crawler; //导入依赖的package包/类
@Test
public void renderPage() throws Exception {
    // setup
    String filename = "about.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("page")) {
        assertThat(output).contains(string);
    }
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:19，代码来源:AbstractTemplateEngineRenderingTest.java

示例4: testRenderFileWorksWhenPathHasDotInButFileDoesNot

import org.jbake.app.Crawler; //导入依赖的package包/类
/**
 * See issue #300
 * 
 * @throws Exception
 */
@Test
   public void testRenderFileWorksWhenPathHasDotInButFileDoesNot() throws Exception {
	String FOLDER = "real.path";

	final String FILENAME = "about";
	config.setProperty(Keys.OUTPUT_EXTENSION, "");
	Renderer renderer = new Renderer(db, outputPath, folder.newFolder("templates"), config, renderingEngine);
	
	Map<String, Object> content = new HashMap<String, Object>();
	content.put(Crawler.Attributes.TYPE, "page");
	content.put(Crawler.Attributes.URI, "/" + FOLDER + "/" + FILENAME);
	content.put(Crawler.Attributes.STATUS, "published");
	
	renderer.render(content);
	
	File outputFile = new File(outputPath.getAbsolutePath() + File.separatorChar + FOLDER + File.separatorChar + FILENAME);
	assertThat(outputFile).isFile();
}
开发者ID:jbake-org，项目名称:jbake，代码行数:24，代码来源:RendererTest.java

示例5: processHeader

import org.jbake.app.Crawler; //导入依赖的package包/类
/**
 * Process the header of the file.
 * @param config 
 *
 * @param contents Contents of file
 * @param content
 */
private void processHeader(Configuration config, List<String> contents, final Map<String, Object> content) {
    for (String line : contents) {
        if (line.equals(HEADER_SEPARATOR)) {
            break;
        }

        if (line.isEmpty()) {
        	continue;
        }
        
        String[] parts = line.split("=",2);
        if (parts.length != 2) {
            continue;
        }

        String key = parts[0].trim();
        String value = parts[1].trim();

        if (key.equalsIgnoreCase(Crawler.Attributes.DATE)) {
            DateFormat df = new SimpleDateFormat(config.getString(Keys.DATE_FORMAT));
            Date date = null;
            try {
                date = df.parse(value);
                content.put(key, date);
            } catch (ParseException e) {
                e.printStackTrace();
            }
        } else if (key.equalsIgnoreCase(Crawler.Attributes.TAGS)) {
            content.put(key, getTags(value));
        } else if (isJson(value)) {
            content.put(key, JSONValue.parse(value));
        } else {
            content.put(key, value);
        }
    }
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:44，代码来源:MarkupEngine.java

示例6: get

import org.jbake.app.Crawler; //导入依赖的package包/类
@Override
public DocumentList get(ContentStore db, Map model, String key) {
    String tag = null;
    if (model.get(Crawler.Attributes.TAG) != null) {
        tag = model.get(Crawler.Attributes.TAG).toString();
    }
    // fetch the tagged documents from db
    return db.getPublishedDocumentsByTag(tag);
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:10，代码来源:TaggedDocumentsExtractor.java

示例7: get

import org.jbake.app.Crawler; //导入依赖的package包/类
@Override
public DocumentList get(ContentStore db, Map model, String key) {
    String tag = null;
    if (model.get(Crawler.Attributes.TAG) != null) {
        tag = model.get(Crawler.Attributes.TAG).toString();
    }
    // fetch the tag posts from db
    return db.getPublishedPostsByTag(tag);
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:10，代码来源:TagPostsExtractor.java

示例8: renderCustomTypePaper

import org.jbake.app.Crawler; //导入依赖的package包/类
@Test
public void renderCustomTypePaper() throws Exception {
    // setup
    config.setProperty("template.paper.file", "paper." + templateExtension);
    DocumentTypes.addDocumentType("paper");
    DBUtil.updateSchema(db);

    Crawler crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    Parser parser = new Parser(config, sourceFolder.getPath());
    Renderer renderer = new Renderer(db, destinationFolder, templateFolder, config);
    String filename = "published-paper.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + "papers" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile);
    for (String string : getOutputStrings("paper")) {
        assertThat(output).contains(string);
    }

}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:28，代码来源:GroovyMarkupTemplateEngineRenderingTest.java

示例9: renderCustomTypePaper

import org.jbake.app.Crawler; //导入依赖的package包/类
@Test
public void renderCustomTypePaper() throws Exception {
    // setup
    config.setProperty("template.paper.file", "paper." + templateExtension);
    DocumentTypes.addDocumentType("paper");
    DBUtil.updateSchema(db);

    Crawler crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    Parser parser = new Parser(config, sourceFolder.getPath());
    Renderer renderer = new Renderer(db, destinationFolder, templateFolder, config);
    String filename = "published-paper.html";

    File sampleFile = new File(sourceFolder.getPath() + File.separator + "content" + File.separator + "papers" + File.separator + filename);
    Map<String, Object> content = parser.processFile(sampleFile);
    content.put(Crawler.Attributes.URI, "/" + filename);
    renderer.render(content);
    File outputFile = new File(destinationFolder, filename);
    Assert.assertTrue(outputFile.exists());

    // verify
    String output = FileUtils.readFileToString(outputFile, Charset.defaultCharset());
    for (String string : getOutputStrings("paper")) {
        assertThat(output).contains(string);
    }

}
开发者ID:jbake-org，项目名称:jbake，代码行数:28，代码来源:GroovyMarkupTemplateEngineRenderingTest.java

示例10: getBody

import org.jbake.app.Crawler; //导入依赖的package包/类
public String getBody() {
    return contents.get(Crawler.Attributes.BODY).toString();
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:4，代码来源:ParserContext.java

示例11: setBody

import org.jbake.app.Crawler; //导入依赖的package包/类
public void setBody(String str) {
    contents.put(Crawler.Attributes.BODY, str);
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:4，代码来源:ParserContext.java

示例12: setup

import org.jbake.app.Crawler; //导入依赖的package包/类
@Before
public void setup() throws Exception {
    currentLocale = Locale.getDefault();
    Locale.setDefault(Locale.ENGLISH);

    ModelExtractorsDocumentTypeListener listener = new ModelExtractorsDocumentTypeListener();
    DocumentTypes.addListener(listener);

    URL sourceUrl = this.getClass().getResource("/");

    sourceFolder = new File(sourceUrl.getFile());
    if (!sourceFolder.exists()) {
        throw new Exception("Cannot find sample data structure!");
    }

    destinationFolder = folder.getRoot();

    templateFolder = new File(sourceFolder, templateDir);
    if (!templateFolder.exists()) {
        throw new Exception("Cannot find template folder!");
    }

    config = ConfigUtil.load(new File(this.getClass().getResource("/").getFile()));
    Iterator<String> keys = config.getKeys();
    while (keys.hasNext()) {
        String key = keys.next();
        if (key.startsWith("template") && key.endsWith(".file")) {
            String old = (String) config.getProperty(key);
            config.setProperty(key, old.substring(0, old.length() - 4) + "." + templateExtension);
        }
    }
    Assert.assertEquals(".html", config.getString(ConfigUtil.Keys.OUTPUT_EXTENSION));
    db = DBUtil.createDataStore("memory", "documents"+System.currentTimeMillis());

    crawler = new Crawler(db, sourceFolder, config);
    crawler.crawl(new File(sourceFolder.getPath() + File.separator + "content"));
    parser = new Parser(config, sourceFolder.getPath());
    renderer = new Renderer(db, destinationFolder, templateFolder, config);

    setupExpectedOutputStrings();
}
开发者ID:ghaseminya，项目名称:jbake-rtl-jalaali，代码行数:42，代码来源:AbstractTemplateEngineRenderingTest.java

示例13: processHeader

import org.jbake.app.Crawler; //导入依赖的package包/类
/**
 * Process the header of the file.
 * @param config 
 *
 * @param contents Contents of file
 * @param content
 */
private void processHeader(Configuration config, List<String> contents, final Map<String, Object> content) {
    for (String line : contents) {
        if (line.equals(HEADER_SEPARATOR)) {
            break;
        }

        if (line.isEmpty()) {
        	continue;
        }
        
        String[] parts = line.split("=",2);
        if (parts.length != 2) {
            continue;
        }

        String utf8BOM = "\uFEFF";
        String key;
        if (parts[0].contains(utf8BOM)) {
        	key = parts[0].trim().replace(utf8BOM, "");
        } else {
        	key = parts[0].trim();
        }
        String value = parts[1].trim();

        if (key.equalsIgnoreCase(Crawler.Attributes.DATE)) {
            DateFormat df = new SimpleDateFormat(config.getString(Keys.DATE_FORMAT));
            Date date = null;
            try {
                date = df.parse(value);
                content.put(key, date);
            } catch (ParseException e) {
                e.printStackTrace();
            }
        } else if (key.equalsIgnoreCase(Crawler.Attributes.TAGS)) {
            content.put(key, getTags(value));
        } else if (isJson(value)) {
            content.put(key, JSONValue.parse(value));
        } else {
            content.put(key, value);
        }
    }
}
开发者ID:jbake-org，项目名称:jbake，代码行数:50，代码来源:MarkupEngine.java

示例14: withDate

import org.jbake.app.Crawler; //导入依赖的package包/类
public FakeDocumentBuilder withDate(Date date) {
	fileModel.put(Crawler.Attributes.DATE, date);
	hasDate = true;
	return this;
}
开发者ID:jbake-org，项目名称:jbake，代码行数:6，代码来源:FakeDocumentBuilder.java

示例15: withCurrentDate

import org.jbake.app.Crawler; //导入依赖的package包/类
private FakeDocumentBuilder withCurrentDate() {
    fileModel.put(Crawler.Attributes.DATE, new Date() );
    return this;
}
开发者ID:jbake-org，项目名称:jbake，代码行数:5，代码来源:FakeDocumentBuilder.java


注：本文中的org.jbake.app.Crawler类示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。