當前位置: 首頁>>代碼示例>>Java>>正文

Java WebClient.getPage方法代碼示例

本文整理匯總了Java中com.gargoylesoftware.htmlunit.WebClient.getPage方法的典型用法代碼示例。如果您正苦於以下問題：Java WebClient.getPage方法的具體用法？Java WebClient.getPage怎麽用？Java WebClient.getPage使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類com.gargoylesoftware.htmlunit.WebClient的用法示例。


在下文中一共展示了WebClient.getPage方法的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: searchDuck

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public static List<String> searchDuck (String keyword) {
    List<String> searchResults = new ArrayList<>();
    try{
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        HtmlPage page = webClient.getPage("https://duckduckgo.com/html/?q=" + keyword);
        List<HtmlAnchor> l = page.getByXPath("//a[@class='result__url']");
        for(HtmlAnchor a: l) {
            searchResults.add(a.getHrefAttribute());
        }


    }
    catch(Exception e){
        System.err.println(e);
    }
    return searchResults;
}
開發者ID:nitroignika，項目名稱:duck-feed-2，代碼行數:18，代碼來源:DuckScrape.java

示例2: getXenToken

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
private String getXenToken(final Account account) { // TODO: Re-write this if it ever gets used
    final WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setJavaScriptEnabled(false);

    final HtmlPage page;
    final HtmlInput token;

    account.getCookies().forEach(c -> webClient.getCookieManager().addCookie(c));

    try {
        page = webClient.getPage(account.getForum().getProtocol() + "://" + account.getForum());
        token = page.getFirstByXPath("//*[@id='XenForo']/body/div[1]/aside[2]/div/div/div[1]/div[2]/form/div/input[2]");

        webClient.close();
        return token.getValueAttribute();
    } catch (Exception e) {
        e.printStackTrace();
    }
    webClient.close();
    return null;
}
開發者ID:Cldfire，項目名稱:Forum-Notifier，代碼行數:23，代碼來源:StatViewController.java

示例3: verifyExampleInClassLevelJavadoc

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
@Test
public void verifyExampleInClassLevelJavadoc() throws Exception {
	Assume.group(TestGroup.PERFORMANCE);

	WebClient webClient = new WebClient();

	MockMvc mockMvc = MockMvcBuilders.standaloneSetup(TestController.class).build();
	MockMvcWebConnection mockConnection = new MockMvcWebConnection(mockMvc);
	mockConnection.setWebClient(webClient);

	WebRequestMatcher cdnMatcher = new UrlRegexRequestMatcher(".*?//code.jquery.com/.*");
	WebConnection httpConnection = new HttpWebConnection(webClient);
	WebConnection webConnection = new DelegatingWebConnection(mockConnection, new DelegateWebConnection(cdnMatcher, httpConnection));

	webClient.setWebConnection(webConnection);

	Page page = webClient.getPage("http://code.jquery.com/jquery-1.11.0.min.js");
	assertThat(page.getWebResponse().getStatusCode(), equalTo(200));
	assertThat(page.getWebResponse().getContentAsString(), not(isEmptyString()));
}
開發者ID:langtianya，項目名稱:spring4-understanding，代碼行數:21，代碼來源:DelegatingWebConnectionTests.java

示例4: main

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public static void main(String[] args) throws IOException {

        // 瀏覽器
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        webClient.getOptions().setUseInsecureSSL(true);//支持https
        webClient.getOptions().setJavaScriptEnabled(true); // 啟用JS解釋器，默認為true
        webClient.getOptions().setCssEnabled(false); // 禁用css支持
        webClient.getOptions().setThrowExceptionOnScriptError(false); // js運行錯誤時，是否拋出異常
        webClient.getOptions().setTimeout(10000); // 設置連接超時時間 ，這裏是10S。如果為0，則無限期等待
        webClient.getOptions().setDoNotTrackEnabled(false);
        webClient.setJavaScriptTimeout(8000);//設置js運行超時時間
        webClient.waitForBackgroundJavaScript(500);//設置頁麵等待js響應時間,

        // proxy
        //webClient.getOptions().setProxyConfig(new ProxyConfig("IP", 80));

        HtmlPage page = webClient.getPage("http://---");
        String pageXml = page.asXml(); //以xml的形式獲取響應文本
        System.out.println(pageXml);

    }
開發者ID:xuxueli，項目名稱:xxl-incubator，代碼行數:22，代碼來源:Demo.java

示例5: completeCloudflareBrowserCheck

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public static Cookie completeCloudflareBrowserCheck(final String url) {
    WebClient completeClient = new WebClient(BrowserVersion.CHROME);
    completeClient.getOptions().setCssEnabled(false);
    completeClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

    final HtmlPage page;
    final HtmlElement submitButton;
    final HtmlForm challengeForm;

    try {
        page = completeClient.getPage(url);
        completeClient.waitForBackgroundJavaScript(5000);

        submitButton = (HtmlElement) page.createElement("button");
        submitButton.setAttribute("type", "submit");

        challengeForm = (HtmlForm) page.getElementById("challenge-form");
        challengeForm.appendChild(submitButton);
        submitButton.click();

        return completeClient.getCookieManager().getCookie("cf_clearance");
    } catch (Exception e) {
        e.printStackTrace();
        return null;
    }
}
開發者ID:Cldfire，項目名稱:Forum-Notifier，代碼行數:27，代碼來源:LoginViewController.java

示例6: getHtmlPageHtmlUnit

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
/**
 * HtmlUnit을 이용한 HTML 코드 파싱.
 * @param eachArchiveAddress 실제 만화가 담긴 아카이브 주소
 * @return 성공 시 html 코드를 리턴
 */
private String getHtmlPageHtmlUnit(String eachArchiveAddress) throws Exception {
	/* 필수! 로그 메세지 출력 안함 -> HtmlUnit 이용시 Verbose한 로그들이 너무 많아서 다 끔 */
	java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF); 
	System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
	
	System.out.print("일반 연결 시도중 ... ");
	
	WebClient webClient = new WebClient();
	webClient.getOptions().setRedirectEnabled(true);
	
	WebRequest req = new WebRequest(new URL(eachArchiveAddress));
	req.setHttpMethod(HttpMethod.POST);
	req.setAdditionalHeader("User-Agent", UserAgent.getUserAgent());
	req.setAdditionalHeader("Accept-Encoding", "gzip"); //20171126 gzip 추가
	req.getRequestParameters().add(new NameValuePair("pass", PASSWORD)); //비밀번호 post 방식 전송
	
	HtmlPage page = webClient.getPage(req);
	
	//Html코드를 포함한 페이지 소스코드가 담길 스트링
	String pageSource = page.asXml();
	
	/** 여기도 페이지 파싱 실패 시 검증하는 코드 들어가야 됨 **/
	
	webClient.close();
	System.out.println("성공");
	return pageSource;
}
開發者ID:occidere，項目名稱:MMDownloader，代碼行數:33，代碼來源:Downloader.java

示例7: navigateToPage

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
private HtmlPage navigateToPage( User user, String uri, boolean enableJS ) throws Exception, IOException, MalformedURLException
{
    final WebClient webClient = new WebClient();
    webClient.getOptions().setCssEnabled( false );
    webClient.getOptions().setJavaScriptEnabled( enableJS );
    addCredentials( user, webClient );

    String pageUrl = new StringBuilder( "http://" ).append( WEBSITE ).append( uri ).toString();
    HtmlPage page = webClient.getPage( pageUrl );
    return page;
}
開發者ID:Vedang18，項目名稱:ProBOT，代碼行數:12，代碼來源:Bookie.java

示例8: cancelBooking

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public void cancelBooking( User user, Meeting meeting ) throws Exception
{
    final WebClient webClient = new WebClient();
    addCredentials( user, webClient );

    String uri = CANCEL_BOOKING + "/" + meeting.getMeetingId();
    String pageUrl = new StringBuilder( "http://" ).append( WEBSITE ).append( uri ).toString();

    URL url = new URL( pageUrl );
    WebRequest requestSettings = new WebRequest( url, HttpMethod.POST );

    Page redirectPage = webClient.getPage( requestSettings );
    logger.debug( "Confirmed Cancellation " + redirectPage.getWebResponse().getContentAsString() );
}
開發者ID:Vedang18，項目名稱:ProBOT，代碼行數:15，代碼來源:Bookie.java

示例9: main

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
    
    // 屏蔽HtmlUnit等係統 log
    LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log","org.apache.commons.logging.impl.NoOpLog");
    java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
    java.util.logging.Logger.getLogger("org.apache.http.client").setLevel(Level.OFF);
    
    String url = "https://www.douyin.com/share/video/6496703951436516621/?mid=6484356820260686606";
    System.out.println("Loading page now-----------------------------------------------: "+url);
    
    /* HtmlUnit 模擬瀏覽器 */
    WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setJavaScriptEnabled(true);              // 啟用JS解釋器，默認為true  
    webClient.getOptions().setCssEnabled(false);                    // 禁用css支持  
    webClient.getOptions().setThrowExceptionOnScriptError(false);   // js運行錯誤時，是否拋出異常
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setTimeout(10 * 1000);                   // 設置連接超時時間
    HtmlPage page = webClient.getPage(url);
    webClient.waitForBackgroundJavaScript(30 * 1000);               // 等待js後台執行30秒

    String pageAsXml = page.asXml();
    
    /* Jsoup解析處理 */
    // Document doc = Jsoup.parse(pageAsXml, "https://bluetata.com/");
    Document doc = Jsoup.parse(pageAsXml);  
    Elements pngs = doc.select("img[src$=.png]");                   // 獲取所有圖片元素集
    // 其他操作
    System.out.println(doc.toString());
}
開發者ID:bluetata，項目名稱:crawler-jsoup-maven，代碼行數:30，代碼來源:htmlunitTest.java

示例10: create

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
/**
 * Starts the CSV link creation process.
 * 
 * @throws Exception If there was an error while getting the CSV specific information from the forum thread.
 */
public void create() throws Exception{
	java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(java.util.logging.Level.OFF);
	
	WebClient webClient = new WebClient(BrowserVersion.CHROME);
    HtmlPage page = (HtmlPage) webClient.getPage(forumThreadLink);
    webClient.waitForBackgroundJavaScriptStartingBefore(waitForJavaScript);
    HtmlAnchor anchor = (HtmlAnchor) page.getByXPath(xPath).get(0);
    csvFileLink = anchor.getHrefAttribute();
    csvFileLink = "http://www.pathofexile.com" + csvFileLink.subSequence(0, csvFileLink.length()-1);
    webClient.close();  
}
開發者ID:jkjoschua，項目名稱:poe-ladder-tracker-java，代碼行數:17，代碼來源:CSVLinkCreator.java

示例11: getPageWhenUrlIsRelativeAndNoPortWillUseLocalhost8080

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
@Test
public void getPageWhenUrlIsRelativeAndNoPortWillUseLocalhost8080() throws Exception {
	MockEnvironment environment = new MockEnvironment();
	WebClient client = new LocalHostWebClient(environment);
	WebConnection connection = mockConnection();
	client.setWebConnection(connection);
	client.getPage("/test");
	verify(connection).getResponse(this.requestCaptor.capture());
	assertThat(this.requestCaptor.getValue().getUrl())
			.isEqualTo(new URL("http://localhost:8080/test"));
}
開發者ID:vikrammane23，項目名稱:https-github.com-g0t4-jenkins2-course-spring-boot，代碼行數:12，代碼來源:LocalHostWebClientTests.java

示例12: getPageWhenUrlIsRelativeAndHasPortWillUseLocalhostPort

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
@Test
public void getPageWhenUrlIsRelativeAndHasPortWillUseLocalhostPort()
		throws Exception {
	MockEnvironment environment = new MockEnvironment();
	environment.setProperty("local.server.port", "8181");
	WebClient client = new LocalHostWebClient(environment);
	WebConnection connection = mockConnection();
	client.setWebConnection(connection);
	client.getPage("/test");
	verify(connection).getResponse(this.requestCaptor.capture());
	assertThat(this.requestCaptor.getValue().getUrl())
			.isEqualTo(new URL("http://localhost:8181/test"));
}
開發者ID:vikrammane23，項目名稱:https-github.com-g0t4-jenkins2-course-spring-boot，代碼行數:14，代碼來源:LocalHostWebClientTests.java

示例13: getNextUrl

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
/**
 * 通過htmlunit來獲得一些搜狗的網址。
 * @param key
 * @return
 * @throws Exception
 */
public String getNextUrl(String key){
	String page = new String();
	try {
		WebClient webClient = new WebClient();
		webClient.getOptions().setCssEnabled(false);
		webClient.getOptions().setJavaScriptEnabled(false);
		
		//去拿網頁
		HtmlPage htmlPage = webClient.getPage("http://pic.sogou.com/");
		//得到表單
		HtmlForm form = htmlPage.getFormByName("searchForm");
		//得到提交按鈕
		HtmlSubmitInput button = form.getInputByValue("搜狗搜索");
		//得到輸入框

		HtmlTextInput textField = form.getInputByName("query");
		//輸入內容
		textField.setValueAttribute(key);
		//點一下按鈕
		HtmlPage nextPage = button.click();

		String str = nextPage.toString();
		page = cutString(str);
		webClient.close();
	} catch (Exception e) {
		// TODO: handle exception
	} finally{
		
	}
	System.out.println();
	return page;
	
}
開發者ID:anLA7856，項目名稱:Crawler，代碼行數:40，代碼來源:CrawUrl.java

示例14: testGetToken

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public void testGetToken() throws FailingHttpStatusCodeException, IOException {
	WebClient wc = weiWS.getInstance();
	WebRequest wr = new WebRequest(new URL("https://mp.weixin.qq.com/cgi-bin/login"), HttpMethod.POST);
	wr.getAdditionalHeaders().put("Accept", "*/*");
	wr.getAdditionalHeaders().put("Accept-Encoding", "gzip, deflate, br");
	wr.getAdditionalHeaders().put("Accept-Language", "en-US,en;q=0.5");
	wr.getAdditionalHeaders().put("Connection", "keep-alive");
	//		wr.getAdditionalHeaders().put("Content-Length", "86");
	wr.getAdditionalHeaders().put("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
	wr.getAdditionalHeaders().put("Host", "mp.weixin.qq.com");
	wr.getAdditionalHeaders().put("Referer", "https://mp.weixin.qq.com/");
	wr.getAdditionalHeaders().put("User-Agent",
			"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0");
	wr.getAdditionalHeaders().put("X-Requested-With", "XMLHttpRequest");
	List<NameValuePair> params = new ArrayList<NameValuePair>();
	params.add(new NameValuePair("f", "json"));
	params.add(new NameValuePair("imgcode", ""));
	//3BCEBC34A480F472E27372F9DE2D5592
	params.add(new NameValuePair("pwd", "3bcebc34a480f472e27372f9de2d5592"));
	params.add(new NameValuePair("username", "[email protected]"));
	wr.setRequestParameters(params);
	Page page = wc.getPage(wr);
	String response = page.getWebResponse().getContentAsString();
	System.out.println(response);
	Set<Cookie> cookies = weiWS.getInstance().getCookieManager().getCookies();
	for (Cookie cookie : cookies) {
		System.out.println(cookie.toString());
	}

	Gson gson = new Gson();
	RespWithTokenJSON jsonObj = gson.fromJson(response, RespWithTokenJSON.class);
	String homePath = jsonObj.getRedirect_url();

	HtmlPage homePage = wc.getPage("https://mp.weixin.qq.com/" + homePath);
	HtmlAnchor logout = (HtmlAnchor) homePage.getElementById("logout");
	System.out.println(logout);
	HtmlPage loginPage = wc.getPage("https://mp.weixin.qq.com");
	System.out.println(loginPage.getUrl());
}
開發者ID:alexmao86，項目名稱:weixinmp4j，代碼行數:40，代碼來源:WeixinmpWebSessionTest.java

示例15: main

import com.gargoylesoftware.htmlunit.WebClient; //導入方法依賴的package包/類
public static void main(String[] args) throws Exception{

		WebClient webClient = HtmlUnitUtil.create();
		System.out.println(DateFormatUtils.format(new Date(), "yyyyMMddHHmmss"));
		for(int i=0;i<10;i++){
			webClient.getPage("https://detail.m.tmall.com/item.htm?spm=0.0.0.0.5FpUFm&id=543557417955&abtest=21&rn=bbe5ec2e6fea4193aa04dbb1b8754dfd&sid=eb80b81938bb970e9c61caa81b4baf51");
			System.out.println("當前已刷>>>>>>>>>>>>>>>>>"+i);
		}
		System.out.println(DateFormatUtils.format(new Date(), "yyyyMMddHHmmss"));
	}
開發者ID:xiaomin0322，項目名稱:alimama，代碼行數:11，代碼來源:ShuaOline3.java


注：本文中的com.gargoylesoftware.htmlunit.WebClient.getPage方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。