当前位置: 首页>>代码示例>>PHP>>正文


PHP Crawler::run方法代码示例

本文整理汇总了PHP中Crawler::run方法的典型用法代码示例。如果您正苦于以下问题:PHP Crawler::run方法的具体用法?PHP Crawler::run怎么用?PHP Crawler::run使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在Crawler的用法示例。


在下文中一共展示了Crawler::run方法的3个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的PHP代码示例。

示例1: Crawler

<?php

require_once '../vendor/autoload.php';
require_once 'crawler.php';
require_once 'CKMoz.php';
require_once 'auth/auth.php';
$startURL = $_POST['url'];
$urls = [];
$crawler = new Crawler($startURL, 3);
$urls = $crawler->run();
var_dump($urls);
die;
$encoded = json_encode($urls);
header('Content-type: application/json');
exit($encoded);
if (!is_array($urls)) {
    throw new Exception("There's no URL's to make the consult.");
}
$groups = array_chunk($urls, 10);
$metricQ = new CKMoz(ACCESS_ID, SECRET_KEY);
$cols = array('title', 'canonURL', 'ExEquityLinks', 'links', 'mozRankURL', 'mozRankSubDomain', 'httpCode', 'pageAuth', 'domainAuth');
$result = array();
foreach ($groups as $group) {
    var_dump($group);
    exit;
    $rs = $metricQ->batchedQuery($group, $cols);
    // if the result is an error of authentication
    if ($rs->code == 401) {
        header('Content-type: application/json');
        exit($rs->data);
    }
开发者ID:alfchee,项目名称:TestDomPDF,代码行数:31,代码来源:CrawlerController.php

示例2: Crawler

<?php

require 'class/Crawler.php';
require 'class/Utils.php';
$crawler = new Crawler();
$crawler->run();
开发者ID:hadikcz,项目名称:PrntscrRandomDownloader,代码行数:6,代码来源:run.php

示例3: jsReload

     print_r($districts);
     jsReload();
     exit;
 } else {
     //Crawl wards
     foreach ($province['districts'] as $districtId => $district) {
         $excerptDistricts = ['318' => 1, 471 => 1, 498 => 1, 755 => 1];
         if (!empty($excerptDistricts[$districtId])) {
             continue;
         }
         if (!isset($district['wards']) || count($district['wards']) < 1) {
             echo '<pre>';
             echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />';
             $url = 'http://www.gso.gov.vn/danhmuc/dmhc.aspx?macap2=' . $districtId;
             $crawler = new Crawler($url);
             $listHtml = $crawler->run();
             $pattern = '/<td align=\'left\' ><font color=\'blue\'>([0-9]{5})<\\/a><\\/td><td align=\'left\' id=\'link\' ><font color=\'blue\'>([^<>]*)<\\/td>/';
             preg_match_all($pattern, $listHtml, $matches);
             //    print_r($matches);
             $wards = [];
             foreach ($matches[1] as $k => $wardId) {
                 $wards[$wardId] = ['name' => $matches[2][$k]];
             }
             $_SESSION['provinceList'][$provinceId]['districts'][$districtId]['wards'] = $wards;
             echo 'Crawled ', htmlentities($province['name']), ' -> ', htmlentities($district['name']), '<br>';
             if (count($wards) < 1) {
                 echo "<br>Da bi chan boi site gov. Reload lai sau 5'</br>";
                 jsReload(120000);
             }
             print_r($wards);
             jsReload();
开发者ID:vantrieuou,项目名称:vietnam-administrative-units,代码行数:31,代码来源:index.php


注:本文中的Crawler::run方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。