使用jsoup从html中获取url

•浏览 1

Fetch urls from html with jsoup

我正在尝试使用 jsoup 获取一个 url，以便从该 url 下载一个图像，但由于某种原因它不起作用。

我首先尝试找到"div "第一次出现在 html 文件中的位置，

而不是获取紧随其后的网址：

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

这是 html 的 url:

查看来源：https://www.google.co.il/search?q=烤墨西哥鸡

我编辑了您的代码以消除错误 403

而不是这个：

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

写这个：

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

似乎链接是动态生成的。 Jsoup 获取不包含 .rg_di 类的 html，因此

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

返回null，我们得到nullpointerexception。

jsoup下载的html片段

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

我们能做的最好的就是获取每个 img 标签并遍历它们，我们得到图标链接列表

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

解析动态内容有很多解决方案。链接

编辑 1

我实现了 htmlunit 来渲染一个页面

a href="http://www.google.co.il/imgres?imgurl=http://michellepicker.files.wordpress.com/2011/03/grilled-chicken-mexican-style.jpg&imgrefurl=http://michellepicker.wordpress.com/2011/04/25/grilled-chicken-mexican-style-black-beans-guacamole/&h=522&w=700&tbnid=4hXCtCfljxmJXM:&zoom=1&docid=ajIrwZMUrP5_GM&ei=iVOqVPmDDYrnaJzYgIAM&tbm=isch"
doc = Jsoup.connect(url).get();
doc = Jsoup.connect(url).userAgent("Mozilla").get();
doc.select("div.rg_di").first();

<img height="104" src="https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT" width="140">

Document doc = Jsoup.connect(url).userAgent("Mozilla").get();

Elements imgs = doc.select("img");

for(Element img : imgs){

  String link = img.attr("src");

  System.out.println(link);

}



/textinputassistant/tia.png

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcT-pctOxpuUcdq118aFU3s2miRfUa6Ev8eF-UsxARHV-vbcOUV8byEtt2YT

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQMq354p43ddqPcpV9-q_05YkmY7XUPgv6Sl2oQLqFxQ5-IkpGAAuFTLMM

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTW-RinkkW_fBdlHzTJn6vNmR85TR58geQgfjQnEJmOqzjq0Oi-z-8zXjg

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRUXLzKi3UyQ6mF9JD20Z1jYNhVxQz7tkhJIEGOL3kua8ptoQrvo8-Nco_X

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTverQlzF_hauCabscWF4wHLb_q7g9M_UDKO6LaldSRHhsTj7CxtVF2yvc

...
import java.io.IOException;

import java.net.MalformedURLException;



import org.jsoup.Jsoup;

import org.jsoup.nodes.Document;

import org.jsoup.nodes.Element;

import org.jsoup.select.Elements;



import com.gargoylesoftware.htmlunit.BrowserVersion;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlPage;





public class Main {

  public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url ="https://www.google.co.il/search?q=grilled+mexican+chicken&es_sm=93&source=lnms&tbm=isch&sa=X&ei=h1OqVOH6B5bjaqGogvAP&ved=0CAgQ_AUoAQ&biw=1920&bih=955";

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); // Chrome not working

    HtmlPage page = webClient.getPage(url); 

    try {

      Document doc = Jsoup.parse(page.asXml());

      Elements divs = doc.select(".rg_di");

      for(Element div : divs){

        Element img = div.select("a").get(0);

        String link = img.attr("href");

        System.out.println(link);

      }

    } catch (Exception e) {

      e.printStackTrace();

    }

  }

}

htmlunit 有自己的 html 解析 api，但我会坚持使用更直观的 jsoup

编辑 2

只要你的目标是在 Android 设备上渲染和解析 HTML 页面，HTMLUnit 就不是一个好的选择 source

HtmlUnit uses Java classes that are not available on Android.

On top of that, HtmlUnit uses a bunch of other libraries, some of which may have their own dependencies on these libraries. So, as awesome as HmlUnit is, I think getting it to run on Android may not be an easy task.

  • 您可以尝试这种解决方案。或者
  • 您可以折磨自己并尝试此解决方案(最好不要)。或者
  • 如果考虑到这个人的经验，重新设计你的软件架构会更好：
  • 创建呈现网页并解析它的java服务器。 HTMLUnit Jsoup
  • 将解析后的数据以 JSON 格式保存在服务器的文件系统中。格森
  • 创建 servlet，当 android 应用程序请求它时发送 JSON 文件。
赞 (0)
« 上一篇
下一篇 »

相关推荐

  • Spring部署设置openshift

    Springdeploymentsettingsopenshift我有一个问题让我抓狂了三天。我根据OpenShift帐户上的教程部署了spring-eap6-quickstart代码。我已配置调试选项，并且已将Eclipse工作区与OpehShift服务器同步-服务器上的一切工作正常，但在Eclipse中出现无法消除的错误。我有这个错误：cvc-complex-type.2.4.a:Invali…
    2025-04-161

  • 检查Java中正则表达式中模式的第n次出现

    CheckfornthoccurrenceofpatterninregularexpressioninJava本问题已经有最佳答案，请猛点这里访问。我想使用Java正则表达式检查输入字符串中特定模式的第n次出现。你能建议怎么做吗？这应该可以工作：MatchResultfindNthOccurance(intn,Patternp,CharSequencesrc){Matcherm=p.matcher…
    2025-04-161

  • 如何让 JTable 停留在已编辑的单元格上

    HowtohaveJTablestayingontheeditedcell如果有人编辑JTable的单元格内容并按Enter，则内容会被修改并且表格选择会移动到下一行。是否可以禁止JTable在单元格编辑后转到下一行？原因是我的程序使用ListSelectionListener在单元格选择上同步了其他一些小部件，并且我不想在编辑当前单元格后选择下一行。Enter的默认绑定是名为selectNext…
    2025-04-161

  • Weblogic 12c 部署

    Weblogic12cdeploy我正在尝试将我的应用程序从Tomcat迁移到Weblogic12.2.1.3.0。我能够毫无错误地部署应用程序，但我遇到了与持久性提供程序相关的运行时错误。这是堆栈跟踪：javax.validation.ValidationException:CalltoTraversableResolver.isReachable()threwanexceptionatorg.…
    2025-04-161

  • Resteasy Content-Type 默认值

    ResteasyContent-Typedefaults我正在使用Resteasy编写一个可以返回JSON和XML的应用程序，但可以选择默认为XML。这是我的方法：@GET@Path("/content")@Produces({MediaType.APPLICATION_XML,MediaType.APPLICATION_JSON})publicStringcontentListRequestXm…
    2025-04-161

  • 代码不会停止运行，在 Java 中

    thecodedoesn'tstoprunning,inJava我正在用Java解决项目Euler中的问题10，即"Thesumoftheprimesbelow10is2+3+5+7=17.Findthesumofalltheprimesbelowtwomillion."我的代码是packageprojecteuler_1;importjava.math.BigInteger;importjava…
    2025-04-161

  • Out of memory java heap space

    Outofmemoryjavaheapspace我正在尝试将大量文件从服务器发送到多个客户端。当我尝试发送大小为700mb的文件时，它显示了"OutOfMemoryjavaheapspace"错误。我正在使用Netbeans7.1.2版本。我还在属性中尝试了VMoption。但仍然发生同样的错误。我认为阅读整个文件存在一些问题。下面的代码最多可用于300mb。请给我一些建议。提前致谢publicc…
    2025-04-161

  • Log4j 记录到共享日志文件

    Log4jLoggingtoaSharedLogFile有没有办法将log4j日志记录事件写入也被其他应用程序写入的日志文件。其他应用程序可以是非Java应用程序。有什么缺点？锁定问题？格式化？Log4j有一个SocketAppender，它将向服务发送事件，您可以自己实现或使用与Log4j捆绑的简单实现。它还支持syslogd和Windows事件日志，这对于尝试将日志输出与来自非Java应用程序…
    2025-04-161
手机版技术脚本