htmlparser-user Mailing List for HTML Parser

Unknown tags are returned as generic tags.
The code that does this is in PrototypicalNodefactory.createTagNode().
If you create a class that implements the NodeFactory interface you can substitute it for the default PrototypicalNodefactory and determine which nodes are not in the list of known tags or handle it any way you want.
You can probably just create a class derived from PrototypicalNodefactory  and override createTagNode ().
Then set it on your parser with setNodeFactory().

----- Original Message ----
From: Binod Dokania <bdo...@is...>
To: htm...@li...
Sent: Monday, March 24, 2008 12:23:40 AM
Subject: [Htmlparser-user] HTML validator for html parser

  
<!--

 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;margin-bottom:.0001pt;font-size:12.0pt;font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;text-decoration:underline;}
span.EmailStyle17
	{font-family:Arial;color:windowtext;}
 _filtered {margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{}
-->
Hi,
   
 I have a query regarding using HTML parser.
 Does html parser provide any html validator before start
parsing a html document with HTML parser?
   
 I have some html documents created using different html
development tools.
 These html documents create different html tags.
 So If we can get html validator which provides the
information about the lines/tags not recognized by html parser.
   
 Any ideas/suggestion on this would be appreciated.
   
 Thanks in advance,
   
 Warm Regards,
 Binod Dokania
   
 


-----Inline Attachment Follows-----

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/


-----Inline Attachment Follows-----

_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user

Hi,

 

I have a query regarding using HTML parser.

Does html parser provide any html validator before start parsing a html
document with HTML parser?

 

I have some html documents created using different html development tools.

These html documents create different html tags.

So If we can get html validator which provides the information about the
lines/tags not recognized by html parser.

 

Any ideas/suggestion on this would be appreciated.

 

Thanks in advance,

 

Warm Regards,

Binod Dokania

 

If you have the tag, it's just
  tag.setImageURL (String url);

The trick is to get the output again.
You probably want a list of everything, then filter to extract your particular IMG tag, fix it, and then call toHtml() on everything.

  NodeList list = parser.parse(null);
  NodeList images = list.extractAllNodesThatMatch(<my special filter>);
  images.elementAt(0).setImageURL ("whatever");
  System.out.println (list.toHtml ());

----- Original Message ----
From: Narindra Jeethan <Nar...@te...>
To: htmlparser user list <htm...@li...>
Sent: Wednesday, March 19, 2008 10:57:45 AM
Subject: [Htmlparser-user] editing tags

 Hi,
 
How do I edit the 
src attribute in img tag within an html file?
 
Thanks,
 
Narindra


-----Inline Attachment Follows-----

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/


-----Inline Attachment Follows-----

_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user

Hi Martin,

You're absolutely correct -- it's because of the incorrect closing (at least
that's what it seems to be), and it occurs multiple times.

Thanks for letting me know -- I ended up writing a quick regex script to
extract this stuff, so it works in its own special way, now. :)

Thanks again,
Wojciech

On Mon, Mar 17, 2008 at 5:41 AM, Martin Sturm <mst...@gm...> wrote:

> 2008/3/11, Wojciech Gryc <woj...@gm...>:
>
> > Specifically, the link appears in the page, but surrounds an image, like
> so:
> >
> > <p><a href="http://feeds.feedburner.com/mydd"><img
> > src="..."></a>
> >
> > When I use a basic tag name filter, the actual tag above doesn't get
> > returned (while other <a> tag links do)... I've been playing around with
> the
> > code and don't know where to go from here. Is it because it surrounds an
> > image? Is there anything I can do to fix this?
>
> If it is only a problem with this particular piece of HTML, then it
> could be useful to post it verbatim. Looking to the part you have in
> your e-mail, it seems that the img-tag is not properly closed (it
> should be <img src="..." /> if it is a XHTML document, that is).
> However, I'm not sure if HTMLParser just ignores this... from previous
> expieriences, I noted that HTMLParser is pretty strict when it comes
> to standard compliance.
>
> --
> Martin Sturm
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>



-- 

Five Minutes to Midnight:
Youth on human rights and current affairs
http://www.fiveminutestomidnight.org/

Hi,
 
How do I edit the src attribute in img tag within an html file?
 
Thanks,
 
Narindra

Hi Henry,
 
Try this:
 
public static String getKeywords(String file){
    try {
    Parser parser = new Parser (file);
    NodeList list = parser.parse (new TagNameFilter ("table"));
    System.out.println(list.toHtml());
    } catch (Exception pe) {
      pe.printStackTrace ();
  }
  return Keywords;
 }
 
Narindra Jeethan
Office: 780.493.7211
Mobile: 780.288.5961
 

________________________________

From: htm...@li... [mailto:htm...@li...] On Behalf Of Henry Tran
Sent: Wednesday, March 19, 2008 12:36 AM
To: Htm...@li...
Subject: [Htmlparser-user] How to extract the content of certain html tableonly



Hi,

 

I would like to read the content of all the tables from a web page using HTML Parser. Below is an example of what make up a html table:

 

</table>

    <tr>

    <td class="propType"><b>Address</b></td>
    <td class="propType"><b>Company</b></td>
    <td class="propType"><b>Department</b></td>
    <td class="propType" align="right"><b>Employee</b></td>
    <td colspan="6"><strong class="propType">
    <td><strong>Firstname</strong></td>
    <td><strong>Surname</strong></td>
    <td><strong>DOB</strong></td>
    <td><strong>Sex</strong></td>
    <td class="even">John</td>
    <td class="even">Smith</td>
     <td class="even">01/02/2001</td>
    <td class="even">Male</td>
  </tr>
</table>

 

I am using the following example provided in html parser filter page but still not quite get there just yet:

 

 1 import java.io.*;
 2 import java.net.*;
 3 import org.htmlparser.*;
 4 import org.htmlparser.filters.TagNameFilter;
 5 import org.htmlparser.filters.NodeClassFilter;
 6 import org.htmlparser.filters.HasParentFilter;
 7 import org.htmlparser.filters.*;
 8 import org.htmlparser.util.*;
 9 
 10 public class DnldURL {
 11    public static void main (String[] args) throws ParserException {
 12        DnldURL dnldURL = new DnldURL();
 13    }
 14    public DnldURL() throws ParserException {
 15       try {
 16          Parser parser = new Parser ("http://www.abc.com");
 17          parser.parse (new HasParentFilter());
 18          NodeList list = new NodeList();
 19          NodeFilter filter = new OrFilter(
 20                         new TagNameFilter ("table"),
 21                         new HasChildFilter(
 22                         new TagNameFilter("tr")));     
 23          for (NodeIterator e = parser.elements(); e.hasMoreNodes(); )
 24    //        System.out.println(e.nextNode().toHtml());
 25              System.out.println(e.nextNode().collectInto(list, filter);
 26       } catch (MalformedURLException mue) {
 27          System.out.println("Ouch - a MalformedURLException ha2pened.");
 28          mue.printStackTrace();
 29         
 System.exit(1);
 30       } catch (IOException ioe) {
 31          System.out.println("Oops- an IOException happened.");
 32          ioe.printStackTrace();
 33          System.exit(1);
 34       } 
 35    }

   

The important thing is to get lines 17, 19-22 correctly set up so that the filter could pick up the content and printed on line 25.

 

Not only am I confused on how to set up the table filter dependencies (<table> ...<tr> ...<td>...) but also how to get line 25 to combine both the filter and to.Html() together.

 

For instance:

    System.out.println(e.nextNode().collectInto(list, filter).toHtml()); which doesn't work currently.
I also would like to set up some dependency on what the content of <table>, <tr> and <td> should be so that only those relevant tables are being retrieved as opposed to all the tables.
Many thanks,
Jack

 

 


________________________________

Get the name you always wanted with the new y7mail email address <http://au.rd.yahoo.com/mail/taglines/au/y7mail/default/*http://au.yahoo.com/y7mail/?p1=ni&p2=general&p3=tagline&p4=other> . 

Hi,
 
I would like to read the content of all the tables from a web page using HTML Parser. Below is an example of what make up a html table:
 
</table>
    <tr>
    <td class="propType"><b>Address</b></td>
    <td class="propType"><b>Company</b></td>
    <td class="propType"><b>Department</b></td>
    <td class="propType" align="right"><b>Employee</b></td>
    <td colspan="6"><strong class="propType">
    <td><strong>Firstname</strong></td>
    <td><strong>Surname</strong></td>
    <td><strong>DOB</strong></td>
    <td><strong>Sex</strong></td>
    <td class="even">John</td>
    <td class="even">Smith</td>
     <td class="even">01/02/2001</td>
    <td class="even">Male</td>
  </tr>
</table>
 
I am using the following example provided in html parser filter page but still not quite get there just yet:
 
 1 import java.io.*;
 2 import java.net.*;
 3 import org.htmlparser.*;
 4 import org.htmlparser.filters.TagNameFilter;
 5 import org.htmlparser.filters.NodeClassFilter;
 6 import org.htmlparser.filters.HasParentFilter;
 7 import org.htmlparser.filters.*;
 8 import org.htmlparser.util.*;
 9 
 10 public class DnldURL {
 11    public static void main (String[] args) throws ParserException {
 12        DnldURL dnldURL = new DnldURL();
 13    }
 14    public DnldURL() throws ParserException {
 15       try {
 16          Parser parser = new Parser (“http://www.abc.com”);
 17          parser.parse (new HasParentFilter());
 18          NodeList list = new NodeList();
 19          NodeFilter filter = new OrFilter(
 20                         new TagNameFilter ("table"),
 21                         new HasChildFilter(
 22                         new TagNameFilter("tr")));     
 23          for (NodeIterator e = parser.elements(); e.hasMoreNodes(); )
 24    //        System.out.println(e.nextNode().toHtml());
 25              System.out.println(e.nextNode().collectInto(list, filter);
 26       } catch (MalformedURLException mue) {
 27          System.out.println("Ouch - a MalformedURLException ha2pened.");
 28          mue.printStackTrace();
 29          System.exit(1);
 30       } catch (IOException ioe) {
 31          System.out.println("Oops- an IOException happened.");
 32          ioe.printStackTrace();
 33          System.exit(1);
 34       } 
 35    }
   
The important thing is to get lines 17, 19-22 correctly set up so that the filter could pick up the content and printed on line 25.
 
Not only am I confused on how to set up the table filter dependencies (<table> …<tr> …<td>…) but also how to get line 25 to combine both the filter and to.Html() together.
 
For instance:
    System.out.println(e.nextNode().collectInto(list, filter).toHtml()); which doesn’t work currently.
I also would like to set up some dependency on what the content of <table>, <tr> and <td> should be so that only those relevant tables are being retrieved as opposed to all the tables.

Many thanks,

Jack
 
 


      Get the name you always wanted with the new y7mail email address.
www.yahoo7.com.au/y7mail

You would use a filter to get the title tag like TagNameFilter ("TITLE") and then ask for the plain text from every node (there's probably only one) in the list of filtered tags, like tag.toPlainText ().

----- Original Message ----
From: Narindra Jeethan <Nar...@te...>
To: htm...@li...
Sent: Tuesday, March 18, 2008 4:48:54 PM
Subject: [Htmlparser-user] Using Filter

 Hi,
 
How do I use the 
filter to return the values between tags?
 
For example I want 
to get test from the title tag
 
<html>
<head>
    
<title>test</title>
</head>
<body>body test</body>
</html>
 
Thanks,
 
Narindra Jeethan
Office: 
780.493.7211
Mobile: 
780.288.5961
 


-----Inline Attachment Follows-----

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/


-----Inline Attachment Follows-----

_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user

Hi,
 
How do I use the filter to return the values between tags?
 
For example I want to get test from the title tag
 
<html>
<head>
    <title>test</title>
</head>
<body>body test</body>
</html>
 
Thanks,
 
Narindra Jeethan
Office: 780.493.7211
Mobile: 780.288.5961
 

2008/3/11, Wojciech Gryc <woj...@gm...>:

> Specifically, the link appears in the page, but surrounds an image, like so:
>
> <p><a href="http://feeds.feedburner.com/mydd"><img
> src="..."></a>
>
> When I use a basic tag name filter, the actual tag above doesn't get
> returned (while other <a> tag links do)... I've been playing around with the
> code and don't know where to go from here. Is it because it surrounds an
> image? Is there anything I can do to fix this?

If it is only a problem with this particular piece of HTML, then it
could be useful to post it verbatim. Looking to the part you have in
your e-mail, it seems that the img-tag is not properly closed (it
should be <img src="..." /> if it is a XHTML document, that is).
However, I'm not sure if HTMLParser just ignores this... from previous
expieriences, I noted that HTMLParser is pretty strict when it comes
to standard compliance.

--
Martin Sturm

Was able to fix the problem using this code, pulled from the
extractAllNodesThatMatch method itself:

=========================
NodeIterator e;
	        for (e = parser.elements (); e.hasMoreNodes (); ) {
	        	Node currentNode = e.nextNode();
	        	currentNode.collectInto(titleList, titleFilter);
	        	currentNode.collectInto(summaryTableList, summaryTableFilter);
	        }
===================

-Daniel

On Tue, Mar 11, 2008 at 9:03 AM, Daniel Dixon <me...@cr...> wrote:
> Hello,
>
>  Anyone know why I can't use two extractAllNodesThatMatch(filter)
>  methods back-to-back on the same Parser instance?
>
>  More specifically I have this code:
>
>  ========================================
>  Parser parser = new Parser(google);
>
>  NodeList titleList = parser.extractAllNodesThatMatch(titleFilter);
>  NodeList summaryTableList = parser.extractAllNodesThatMatch(summaryTableFilter);
>  ========================================
>
>  The Google search results page I'm parsing has a series of these:
>
>  <a href="blah">Title</a>
>  <table><tr><td>.....Summary info....</td></tr></table>
>
>  The two filters above, when independent, work fine.  Run them
>  back-to-back and the second will come up empty.  I don't see where the
>  extractAllNodesThatMatch method literally pulls the nodes out of the
>  captured source, thus affecting the second filter. Here are my
>  filters:
>
>  ========================================
>  // filter to pull out titles (all links that are next to a table)
>  NodeFilter titleFilter = new AndFilter (
>                 new NodeClassFilter (LinkTag.class),
>                 new HasSiblingFilter (new NodeClassFilter(TableTag.class))
>  );
>  // filter to pull out summaries (all tables that are next to a title link)
>  NodeFilter summaryTableFilter = new AndFilter (
>                 new NodeClassFilter (TableTag.class),
>                 new NodeClassFilterOnPreviousSibling (LinkTag.class)
>  // custom filter
>  );
>  ========================================
>
>  Thanks for the help.  I've already tried subclassing the Parser so
>  that I could implement the clone() method, but got the same result.
>
>  -Daniel
>



-- 
-------------------------------
Daniel
me...@da...

www.OneDanShow.com
-------------------------------

Hello,

Anyone know why I can't use two extractAllNodesThatMatch(filter)
methods back-to-back on the same Parser instance?

More specifically I have this code:

========================================
Parser parser = new Parser(google);

NodeList titleList = parser.extractAllNodesThatMatch(titleFilter);
NodeList summaryTableList = parser.extractAllNodesThatMatch(summaryTableFilter);
========================================

The Google search results page I'm parsing has a series of these:

<a href="blah">Title</a>
<table><tr><td>.....Summary info....</td></tr></table>

The two filters above, when independent, work fine.  Run them
back-to-back and the second will come up empty.  I don't see where the
extractAllNodesThatMatch method literally pulls the nodes out of the
captured source, thus affecting the second filter. Here are my
filters:

========================================
// filter to pull out titles (all links that are next to a table)
NodeFilter titleFilter = new AndFilter (
		new NodeClassFilter (LinkTag.class),
                new HasSiblingFilter (new NodeClassFilter(TableTag.class))
);
// filter to pull out summaries (all tables that are next to a title link)
NodeFilter summaryTableFilter = new AndFilter (
		new NodeClassFilter (TableTag.class),
                new NodeClassFilterOnPreviousSibling (LinkTag.class)
// custom filter
);
========================================

Thanks for the help.  I've already tried subclassing the Parser so
that I could implement the clone() method, but got the same result.

-Daniel

Hi,

I'm a fairly new user of the HTML Parser software, and am already grateful
for all the work that has gone into it... I'm currently building some
software to extract RSS feed links. These appear in many different forms,
and there's one particular site that's giving me some trouble (www.mydd.com)...


Specifically, the link appears in the page, but surrounds an image, like so:

*<p><a href="http://feeds.feedburner.com/mydd"><img src="..."></a>*

When I use a basic tag name filter, the actual tag above doesn't get
returned (while other <a> tag links do)... I've been playing around with the
code and don't know where to go from here. Is it because it surrounds an
image? Is there anything I can do to fix this?

I'd be grateful for any help.

Thank you,
Wojciech

-- 

Five Minutes to Midnight:
Youth on human rights and current affairs
http://www.fiveminutestomidnight.org/

Hi,

I searched high and low for an answer to this question but I don't 
seem to be able to find it. I use htmlparser 1.6 (what a great product 
by the way) but for one of my applications I need to change the 
UserAgent used by htmlparser when requesting pages.

At the moment that always seems to be: HTMLParser/1.6

Is there any way I can change that field?

Thanks in advance

Brian.



__________________________________________________
Up to 33% off Norton Security from Tiscali - http://www.tiscali.co.uk/securepc/