htmlparser-user Mailing List for HTML Parser

Hello Ernest,

This is the function I use in order to extract the text. I hope it helps
you.



public StringBuilder textExtractor(String URL){

StringBuilder textInPage = null;
    try {

    Parser parser = new Parser(URL);
    TextExtractingVisitor visitor = new TextExtractingVisitor();
    parser.visitAllNodesWith(visitor);

    textInPage = new StringBuilder(visitor.getExtractedText());


} catch (ParserException ex) {
    Logger.getLogger(HTMLAnalizer.class.getName()).log(Level.SEVERE, null,
ex);
}
return textInPage;
    }


Regards,

Miguel

On 24 August 2012 21:14, Ernest Cronin <ern...@gm...> wrote:

> Hi,
>
> I use the parser a lot for work. one thing i've noticed is that in many
> news articles there are comment sections, and in these sections, plain
> text. but the parser doesn't pick them up. what is about the comment
> sections that make it unreadable? is there a different class i should be
> using?
>
> Thank you,
> ernest
>
> On Wed, Aug 17, 2011 at 4:25 PM, ernest cronin <ern...@gm...>wrote:
>
>> Hi,
>>
>> I have been trying to use the parser for some time and I have been unable
>> to get it to do exactly what I want, which is to gather only the plaintext
>> without javascript or style stuff. Here is the code I've been running:
>>
>>   public class Test
>>    {
>>       public static void main (String[] args)
>>       {
>>          try
>>          {
>>             Parser parser = new Parser (args[0]);
>>      TextExtractingVisitor visitor = new TextExtractingVisitor();
>>     parser.visitAllNodesWith(visitor);
>>       String textInPage = visitor.getExtractedText();
>>    System.out.println(textInPage);
>>          }
>>             catch (ParserException pe)
>>             {
>>                pe.printStackTrace ();
>>             }
>>       }
>>     }
>>
>> I could really use some help with this!
>>
>> Thanks,
>> Ernest
>>
>>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

Hi,

I use the parser a lot for work. one thing i've noticed is that in many
news articles there are comment sections, and in these sections, plain
text. but the parser doesn't pick them up. what is about the comment
sections that make it unreadable? is there a different class i should be
using?

Thank you,
ernest

On Wed, Aug 17, 2011 at 4:25 PM, ernest cronin <ern...@gm...>wrote:

> Hi,
>
> I have been trying to use the parser for some time and I have been unable
> to get it to do exactly what I want, which is to gather only the plaintext
> without javascript or style stuff. Here is the code I've been running:
>
>   public class Test
>    {
>       public static void main (String[] args)
>       {
>          try
>          {
>             Parser parser = new Parser (args[0]);
>      TextExtractingVisitor visitor = new TextExtractingVisitor();
>     parser.visitAllNodesWith(visitor);
>       String textInPage = visitor.getExtractedText();
>    System.out.println(textInPage);
>          }
>             catch (ParserException pe)
>             {
>                pe.printStackTrace ();
>             }
>       }
>     }
>
> I could really use some help with this!
>
> Thanks,
> Ernest
>
>

hello,
Can anyone help me in my work? I am stuck somewhere. I am parsing a page
using htmlparser. In the page there can be a call to a particular function.
Let function is
f(a,b,c) {a+b+c;}. And it is called somewhere in page like f(p,q,r).
I want to ask that, how will i come to know that there is a call from the
page to 'f'. Is there any provision that can be used to identify the call
made and to which function??
Please help me, I need an urgent help.

Hello AniketP,


I had the same problem but whit the bold and italics tags (<b> and <i>
respectively). Here is my solution for <i> tags:


Create a class for the tag you are interested in, that extends CompositeTag:

public class ItalicTag *extends CompositeTag*{

    private static final String[] mIds = new String[] {*"I"*};  //Change
this as appropriate
    public ItalicTag ()
    {
    }
    public String[] getIds ()
    {
        return (mIds);
    }
    public String[] getEnders ()
    {
        return (mIds);
    }
    public String[] getEndTagEnders ()
    {
        return (new String[0]);
    }
}



//In your main class:

factory = new PrototypicalNodeFactory();   // create a factory
factory.registerTag(new  ItalicTag  ());   //register your new tag


try {

    Parser parser = new Parser (URL);
    parser.setNodeFactory(factory);
    NodeList list;

    NodeFilter tagfilter = new NodeClassFilter(ItalicTag.class);

    list = parser.extractAllNodesThatMatch(tagfilter);

    for (Node node : list.toNodeArray()) {
String texto = *extractText*(node);      // In this function we will
extract the content between tags (<i> </i>)
           }
} catch (ParserException ex) {
   //do something
}



//the extactText method :

 /**
     *Gets the text that is enclosed between labels. In order to do that
     *it studies the children components in the labels in a recursive way.
     *
     * @param studiedNode
     * @return Text between nested tags
     */
    public String  *extactText  *(Node  studiedNode  ) {
        Node node;
        String text = "";
        boolean exit= false;

        try {
            for (SimpleNodeIterator e =
studiedNode   .getChildren().elements(); e.hasMoreNodes() && !exit;) {
                node = e.nextNode();
                if (node instanceof CompositeTag) {
                    text =  extactText  (node);
                } else {
                    if (node != null) {
                        text = node.getText();
                    }
                    exit= true;

                }
            }
        } catch (NullPointerException ex) {
            // do something
        }


        return text.trim();
    }


I hope this helps.



On 8 August 2012 10:57, Aniket P <ani...@gm...> wrote:

> Hello all,
> Currently I am using the htmlparser in my work. I want to extract <script>
> </script> part, and more specifically I want to extract different functions
> in <script> </script>. After that I need to execute those functions. So can
> anyone please help me how to use that??
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

Hello all,
Currently I am using the htmlparser in my work. I want to extract <script>
</script> part, and more specifically I want to extract different functions
in <script> </script>. After that I need to execute those functions. So can
anyone please help me how to use that??