Aquileo | Recent changes to tickets

Aquileo | HtmlNodeList iterators are broken

Sun, 08 Feb 2015 17:15:04 -0000

The HtmlNodeList iterator contract is broken for 2.7. The next() method is not called. The next() method is called. To fix this, create a new class HtmlNodeListIter() with init and next methods. In init() pass the nodeList being iterated and save it and the counter, and return self. In next() use the same logic as in next from HtmlNodeList. Make sure to actually call the StopIteration() constructor.

Aquileo | HTMLDOM cannot correclty parse html element without proper closing tag

Wed, 03 Sep 2014 12:24:09 -0000

Htmldom cannot parse tags which aren't closed properly.
For example:
<AREA SHAPE="RECT" COORDS="2,2,95,30" HREF="../index.shtml" alt="Home">
According to html standats it is acceptable, but htmldom fails to correctly parse it.

For example:

from htmldom import htmldom
dom = htmldom.HtmlDom().createDom("""
<body>
<MAP NAME="top_nav_map">
    <AREA SHAPE="RECT" COORDS="2,2,95,30" HREF="../index.shtml" alt="Home">
    <AREA SHAPE="RECT" COORDS="99,2,220,30" HREF="../Components.shtml" alt="Components">
    <AREA SHAPE="RECT" COORDS="224,2,319,30" HREF="../HardwareMain.shtml" alt="Hardware">
    <AREA SHAPE="RECT" COORDS="324,2,402,30" HREF="../Boards.shtml" alt="Boards">
    <AREA SHAPE="RECT" COORDS="406,2,477,30" HREF="../BooksMain.shtml" alt="Books">
    <AREA SHAPE="RECT" COORDS="482,2,535,30" HREF="../Kits.shtml" alt="Kits">
</MAP>
<h1>Hello</h1>
</body>
""")
table = dom.find("body")
print(table.html())

This code print:

<body>
    <map NAME="top_nav_map">
        <area COORDS="2,2,95,30" SHAPE="RECT" alt="Home" HREF="../index.shtml">
            <area COORDS="99,2,220,30" SHAPE="RECT" alt="Components" HREF="../Components.shtml">
                <area COORDS="224,2,319,30" SHAPE="RECT" alt="Hardware" HREF="../HardwareMain.shtml">
                    <area COORDS="324,2,402,30" SHAPE="RECT" alt="Boards" HREF="../Boards.shtml">
                        <area COORDS="406,2,477,30" SHAPE="RECT" alt="Books" HREF="../BooksMain.shtml">
                            <area COORDS="482,2,535,30" SHAPE="RECT" alt="Kits" HREF="../Kits.shtml">
                            </area>
                            <h1>
                                Hello
                            </h1>
                        </area>
                    </area>
                </area>
            </area>
        </area>
    </map>
</body>

Aquileo | Attributes cannot contain quotes

Sat, 28 Jun 2014 06:05:46 -0000

The HTML spec allows attributes delimited with double quotes to contain single quotes and vice versa. However code like

page = """<a title="It's bugged!"></a>"""
dom = htmldom.HtmlDom().createDom(page)

enters an infinite loop. It looks the the regular expression used for attributes does not allow for this.

Aquileo | #3 Parse issue when element attributes either don't have a value or are not wrapped in quotes

Wed, 19 Mar 2014 19:48:06 -0000

Sorry, I actually forgot that I e-mailed you about this issue. I had prepared this ticket before I e-mailed you, and now I saw that this ticket was unsaved, and I saved it without a second thought.

Aquileo | Parse issue when element attributes either don't have a value or are not wrapped in quotes

Wed, 19 Mar 2014 19:44:07 -0000

Hello,

I'm having unexpected trouble while parsing HTML code when the value-part for an element attribute is either omitted or is not wrapped in quotes. While I'm not completely certain that it's a bug, it's not obvious to me that it's not a bug.

Code:

from htmldom import htmldom

htmlInput = """<form>
    <select name="country_code">
        <option value="GB" selected>United Kingdom</option>
        <option value="AL">Albania</option>
    </select>
</form>"""

dom = htmldom.HtmlDom().createDom(htmlInput)
form = dom.find("form")

print("Countries:")
for option in form.find("select[name=country_code] > option"):
    value = option.attr("value")
    text = option.text()
    print("    {0} = {1}".format(value, text))

Output:

Countries:
    AL = selected>United Kingdom
    AL = Albania

I expected the first element to have the value "GB" and the text "United Kingdom".

If I do this:

<option value="GB" selected>United Kingdom</option>

<option value="GB" selected=>United Kingdom</option>

<option value="GB" selected=something>United Kingdom</option>

then the issue exists.

However, if I do this:

<option value="GB" selected="">United Kingdom</option>

<option value="GB">United Kingdom</option>

then I get the expected result.

Thank you for any help you can provide!

Aquileo | #2 Nodelists containing nested elements loop infinitely.

Thu, 02 Jan 2014 17:30:38 -0000

In order to make an object iterator python requires us to implement "__iter__" and "next" function(for python 2.7) and "__next__"(for python 3.x). You must be testing this code in python 2.x interpreter. The current HtmlNodeList does not implement "next" function, that is why the above code is not working but will work fine in python 3.x interpreter.

I knew about this bug when i implemented the code(i am really ashamed to say this). I implemented the "next" function but it didn't work. The control was not even entering the "next" function. I spent hours on it but failed to identify the bug. So i just implemented "__next__" function to make it work on python 3.x interpreters.

But after seeing this ticket, i just gone through the entire code and to my horror i saw a "next" function defined(which returns siblings of the current node list) below "__next__" function. This new def was overriding the previous function def of "next", resulting in infinite loop.

This is one of those tricky bugs where you are really confident that some part of the code works correctly and you don't even want to take a look at it. And it is also one of the disadvantageous of dynamic languages as they don't complain even if they see multiple functions with the same name.

I will correct the code upload the new one as soon as possible.

Happy coding :)

Aquileo | #2 Nodelists containing nested elements loop infinitely.

Thu, 02 Jan 2014 17:25:16 -0000

I will correct the code upload the new one as soon as possible.

Happy coding :)

Aquileo | Nodelists containing nested elements loop infinitely.

Thu, 02 Jan 2014 09:02:03 -0000

If you create a dom containing nested elements, ala:

<html><body>
  <div><p>Level One</p>
       <div><p>Level Two</p></div>
  </div>
</body></html>

And then try to iterate over a nodelist containing the nested elements, it loops infinitely

for item in dom.find( "div" ):
    print item.html()

Aquileo | #1 Problem found in createDom() method

Sun, 07 Apr 2013 09:57:42 -0000

Thank you very much for finding the bug.

Aquileo | Problem found in createDom() method

Tue, 02 Apr 2013 17:34:07 -0000

Hello,
I wanted to start using htmldom just a while ago, and everytime I was using createDom() exception was raised. I've read the code and found a solution. It was all about changing import statement in line 317 from import urllib.request as urllib2 to import urllib2. Hope it helps if anyone have a similar problem.