Package lxml :: Package html :: Module clean :: Class Cleaner
[frames] | no frames]

Class Cleaner

source code

object --+
         |
        Cleaner

Instances cleans the document of each of the possible offending elements. The cleaning is controlled by attributes; you can override attributes in a subclass, or set them in the constructor.

scripts:
Removes any <script> tags.
javascript:
Removes any Javascript, like an onclick attribute.
comments:
Removes any comments.
style:
Removes any style tags or attributes.
links:
Removes any <link> tags
meta:
Removes any <meta> tags
page_structure:
Structural parts of a page: <head>, <html>, <title>.
processing_instructions:
Removes any processing instructions.
embedded:
Removes any embedded objects (flash, iframes)
frames:
Removes any frame-related tags
forms:
Removes any form tags
annoying_tags:
Tags that aren't wrong, but are annoying. <blink> and <marque>
remove_tags:
A list of tags to remove.
allow_tags:
A list of tags to include (default include all).
remove_unknown_tags:
Remove any tags that aren't standard parts of HTML.
safe_attrs_only:
If true, only include 'safe' attributes (specifically the list from feedparser).
add_nofollow:
If true, then any <a> tags will have rel="nofollow" added to them.
host_whitelist:

A list or set of hosts that you can use for embedded content (for content like <object>, <link rel="stylesheet">, etc). You can also implement/override the method allow_embedded_url(el, url) or allow_element(el) to implement more complex rules for what can be embedded. Anything that passes this test will be shown, regardless of the value of (for instance) embedded.

Note that this parameter might not work as intended if you do not make the links absolute before doing the cleaning.

whitelist_tags:
A set of tags that can be included with host_whitelist. The default is iframe and embed; you may wish to include other tags like script, or you may want to implement allow_embedded_url for more control. Set to None to include all tags.

This modifies the document in place.

Instance Methods
 
__init__(self, **kw)
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
source code
 
__call__(self, doc)
Cleans the document.
source code
 
allow_follow(self, anchor)
Override to suppress rel="nofollow" on some anchors.
source code
 
allow_element(self, el) source code
 
allow_embedded_url(self, el, url) source code
 
kill_conditional_comments(self, doc)
IE conditional comments basically embed HTML that the parser doesn't normally see.
source code
 
clean_html(self, html) source code

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __str__

Class Variables
  scripts = True
  javascript = True
  comments = True
  style = False
  links = True
  meta = True
  page_structure = True
  processing_instructions = True
  embedded = True
  frames = True
  forms = True
  annoying_tags = True
  remove_tags = None
  allow_tags = None
  remove_unknown_tags = True
  safe_attrs_only = True
  add_nofollow = False
  host_whitelist = ()
  whitelist_tags = set(['embed', 'iframe'])
Properties

Inherited from object: __class__

Method Details

__init__(self, **kw)
(Constructor)

source code 
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
Overrides: object.__init__
(inherited documentation)

kill_conditional_comments(self, doc)

source code 
IE conditional comments basically embed HTML that the parser doesn't normally see. We can't allow anything like that, so we'll kill any comments that could be conditional.