Releases Tags

• Stable v1.2.5 1d816b419b

  Version 1.2.5

  ehlers released this 1 year ago | 1 commits to master since this release

  Use requests to fetch web pages

  • With HTTP keep-alive the TLS negotiation will be done
    only once, that results in a much faster web access.
    I have noticed an increase to double speed.
  • By disabling automatic redirection, the program no
    longer follows redirections that leave the website.

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.2.4 9d20184d29

  Version 1.2.4

  ehlers released this 1 year ago | 2 commits to master since this release

  Add option for crawl rate limitation

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.2.3 bd2fccb30a

  Version 1.2.3

  ehlers released this 2 years ago | 4 commits to master since this release

  Allow compressed transport of content

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.2.2 c2930fcf85

  Version 1.2.2

  ehlers released this 2 years ago | 5 commits to master since this release

  Use user agent ‘sitemap_gen’ for checking robots.txt

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.2.1 8055226812

  Version 1.2.1

  ehlers released this 3 years ago | 7 commits to master since this release

  Fixed getting ‘Last-Modified’ date

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.2.0 5f9137f845

  Version 1.2.0

  ehlers released this 3 years ago | 8 commits to master since this release

  Port to Python 3
  Support robots.txt with wildcards

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.1.0 9ef8374f71

  Version 1.1.0

  ehlers released this 12 years ago | 13 commits to master since this release

  Added support for the ‘nofollow’ tag and for robots.txt.

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.0.3 099fa5fa11

  Version 1.0.3

  ehlers released this 13 years ago | 14 commits to master since this release

  Added handling of HTTP redirects by Pavel “ShadoW” Dvořák

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.0.2 f81a4597b2

  Version 1.0.2

  ehlers released this 13 years ago | 15 commits to master since this release

  Added handling of BASE HREF tag.

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.0.1 1b8232dc36

  Version 1.0.1

  ehlers released this 13 years ago | 16 commits to master since this release

  Added missing XML entity escaping.

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)

   
• Stable v1.0 c69db719b5

  Version 1.0

  ehlers released this 14 years ago | 17 commits to master since this release

  Download from http://toncar.cz/opensource/sitemap_gen.html

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)