projects / arachne.git / commitdiff
? search:
summary | shortlog | log | commit | commitdiff | tree
raw | patch | inline | side by side (from: 83a84de)
Remember crawled pages and don’t crawl them again.
authorDavid ‘Bombe’ Roden <bombe@pterodactylus.net>
Mon, 16 Mar 2009 23:09:47 +0000 (00:09 +0100)
committerDavid ‘Bombe’ Roden <bombe@pterodactylus.net>
Mon, 16 Mar 2009 23:09:47 +0000 (00:09 +0100)
Don’t add pages that are already scheduled for crawling.


No differences found
Arachne – A Spider for Freenet Sites (defunct)