crawl blacklist not working

[cweiske]

The crawlblacklist did not seem to work and began indexing things
explicitly blacklisted. I even put an echo statement in the code to
confirm the regex matched the url, but it still indexed it anyway.

---

reported by dan via e-mail