glenn re-raises the rss bandwidth issue. i’ll admit i’m lazy, and haven’t implemented if-modified-since handling for my own aggregator (which only polls hourly, and only news sites — i don’t read blogs via an aggregator), or if-modified-since handling for the rss feeds i produce (particularly the scraped feeds).

but one thing i have done is implement a system that serves up 403 responses to people who poll the scraped feeds more than once an hour. it blocks over ten thousand requests a day. what’s amazing to me is the number of persistent attempts in the face of repeated errors. for my polling, i am emailed errors when they occur, so i would know pretty quickly when i’ve been blocked. does make if-modified-since requests. of course, since it is only making requests when there should be changes (it only does so in response to pings, it doesn’t poll), it doesn’t make much difference.

« being taunted by another company november 14, 2004 6:08pm »