diff options
| author | Chris Gahan <chris@ill-logic.com> | 2006-02-09 18:20:55 +0000 |
|---|---|---|
| committer | Chris Gahan <chris@ill-logic.com> | 2006-02-09 18:20:55 +0000 |
| commit | 5720064dde5ff1205bf072ffe01e7ab070b2152a (patch) | |
| tree | 8a003f47697e92bc8d5da4c54aaea483e7745344 /bin/rbot | |
| parent | 345df89e4d04c89c7cd43e21e918bf0b83bb1205 (diff) | |
Changed the way the URL grabber gets urls. Instead of using HEAD, it uses GET, but only grabs the first 50k of the page to check it for a header.
Diffstat (limited to 'bin/rbot')
0 files changed, 0 insertions, 0 deletions