UltraBB Forums Home 
Search     Members Calendar Help Home
Search by username
Not logged in - Login | Register 

Come on over and join us at the new place- www.ultrabb.net/forum
Same log in as this board but you will have to log in. If you have any problems doing so shoot us an e-mail support@ultrabb.net and we will help you get in.


 Moderated by: Jim
New Topic Reply Printer Friendly
Yahoo Slurp - Software Assistance - UltraBB - UltraBB Forums
AuthorPost
 Posted: Thu Apr 17th, 2008 06:19 am
PMQuoteReply  
1st Post
zzkenoman
Epicurean Rider


Joined: Sat Mar 15th, 2008
Location: Midwest, Iowa USA
Posts: 27
Status: 
Offline
Mana: 
I was getting enormous web crawlers to my forum and wanted to let you know Yahoo had launched a new indexing program April 14th.  I found instructions to lessen the problem, I think.

http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html

Back To Top PMQuoteReply

 Posted: Thu Apr 17th, 2008 09:06 am
PMQuoteReply  
2nd Post
Jim
Administrator


Joined: Wed Apr 11th, 2007
Location:  
Posts: 1963
Status: 
Offline
Mana: 
It's actually a good thing, well for your sites popularity anyway. I can put a file in there if you want or you can, but basically even the delay will slow them down meaning you will not get crawled as much.

Maybe Yahoo follows the rules but I have seen with my own eyes bots that ignore the robots.txt file and crawl everything anyway. Whatever you want to do is fine but I would only exclude directories outside the main one.

Back To Top PMQuoteReply

 Posted: Sun Apr 20th, 2008 01:34 pm
PMQuoteReply  
3rd Post
John Floyd
License Holder


Joined: Sun Jan 27th, 2008
Location: South Mills, North Carolina USA
Posts: 128
Status: 
Offline
Mana: 
I don't care if they ever crawl my site, my Membership is limited and private, Could you apply this fix for me.

Back To Top PMQuoteReply

Current time is 12:31 pm  
UltraBB Forums > UltraBB > Software Assistance > Yahoo Slurp



WowUltra 1.15 Copyright © 2007-2008 by Jim Hale
Page processed in 0.1474 seconds (15% database + 85% PHP). 19 queries executed.