Index
> Feedback > why the hell you block googlebot? |
| Author |
|
|
asmrox
i tried to search some info, and my old topics but...
http://board.flatassembler.net/robots.txt you blocked google. Why? http://www.google.com/search?q=site:board.flatassembler.net |
|||
05 Mar 2008, 17:53 |
|
|
Ehtyar
I have to second this, it's bugged me for a while, phpbb search just can't match google.
Ehtyar. |
|||
|
05 Mar 2008, 21:24 |
|
|
edfed
and if flat board is very slow, we cannot access it in less than 3 seconds...
then, yes, changing the host, leave this bot blocked (make this board elitist cause bad referenced by google) and everything will be ok. |
|||
|
06 Mar 2008, 10:21 |
|
|
revolution
If a new host can cope with the traffic then I see no reason to continue to block the bots. Unless there is anther reason for blocking the bots that has not been revealed?
|
|||
|
06 Mar 2008, 10:27 |
|
|
edfed
bots are bad. they pollute the screen with random posts.
|
|||
|
06 Mar 2008, 11:14 |
|
|
revolution
edfed: The bots being discussed are not "chat bots"
|
|||
|
06 Mar 2008, 11:22 |
|
|
edfed
ok! then i hope it will never be a forum bot, like ads by ggogle...
there are a lot of forums with a post from google or else company. if it appens on fasm board, i'll be sad. |
|||
|
06 Mar 2008, 11:44 |
|
|
revolution
edfed: read up on "search bots", they index the forum so that you can search for things on sites like google. They never make posts.
|
|||
|
06 Mar 2008, 11:47 |
|
|
asmrox
http://siteexplorer.search.yahoo.com/search?p=http://board.flatassembler.net
http://search.live.com/results.aspx?q=site:board.flatassembler.net http://www.altavista.com/web/results?q=site:board.flatassembler.net you blocked only google, other sucky crwalers index it. |
|||
|
07 Mar 2008, 12:53 |
|
|
Tomasz Grysztar
google bot was much more agressive than the other ones.
|
|||
|
07 Mar 2008, 13:10 |
|
|
revolution
I thought this:
http://board.flatassembler.net/robots.txt wrote: User-agent: * |
|||
|
07 Mar 2008, 13:45 |
|
|
f0dder
revolution wrote: I thought this: Only bots that actually respect robots.txt?  - and some of them might be looking for hardcoded strings, not accepting wildcard..._________________  - carpe noctem |
|||
|
07 Mar 2008, 13:50 |
|
|
revolution
f0dder wrote: Only bots that actually respect robots.txt? |
|||
|
07 Mar 2008, 13:54 |
|
|
itsnobody
if the search bots were unblocked the traffic would increase by like 10 times
|
|||
|
08 Mar 2008, 12:37 |
|
< Last Thread | Next Thread > |
Forum Rules:
|