Grant
|
57938f78c7
|
Add /user_blocks/ to robots.txt disallow
Add Disallow: /user_blocks/ as the content is not suitable for indexing.
|
2012-10-26 19:20:31 +01:00 |
|
Tom Hughes
|
7a8fe086cd
|
Don't allow robots to fetch user edit lists
|
2010-04-20 22:40:32 +01:00 |
|
Grant Slater
|
4bbd7f5310
|
Update robots.txt to Disallow browser [node|way|relation] history. Content now worth indexing in most cases.
|
2010-03-03 11:41:43 +00:00 |
|
Tom Hughes
|
42b0dd52fd
|
New robots file from Firefishy.
|
2010-02-26 19:02:17 +00:00 |
|
Tom Hughes
|
53d30d70af
|
Stop robots crawling the object browser.
|
2009-02-06 10:12:50 +00:00 |
|
Grant Slater
|
a46a4b4eb9
|
Bots Disallow /edit* & /login*, Nothing to see here... move right along.
|
2008-10-26 17:26:39 +00:00 |
|
Grant Slater
|
b13278c815
|
Block bots from /trace/, so they don't attempt to eat all the GPX data.
|
2008-10-09 18:31:36 +00:00 |
|
Steve Coast
|
b4c0bfd8fa
|
more basic framework
|
2006-07-28 21:53:30 +00:00 |
|
Steve Coast
|
06f3473d1c
|
beginning of osm on rails
|
2006-07-28 21:41:39 +00:00 |
|