Guillaume Rischard
|
bb4532b6dc
|
Add wiki for documentation and regional extracts
|
2024-07-30 15:21:31 -05:00 |
|
Guillaume Rischard
|
de95834070
|
Add planet link and explanation to robots.txt
|
2024-07-30 15:00:46 -05:00 |
|
Tom Hughes
|
37966859da
|
Revert "Block indexing of diary entries"
This reverts commit fc67e63589.
|
2019-07-02 19:45:23 +01:00 |
|
Tom Hughes
|
fc67e63589
|
Block indexing of diary entries
|
2019-06-23 00:23:21 +01:00 |
|
Tom Hughes
|
7a91da329d
|
Allow indexing of diary entries again
|
2019-06-16 00:54:30 +01:00 |
|
Tom Hughes
|
7f61fc22de
|
Allow indexing of diary entries again
|
2019-06-16 00:53:38 +01:00 |
|
Andy Allan
|
31e1204dfa
|
Remove dead link to sitemap
Fixes #1113
|
2019-06-05 16:18:48 +02:00 |
|
Tom Hughes
|
71cbb003f8
|
Exclude per-user diary pages from indexing
|
2019-05-19 11:56:47 +01:00 |
|
Tom Hughes
|
25951cae38
|
Block indexing of diaries
|
2019-05-18 10:11:12 +01:00 |
|
Grant Slater
|
6c1917ce73
|
Add Host: to robots.txt
|
2015-12-26 23:53:59 +00:00 |
|
Tom Hughes
|
2ec712d6ea
|
Remove redundant robot rules
|
2014-06-17 22:14:36 +01:00 |
|
Tom Hughes
|
dc931c4ac4
|
Remove robots.txt block for /user/*/diary
|
2013-10-17 08:55:52 +01:00 |
|
Tom Hughes
|
39c0df691e
|
Remove special casing of Googlebot and Slurp in robots.txt
|
2013-06-27 18:43:56 +01:00 |
|
Grant
|
57938f78c7
|
Add /user_blocks/ to robots.txt disallow
Add Disallow: /user_blocks/ as the content is not suitable for indexing.
|
2012-10-26 19:20:31 +01:00 |
|
Tom Hughes
|
7a8fe086cd
|
Don't allow robots to fetch user edit lists
|
2010-04-20 22:40:32 +01:00 |
|
Grant Slater
|
4bbd7f5310
|
Update robots.txt to Disallow browser [node|way|relation] history. Content now worth indexing in most cases.
|
2010-03-03 11:41:43 +00:00 |
|
Tom Hughes
|
42b0dd52fd
|
New robots file from Firefishy.
|
2010-02-26 19:02:17 +00:00 |
|
Tom Hughes
|
53d30d70af
|
Stop robots crawling the object browser.
|
2009-02-06 10:12:50 +00:00 |
|
Grant Slater
|
a46a4b4eb9
|
Bots Disallow /edit* & /login*, Nothing to see here... move right along.
|
2008-10-26 17:26:39 +00:00 |
|
Grant Slater
|
b13278c815
|
Block bots from /trace/, so they don't attempt to eat all the GPX data.
|
2008-10-09 18:31:36 +00:00 |
|
Steve Coast
|
b4c0bfd8fa
|
more basic framework
|
2006-07-28 21:53:30 +00:00 |
|
Steve Coast
|
06f3473d1c
|
beginning of osm on rails
|
2006-07-28 21:41:39 +00:00 |
|