The official blog for morguefile.com and fotokoi.com
Thursday, July 15, 2010
5 Web Files That Will Improve Your Website
sixrevisions covers robots.txt, favicon.ico, sitemap.xml, dublin.rdf and opensearch.xml. Their purposes range from helping search engines index your site accurately, to acting as usability and interoperability aids.
i have been trying to figure out a lot of this for a while at www.cutcaster.com. Regarding robot.txt files. I know they are generally used for blocking crawlers from accessing content on your site but is there a command for pointing them towards your content i.e. images that you want indexed at your site.
i have been trying to figure out a lot of this for a while at www.cutcaster.com. Regarding robot.txt files. I know they are generally used for blocking crawlers from accessing content on your site but is there a command for pointing them towards your content i.e. images that you want indexed at your site.
ReplyDeleteThanks