Home » Unlabelled » Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?
Wednesday, 19 June 2013
Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?
Usually I would use the robots.txt to keep a search engine from indexing an entire directory on a site. This would often be directories dealing with admin functions or directories that only contain script or image libraries.
No comments:
Post a Comment