Wednesday, 19 June 2013

Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?

Usually I would use the robots.txt to keep a search engine from indexing an entire directory on a site. This would often be directories dealing with admin functions or directories that only contain script or image libraries.

No comments: