To stop unwanted content material within the lookup indexes, webmasters can instruct spiders to not crawl particular data files or directories in the standard robots.txt file in the root directory in the area. Also, a web site is usually explicitly excluded from a search engine's database through the use of https://news.rafeeg.ae