SEO – Restrict page or link not to crawl by Search Engine

Introduction There are other ways to instruct search engine crawlers how to behave when indexing and following links besides using a robots.txt file.