what is robots.txt

  By Junaid A   Posted on June-22-2018 

what is robots.txt?

Answer:

this is simple robots.txt file that allow and block agents

for example.

User-agent: *
User-Agent: Googlebot

 

with the help of this file you can allow, block or deny agents and page URL as well as.

 

By  asifjans    22-Jun-2018 Views  605


Comments

Your Comments

You may also read following recent articles

Recent Column 7 Ways to Retain More of Every Book You Read!
04-Feb-2023    11
Recent Column Top 5 Most Tech Skills for 2023!
04-Feb-2023    10
Recent Column Dubai’s Luxury Villas Prices Spike 100% Thanks to the Richest!
03-Feb-2023    17
Recent Column Rupee Drops to Historic Low After Comments From Dar, PM Shehbaz Cause Market to Panic!
03-Feb-2023    18
Recent Column What Is Technical SEO? Basics and 10 Best Practices!
03-Feb-2023    24
Recent Column Top 10 Most Important IT Skills in 2023!
02-Feb-2023    36
Recent Column How To Create A Content Strategy Framework!
02-Feb-2023    18
Recent Column Can GPT-3 pay in the future?!
02-Feb-2023    6
Recent Column What is digital marketing? !
02-Feb-2023    46
Recent Column Content Planning: How to Create a Content Plan, Step by Step!
01-Feb-2023    16
Recent Column Top Software Testing Trends in 2023!
01-Feb-2023    67
Recent Column How To Make Money on YouTube in 2023!
31-Jan-2023    36
Recent Column The 6 Best Entry-Level Sales Jobs To Start Your Career!
31-Jan-2023    23
Recent Column Best Web and Desktop Application Security Testing Techniques!
31-Jan-2023    52
Recent Column Affiliate Marketing Program for Beginners!
31-Jan-2023    5