/    Sign up×
Community /Pin to ProfileBookmark

URL restricted by robots.txt

Copy linkTweet thisAlerts:
Jul 20.2018

Hi All,

My apologies if this is not in the correct forum?

Really hope someone can help? When I try to submit my sitemap to Webmaster Tools I get the following error:

“We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.” URL restricted by robots.txt

The sitemap is created using the Yoast plugin so I’m sure it’s fine, I can view the sitemaps, but for some or other reason I’m stuck!

Any advice would be appreciated ?

to post a comment
SEO

6 Comments(s) ↴

Copy linkTweet thisAlerts:
@rootJul 20.2018 — Google by the looks of it isn't able to crawl your site, if you really want to protect your site from being crawled, then you have two options, turn off directory listing so that only the pages can be crawled or don't get your site listed.
Copy linkTweet thisAlerts:
@BevenauthorJul 20.2018 — But I want my site listed and crawled.
Copy linkTweet thisAlerts:
@rootJul 21.2018 — > @Beven#1594106 But I want my site listed and crawled.

THEN DON'T USE ROBOTS.TXT

Then google will crawl it just fine.
Copy linkTweet thisAlerts:
@efusionworldJul 24.2018 — Google is unable to crawl this URL for a variety of reasons. In some cases, Google may not be able to reach your server and this will resolve itself in a few days. If the warning continues to appear, please reach out to Google for further assistance.
Copy linkTweet thisAlerts:
@tamil96Jul 25.2018 — You can remove the sitemap page from robots file.
Copy linkTweet thisAlerts:
@bonzertechAug 06.2018 — I think you are not using any X-Robots tags in your headers. (like not the <head></head> in your HTML document).
×

Success!

Help @Beven spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 6.17,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @nearjob,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,
)...