/    Sign up×
Community /Pin to ProfileBookmark

Robots.txt not working

Copy linkTweet thisAlerts:
Mar 16.2005

I noticed that Google has indexed and shows all of my .pdf documents on my website.

I didn’t want them to be accessed this way, so I put them in a special folder called:

[QUOTE]

/graph-paper/

[/QUOTE]

and told my robots.txt file this:

[QUOTE]

User-agent: *
Disallow: /graph-paper/

[/QUOTE]

Did I do it wrong?

to post a comment
SEO

1 Comments(s) ↴

Copy linkTweet thisAlerts:
@_lt_Eddie_gt_Apr 01.2005 — That is correct but are you sure Google didn't get to your files before the robots.txt was uploaded? I would rename your files to prevent ripping and wait for the next index. You could also try specifying the particular files directly if it's possible, ie. you haven't got 1000's of them.
×

Success!

Help @shopkanji spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 4.29,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,

tipper: @Samric24,
tipped: article
amount: 1000 SATS,
)...