/    Sign up×
Community /Pin to ProfileBookmark

Basic Google / Search Engine Questions

Copy linkTweet thisAlerts:
Mar 29.2005

Hey people,

Hi i have just created a site and gone live with it, I have done all the usual tests and am now ready to submit to google – I have heard that as well as providing your Url for the site that you need to upload a .txt file called robots so that the Google web crawlers can index your site.

Is this true, or just a myth?

Thanx Everyone

to post a comment
SEO

2 Comments(s) ↴

Copy linkTweet thisAlerts:
@AdamGundryMar 29.2005 — It's true that Google pays attention to robots.txt, but they are not essential for indexing your site. They are mainly used to prevent robots (or a subset of them) from indexing certain pages.

Check out [url=http://www.robotstxt.org/]robotstxt.org[/url] for more information.

Adam
Copy linkTweet thisAlerts:
@_lt_Eddie_gt_Apr 01.2005 — Instead of chasing all the search engines though, make them come to you. Get listed in a directory that's already listed in G, Y and M, such as [url=www.dmoz.org]DMoz[/url].
×

Success!

Help @Lightfoot82 spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 5.15,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @AriseFacilitySolutions09,
tipped: article
amount: 1000 SATS,

tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,
)...