Does anyone know how search engines deal with custom 404 error pages.
so for example if you set a site map as your 404 page would search engines such as google carry on following links or would it recognise it’s a 404 and stop or rank you lower in the search engines for it?
@IntensityDec 26.2004 — #I don't believe that the search engine spider will come accross a 404 page because if the link is invalid/broken then the spider/robots will not follow these, since none existing links that direct to a non important 404 error page is no help to the search engines indexing.
One thing that you can use to your advantage is by using your 404 error page as a great navigational site map so your visitors have the option to click a link so they can find the information that they are looking for.
@kaibobbleauthorDec 26.2004 — #see I have a lot of products on my site and they all have dynamic URL's. I thought it was a clever idea to put all the product information links as static dead links such as "..../dvd_region2.html" then do a dynamic 404 page to read the url and find the latter of the string as my product name in my database and show it as an info page for that product.
I guess then it wasn't such a good idea. I was reading aboung mod_rewrite and .htaccess file to parse dynamic pages and output them as HTML - would this work better for search engines?
@AdamGundryDec 27.2004 — #Whether a search engine indexes the page depends on how your server is configured. If you are redirecting the error to a CGI script (or similar) then it should return a Status header, which could be 200 if the request was ok or 404 if the page was not found.
It all gets rather complex, so using mod_rewrite is probably a better choice. Check out the [url=http://httpd.apache.org/docs-2.0/mod/mod_rewrite.html]Apache documentation[/url] and [url=http://httpd.apache.org/docs-2.0/misc/rewriteguide.html]URL Rewriting Guide[/url].
Apache documentation will help you a lot in URL rewriting rules. They are very good docs to read and provide needfull information. I have always used Apache DOCS for any help in URL rewriting and mod rewrite help.
I'm not kidding, have a look at the [URL=http://slettes.dk/redirect_to/?url_name=http://www.udm4.com/] Accessible Website Menu - Ultimate Drop Down Menu 4.3 by Brothercake [/URL]