Sitecore lets you specify a page to use for ‘ItemNotFound’ errors in web.config, and it’s always good practice to have a pretty ‘page not found’ page:

<setting name="ItemNotFoundUrl" value="/404" />

However, Sitecore hasn’t always been the best for actually returning the right HTTP status code for the 404 page. It redirects (more on that later) your failed request to a specified page in Sitecore – but the serves that page correctly, including an HTTP 200 status. This is sad, as search engines don’t understand that the page actually hasn’t been found.

In Sitecore 6 you had to handle this yourself …

For search engine optimization, you should return the HTTP 404 status code for any invalid URL by setting the Status property of the current System.Web.HttpResponse.

Yes, that’s basically “you’ll have to deal with this yourself”. Nice. Is there really nothing built in?

Well, some instances seem to handle this a little bit better, though. In their web.config there is mention of the IIS404Handler.

<preprocessRequest help="Processors should derive from Sitecore.Pipelines.PreprocessRequest.PreprocessRequestProcessor">
....
<processor type="Sitecore.Pipelines.PreprocessRequest.IIS404Handler, Sitecore.Kernel" />
....
</preprocessRequest>

Interesting – how does this work? Well, if the request is a 404 request, it sets the correct 404 status code:

Great! Wait, how does it know it’s a 404 Request? What’s that  Sitecore.Web.WebUtil.Is404Request() call?

Fail. So it’s a 404 if the page being requested has a url containing ‘?404;’. What the hell was the person writing this thinking? That’s clearly never going to work, and I can’t believe that it seems to be looking at the GET parameters.

Thus, we are back to having to handle 404s, etc., by ourselves. Worse, we’ve not dealt with the second problem – you are REDIRECTED to the error page. This is bad for search engine optimization.

The solution to this, to me seems to be to add a processor into the pipeline – which is described in some of these…

References:

• http://www.partechit.nl/en/blog/2012/11/return-404-status-code-when-itemnotfound-page-is-loaded
• http://herskind.co.uk/blog/sitecore/2012/03/sitecore-404-without-302/
• https://briancaos.wordpress.com/2013/03/21/sitecore-404-without-302/
• http://sitecoreblog.patelyogesh.in/2013/06/handle-404-page-not-found-in-sitecore.html