This one was a real Sherlock Holmes case. The site I’ve been working on has Azure App Insights monitoring it, and on the live site we started to see lots of exceptions that hadn’t appeared in testing.

Digging into the exceptions, I noticed that:

• The exception was System.Web.HttpUnhandledException with an inner System.NullReferenceException
• The method throwing the exception was Sitecore.Form.Core.Ascx.Controls.SimpleForm.OnAddInitOnClient, which is part of the Web Forms for Marketers suite.
• The user agent was always Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) – so it was Googlebot making the request. There’s no reason that should cause an exception though!

I realised after a while that Googlebot is a robot, and so is excluded from analytics, so this problem probably had the same cause as the exceptions our testing was starting to throw up. The OnAddInitOnClient method isn’t one of mine, though – it’s part of Sitecore’s Web forms for Marketers – so I opened up Reflector and down the rabbit hole I went…

The relevant method looks like this:

There are a few points to note. First, the entire if statement marked by 2) is duplicate. Clearly this should be an ‘else’ statement.

Secondly, after some testing I was able to determine that the NullReferenceException was being caused by trying to access Tracker.Current.CurrentPage, marked by 1). This was null, so trying to get ‘VisitPageIndex‘ cause a yellow screen of death.

This does fit with the Sitecore.Analytics.ExcludeRobots.config excluded robots configuration. The point of that is to turn off analytics for those robots, to avoid pollution of Sitecore’s analytics data. So, excluded agents – such as Googlebot, or our tester when we excluded her IP address – will have uninitialised analytics, and this results in an exception from SimpleForm.OnAddInitOnClient.

The worst part of this is that it could be really hard to detect. We were lucky – if we hadn’t accidentally excluded our tester, and if we hadn’t had Azure App Insights showing us exceptions, we’d have had very little evidence to expose the problem. All we would have known is that we didn’t appear in Google’s (or other engine’s) search results – because Googlebot (or other robots) were failing to index pages that had forms on them.

It is worth noting that entries would appear in Sitecore’s log files. We all review those, right?

So, the fix? Well, we have a form control that was inheriting from SimpleForm.  I decided to subclass SimpleForm and override OnAddInitOnClient. I’d fix that, and then make our form control use the fixed version:

public class FixSimpleFormShim : Sitecore.Form.Web.UI.Controls.SitecoreSimpleFormAscx
{
protected override void OnAddInitOnClient()
{
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.Append("$scw(document).ready(function() {");
stringBuilder.AppendFormat("$scw('#{0}').webform(", this.ID);
stringBuilder.Append("{");
stringBuilder.AppendFormat("formId:\"{0}\"", this.FormID);
if (this.Form != null && this.Form.IsDropoutTrackingEnabled && !this.FastPreview && this.PageItem != null)
{
string currentPage = string.Empty;
try
{
if( Tracker.Current != null )
{
ICurrentPageContext ctx = Tracker.Current.CurrentPage;
currentPage = Tracker.Current.CurrentPage.VisitPageIndex.ToString(CultureInfo.InvariantCulture);
}
}
catch
{
// Accessing Tracker.Current.CurrentPage fails on requests that are excluded from Analytics by Sitecore.Analytics.ExcludeRobots.config
// Tracker.Current.CurrentPage is Null
}
stringBuilder.AppendFormat(", pageId:\"{0}\", pageIndex:\"{1}\", eventCountId:\"{2}\", tracking : true", this.PageItem.ID, currentPage, this.EventCounter.ClientID);
}
stringBuilder.Append("})});");
this.Page.ClientScript.RegisterClientScriptBlock(base.GetType(), "sc-client-webform" + this.ClientID, stringBuilder.ToString(), true);
}
}

Or a tidier version from Reflector:

This doesn’t entirely fix the issue – agents excluded from analytics have problems submitting forms, still – but it does at least let the page render successfully for crawlers, which shouldn’t be submitting forms!

The image at the top is from the morning I deployed the fix; you can see all exceptions suddenly stopped…