I’ve had to look into Search latency in Office 365 for one of our customers. This is the time taken between a document being uploaded into Office 365 and it being available in Search. The results are interesting, though do raise a few more questions.

Introduction

Office 365 is supposed to be ‘feature equivalent’ to a normal SharePoint system, but unfortunately the lack of support for managed code does make handling Large Lists (5000 items or more) something of a headache when relating data to documents (I’ve written on that before, and more on that in a future post), or when retrieving documents.

We have a scenario where we are using multi-choice Managed Metadata to tag documents. Normally, to help retrieve documents from a large list, you’d use a list column index – but unfortunately, you can’t index multi-choice columns of any type in SharePoint. Naturally, my thoughts turned to Search; we’ve previously successfully used search as a way of retrieving items and documents in a variety of large-list scenarios. However, when I started test search during development, I found I wasn’t getting the results I expected. It transpires that documents were taking over 2 hours to index, but I didn’t know a more exact time.

Experiment

To determine this latency period in more detail I wrote a test application. It should:

• Once an hour, upload a new document to each of my test site collections.
• Every 15 minutes run a search to see which documents were in the search index.
• Log the results
• Wait another 15 minutes.

All 24 documents were all extracts of different Shakespeare plays, in docx format, and from 13 to 20Kb in size – so small. Search terms were carefully selected to only appear in individual documents, though for some files the search term was present in the document file name.

I use two different Office 365 tenants for the tests. I used our company tenant, and I set up a brand new Office 365 trial tenant. I created 5 sites collections in each. This results in 240 separate tests.

The application was started at 1130 on Monday 25th, and completed in the afternoon of the 26th.
Results

The results of the test show a number of interesting features. Foremost is that indexing was slower on our company O365 tenant than a new Trial tenant.

The difference is marked – an average time of 1h47 for documents in our tenant, compared to 20 minutes for the Trial tenant. That’s one fifth of the time for the Trial tenant.

Looking at the results by round of tests it can be seen that the Trial tenant was pretty consistent, but the company one is much less so:

Looking at the results in more detail it should be noted that 3 documents did not appear in the index by the end of the test. That’s 3 documents from 240 that aren’t indexed – or over 1%.

Scatter graphs of these results for the two tenants are telling:

This shows that the Trial tenant was consistently quite quick (note the different axis scale). There are two ‘lines’ on the scatter graph where results fall – around 15 minutes, and a little over 30. This matches documents being found on the first or second checks by the test application. Most would appear to be picked up on the first crawl after being added.

Our company tenant, on the other hand, shows a number of features. Average latency starts at over 4h40. It does drop overnight – as one would expect out of peak European office hours – to a comparable level and then rise with a curious peak in the morning.

Conclusion

I conclude that the new tenant is faster at indexing documents than our company tenant, except at periods of low usage. Performance is good on the trial tenant, but poor on our tenant. Both tenants can, however, have some documents take particularly long periods to index (or fail to index) documents. Indexing performance does seem to be better late at night when Office 365 has less traffic.

If our company Office 365 tenant is representative of a normal tenant, then search latency in Office 365 is not good. Business users don’t expect to have to wait for 1 hour 40 minutes to be able to find and retrieve a document that they themselves may have uploaded. I note that Chris O’Brien has already reported similar issues. From that it might be worth quoting the figures he found from Microsoft:

“Search crawls occur continuously to make sure that content changes are available through search results as soon as possible. Recently uploaded documents may not immediately be displayed in search results because of the time that’s required to process them. SharePoint Online targets between 15 minutes and an hour for the time between upload and availability in search results (also known as index freshness). In cases of heavy environment use, this time can increase to six hours.”

(Like Chris, I’m sure I remember reading a much shorting indexing time quoted in the service description – but I can’t find it anymore, and I could be wrong).

I also note other users reporting the same issue. Therefore, extreme care must be taken when offering Office 365 solutions that involve search or large lists.

However, there is an interesting question here – why should our tenant be so much slower than the trial one?

1. Volume of content? Well, our tenant has about 12Gb allocated storage – and I’d estimate about 4 Gb in use. That isn’t a huge amount data, and I wouldn’t expect the amount of data to worry an incremental crawl.
2. Larger numbers of documents to index? Well, it’s possible that one of our users added large numbers of documents into our O365 tenant just before I ran the test – hence the slow times at first – but I think it’s pretty unlikely. Most of our traffic is to our on-premise SharePoint system. Hopefully further testing should establish if our results are being skewed by other user’s traffic.
3. Upgraded vs New? Our O365 tenant was upgraded a few months ago. It seems fine (other than the curious difference in indexing speeds), and it’s hard to imagine that that would be a factor – but I guess it’s possible, and kind of hard to eliminate.

Therefore, at this point it is difficult to know what the cause of this difference in indexing times is. If the issue is the volume of content, this would give me grave concern. The amount of content shouldn’t bother an incremental crawl much, and we don’t have that much content really – certainly not the hundreds of gigabytes that some users have.

Thus, I don’t really have an explanation at this time. I’ve raised it with Microsoft and shall await a response.

Edit:Part II is here