Last year I was working on a system where we were creating a lot of sites. I mean, tens of thousands of sites, and all within one site collection (they’d all have a few documents and tasks, but not many). Anyway, we were told by a partner’s developer that:
- A Site Collection’s Root Site could only have 127 subsites before performance issues would start
- Each of those subsites could have up to 2000 subsites before performance issues would start
This seemed a bit suspicious to me. I had read on MSDN about the capacity boundaries and limits, and knew about the 2000 subsites per site note:
Subsite
2,000 per site view
The interface for enumerating subsites of a given Web site does not perform well as the number of subsites surpasses 2,000. Similarly, the All Site Content page and the Tree View Control performance will decrease significantly as the number of subsites grows.
Okay, that’s fine. And I knew that the limit on the number of Sites per Site Collection is 250,000. But I couldn’t find anything about this ‘127’ subsites under the root site thing, and I couldn’t help but wonder if this was where the ‘127’ value had come from (125 x 2000 = 250,000) as I simply didn’t see why a root site should be any different to any other type of site. Thus, I decided to test it.
I created a site template that I’d use with all tests – it was pretty lightweight, with a documents library, announcements list, and task list. I then wrote a console application to create a large number of sites based on that template.
My first test was simply creating a large number of sites directly underneath the root site – I wanted to see if there was some ‘127’ subsite threshold. The graph below shows the creation time of each site (in milliseconds), and a moving average.
Well, that was pretty interesting. A flat average creation time per site until a little over 2000 sites, when there is a step change in the creation time. That suggests there is no problem with having more than 127 subsites of the root site.
My next question was about how to ‘avoid’ this limit. If we had ‘spreading’ sites underneath our root site, and then created up to 2000 sites under those spreading sites, could we create much more than 2000 sites? The technet documentation says that we should be able to, but I decided to test this anyway. So, the structure would be something like:
The following graph shows the result of creating a large number of sites using these ‘spreading sites’. The time is in milliseconds:
This shows a fairly flat average creation time (with a few spurious spikes – presumably something else was happening on the machine) for over 28,000 sites. There doesn’t seem to be much performance degradation.
Therefore, my conclusions:
- Root Sites are like any other site – and that if you put more than 2000 subsites within it, you may see a sudden performance drop.
- If you insert another layer of sites into your heirarchy – so that no site has more than 2000 subsites – SharePoint works quite happily.
Andy Burns' Blog 
Can you elaborate a bit on the use of creation time as the sole scalability metric? Can it be expected that all other operations will scale along the same lines?
For lists, the breaking of permissions inheritance has serious effects on performance in high volume of items. This is more common with spwebs than with list items so it might be something to test.
Also its worth mentioning a bit of history. In SPS 2003 (portal server) Microsoft used so-called bucketing to ensure scalability with the infamous C1 bucket sites in the urls:)
Mads
Twitter.com/madsnissen
Excellent article, it’s really great to do testing like this, and I wish there was more like it out there.
With that said, there are quite a few things that would leave me very nervous indeed about a single site collection of this size.
Here are *just a few*:
1) If a site, somewhere in this hierarchy, suddenly starts to grow, your options for managing that growth are very limited indeed. Everything will have to live in a single Content Database forever. Seriously limiting your scale, and ability to manage Service Levels.
2) I would be very curious about the performance of the site from a UX point of view. The default SharePoint UI is simply not designed to deal with large numbers of sub-sites, making it very clumsy to navigate around in.
3) You are walking a very fine line when it comes to a stack of other limits inside SharePoint. For example the size of the ACL could grow particularly fast with lots of sites with even modest permission requirements, impacting, among other things, search. (http://weblogs.asp.net/erobillard/archive/2008/09/11/sharepoint-security-hard-limits-and-recommended-practices.aspx, there are limits in 2010 too, not enough time to dig them out)
4) API’s like the SiteData API may let you down, there are lots of “gotchas” in there, and very few decent work-arounds. (there are more than 1 gotcha)
5) You get some very long URL’s indeed, which apart from being a pain generally, have the potential to break things, including going beyond what a browser can handle. (http://support.microsoft.com/default.aspx?scid=kb;en-us;208427)
6) There are all the challenges around lifecycle management, backing up/restoring/archiving of subsites can be pretty painful, and inexact.
And I could go on, and I’m sure you know a lot of this stuff already. The only reason I’m commenting, is in my experience, if you are at the point where you are working out if SharePoint can cope with the number of sites you want to create, then there is a good chance you are putting in place the worng solution architecture, and that trouble is awaiting you in the months ahead.
Thanks for the post!
Daniel
@Mads – You are quite right, there are lots of other factors that do need considered – the number of groups, permissions, total data size – all of which could blow such a solution out of the water – or at least, into the realms of creating Site Collections instead.
I did also record and map memory usage (I was looking for leaks that might affect things) but it didn’t show anything interesting, so I didn’t blog about it.
For example, I’ve been asked to build a couple of systems like this. In one, the customer was happy with ‘everyone has the same rights, across the entire site collection’, which was fine, and they were storing limited data for each. In another, the customer wanted over 30,000 project sites, each with their own permissions, sometimes to a list/library level – and some of those projects could get to be large.
And thanks, I started with SharePoint in the 2007 Beta – I’ve always wondered what the C1 type stuff in 2003 systems was.
@Daniel – Well, in short, I agree with nearly everything you said. There are a *lot* of things that could be an issue, and certainly, creating tens of thousands of sites isn’t something I’d rush out and do without planning. Permissions *is* a particular problem at this level.
One of the points I would take issue with is 5) – you’re creating the site, so you can choose it’s URL, and you can work to keep that short. For example: http://example.com/projects/1/1234 – it’s not that long. Granted, it isn’t very friendly either – but that’s a design decision you can make.
The other point, 2) about navigation, is a good one. Fortunately, building custom navigation is something that I’ve a fair bit of experience of! Yes, with such a solution, we’d need to provide something other than the out of the box stuff. (Hint – think Site Directory)
One point that I’m not familiar with is the Site Data API letting causing problems. I’d be interested to hear more.
I totally agree that you don’t want to go creating thousands of sites at the drop of the hat, but there are times that it is a valid information architecture.
Nice Article … 🙂
nice … 🙂
But when we have a requirement to create more than this much of sites ?
Is there any requirement???????? 😦
Excellent post…..