I stumbled onto a weird problem this week and I wanted to share it with you all. The problem itself is kind of weird, but that's not really what surprised me about it. What surprised me is that it took so long to find and more people aren't having it. Here is the most succinct way I can describe the it:
"When a default quota template is defined for the web app if a Farm Administrator that is NOT the System Account tries to create a site collection where that Farm Administrator is NOT a site collection administrator the creation will fail with an 'Access Denied' error."
That's a mouthful, so let's walk through the steps to reproduce it, which might help explain it some. To reproduce this error you'll need three accounts:
- SP_Farm – This account is used for everything except content crawl. Bits were installed as this user, PSConfig was run as this user. The Timer Job service runs as this user, and all your app pools run as this user. When you log into a SharePoint page with this account the Welcome banner on the top right says "Welcome System Account."
- Todd – Obviously this account can have any name. What's important is that this account is in the Farm Administrators group in SharePoint, and it is used for day to day SharePoint administrative tasks instead of SP_Farm. When you log into a SharePoint page with this account the Welcome banner on the top right does not say "Welcome System Account." It welcomes the user by name instead.
- Jill – This account must be named this. Okay, I'm kidding, it can be named anything. This account is a regular old user account. This account represents someone that called the helpdesk or used whatever process you have for creating a Site Collection. To reproduce this bug you don't actually have to log in as this account, it just must exist.
Now that we've got all the accounts created, let's get down to reproducing the error. Verify the Todd user is in the Farm Administrators group by going to Central Admin > Operations > Update farm administrator's Group. You can make this user a farm administrator by adding them individually, or adding a group they are in. Either way works. Now verify you're logged into Central Admin as this user. You should see this in the upper right hand corner:
If you see "Welcome System Account" instead you're logged in with the wrong account and the problem won't exist.
Next go to Central Admin > Application Management >Web application general settings. Make sure you take note of which Web app's general settings you're changing. I recommend your portal or main content web app for this demonstration. In the Default Quota Template area select a quota as the default. If none show up in the dropdown go to Central Admin > Application Management > Quota templates and create one. The name and the sizes don't matter. Put anything in there. Now go back to Application management and click Create Site Collection. Make sure you're creating this site collection in the same web app that you just assigned the quota to. You can put whatever you'd like as the title, though I've found "Todd is cool" seems to work the best. Same goes for the URL of the new site collection. The important part is that you assign the Jill account and only the Jill account as a site collection administrator. DO NOT put the Todd or SP_Farm accounts as an administrator. Verify at the bottom of the page that a default quota is being applied. You page should look something like this:
I've pointed out the parts that are most import. It's important that a non privileged account is the only administrator, and that a default template is being applied. When you hit Ok to create the site collection you should be greeted with this:
Huh? Todd is a Farm Admin, he should be able to create site collections. What gives?
Well, honestly I don't know what gives. I've pored through the ULS logs and I have yet to determine exactly what Todd doesn't have access to. Here's what I do know, if I hit the back arrow and add Todd as a site administrator, the error goes away. I also know that if SP_Farm tries to create a site collection with just Jill as an administrator, it works. I also know that if I go back to the web app general settings and remove the default quota template, now Todd can create Jill's site collection. I have no idea what triggers that error. I have confirmed it on SP2 (build 6421) and the June CU (build 6510). I also know you get this behavior whether you use Central Admin or STSADM.
A couple of other notes. It seems the site collection creation process gets started, but is interrupted before it can complete. You'll notice the site collection you tried to create does not exist if you try to browse to it, or list the site collections in Central Admin or with STSADM. However, enough of it is written to the content database that if you detach the content database it was going into and reattach it, the site collection is now there. It doesn't have a template, but it's there. Go ahead, try it, I'll wait…
This is because the entries for the site collection are created in the ContentDB's Sites table, but are never written to the ConfigDB's sites table. When you attach a ContentDB SharePoint walks through its Sites table to see what site collections exist in it, and it adds them to the ConfigDB's Sites table. I mention this because your Site Collection count will jump if you ever detach and reattach this ContentDB. Also, it doesn't matter what quota template is picked on the site collection page itself, it only matters if a default has been chosen for the web app. If you DO have a default chosen but set it to "None" when you create the site collection, it still fails. If you DO NOT have a default chose, but chose a quota at site collection creation, it does not fail. I have no way to explain this.
While this problem is completely nonsensical that's not what was the most surprising to me about it. I'm surprised that after three years of using the product I've never stumbled across this. It's the meeting of several best practices I preach. I tell people to always set quotas from the beginning, even if they think they'll never need them. I tell people not to use the SP_Farm account for day to day tasks. I tell people to put the user in charge of the site collection and not to make IT manage things at that level. Somehow though, those things have never converged to create this problem before this week. Weird. I did find one forum thread about this problem as I researched. When I scrolled down and saw that one of the guys trying to help the poster was me, I knew he and I were both in trouble. J
This doesn't seem to be a big problem, and the workarounds are easy. Any one of the following things will get you around this issue.
- Don't set a web app default quota
- Create site collections as the System Account
- Add your farm user as an administrator when you create the site collection, then immediately remove them
Any of those will get you around this. I spent the better part of four days trying to get this figured out. I hope this blog post helps at least one person and saves them some time and frustration.