Skip to main content
Trans Am

Todd Klindt's SharePoint Admin Blog

Go Search
Home
Blog
Netcast homepage
SharePoint Terminology Wiki
  

Todd Klindt's home page > Todd Klindt's SharePoint Admin Blog > Posts > Installing Remote Blob Store (RBS) on SharePoint 2010
Installing Remote Blob Store (RBS) on SharePoint 2010

In my daily perusing of SharePoint blogs a couple of weeks ago I saw a blog post on Jie Li's blog on how to install Remote Blob Store (RBS) on SharePoint 2010. Imagine my excitement! I was under the impression that using RBS was going to be something we were mortals would not be able to use, but here it was. Why was I so excited? Well, with SharePoint 2003 all content had to go in content databases, no exceptions. As people started putting more content, and larger content into SharePoint the question became pretty common, "Can I store my large files outside of SQL?" The answer was always no. Storing files outside of SQL became the holy grail of SharePoint content storage. It would let you take advantage of all the reasons you used SharePoint; metadata, web presentation, workflows, etc. It also let you use storage that was much cheaper than SQL storage normally is. When SharePoint 2007 came out the answer to that question remained the same, "All content must be stored in SQL, no exceptions." However, after a hotfix the support for External Blob Storage (EBS) was added to SharePoint. This was almost what we needed. The hooks were there in SharePoint, but there was no way to actually use them. It was handled much like how SharePoint handles antivirus software. There were hooks into SharePoint for it, but the functionality itself didn't exist in the product. That part had to be handled by third parties or ISVs. Not a lot of ISVs jumped on this, as the interface exposed to them wasn't great, and it wasn't clear what the upgrade path would look like. Again, storing large files outside of SQL seemed just out of reach for the average SharePoint administrator, until now…

That brings me back to Jie's post. I knew that SharePoint 2010 would ship with RBS support, but I wasn't sure what RBS products we'd be able to use. Jie's post not only showed how RBS could be used, but provided an actual RBS provider. That's was pure gold. The RBS provider Jie linked to uses SQL 2008's Filestream support. That works for me. The rest of this blog post is how I installed RBS and got it working.

Again, everything I've done here is based on Jie's blog post and the links inside. Only a couple of things are things I had to figure out myself.

I already had a VM with Windows 2008 R2, in a domain. It had SQL 2008 R2 with the November CTP on it. It also had SharePoint Beta 2 (build 14.0.4536.1000) installed on it, and I had been using it a while. There were several content databases defined with a few site collections. The first thing I had to do was enable Filestream in SQL 2008 R2. I used these instructions on MSDN. Once I had the instructions, it went very smoothly. The only snag I had was I initially ran it as the wrong user. In my VM, SQL is running as contoso\administrator. SharePoint is all running as contoso\sp_farm and that's the account I logged in as normally. That account did not have the permissions to enable Filestream. After I ran it all as Administrator it worked. Now came the fun part, RBS…

In preparation for RBS I created a new Content Database for it to use, since RBS is scoped at the content database level. I used Window PowerShell to do it, since that's what the cool kids are using.

The command I used was New-SPContentDatabase –name WSS_Content_Blob –WebApplication http://shaerpoint –MaxSiteCount 1 –WarningSiteCount 0

Next I went into SQL Management Studio, as contoso\sp_farm, and ran a few TSQL commands to configure my new database for RBS. Here are the commands I used:

Here's the text:

use [WSS_Content_Blob]

if not exists (select * from sys.symmetric_keys where name = N'##MS_DatabaseMasterKey##')create master key encryption by password = N'Admin Key Password !2#4'

 

use [WSS_Content_Blob]

if not exists (select groupname from sysfilegroups where groupname=N'RBSFilestreamProvider')alter database [WSS_Content_Blob]

add filegroup RBSFilestreamProvider contains filestream

 

use [WSS_Content_Blob]

alter database [WSS_Content_Blob] add file (name = RBSFilestreamFile, filename = 'c:\Blobstore') to filegroup RBSFilestreamProvider

 

I was following the directions from this MSDN post, but I had to make a couple of changes. First, I had to make sure I had the correct database name, WSS_Content_Blob, everywhere necessary. Second, I manually created c:\Blobshare before running the TSQL commands. That was a mistake. SQL needs to make it on its own when you run the third command. Deleting c:\Blobstore fixed that and the command ran successfully.

Next step was to install the RBS bits on the SharePoint servers. (Download RBS_X64.msi) In my case it was pretty simple because SQL and SharePoint were on the same machine, and I only had a single SharePoint server. Here's the command I used to install the RBS bits:

Text:

msiexec /qn /lvx* rbs_install_log.txt /i RBS_X64.msi TRUSTSERVERCERTIFICATE=true FILEGROUP=PRIMARY DBNAME="WSS_Content_Blob" DBINSTANCE="sharepoint" FILESTREAMFILEGROUP=RBSFilestreamProvider FILESTREAMSTORENAME=FilestreamProvider_1

This install seemed to happen immediately. However it kicked off an msiexec service that ran for a couple of minutes. I had to watch it in Task Manager to see when it was finished. I was also monitoring the rbs_install_log.txt file to see when the process was finished. I was looking for the phrase "Installation completed successfully."

If I would have had more SharePoint servers I would have had to install that on all of them, or SharePoint would have been very unhappy.

You can double-check the database was changed correctly if the rbs tables appear in it. That will look like this in SQL Management Studio:

The last step, before feeling the glory that is RBS, is to go back to our old friend, Windows PowerShell and enable RBS on our content database from a SharePoint perspective. Here's what the PowerShell looked like:

Here's one place I had to deviate from the MSDN article. It says the first line should be $cdb = Get-SPContentDatabase –WebApplication http://sitename That didn't work for me. I suspect the person writing that article had a web application with a single content database, so that command would only return one SPContentDatabase. The web application I was testing this on had three content databases, so $cdb had a collection of SPContentDatabases, which doesn't work for the next line. Reading through the PowerShell it looks pretty clear that $cdb should contain a single SPContentDatabase object. I went through the RBS install three times and had the same behavior each time.

The last step was to test RBS and upload a file and see where it ends up. The rbs_install_log.txt was 1.2 MB and a good candidate for the test. I created a site collection (with PowerShell of course) and uploaded that text file. To see if RBS worked, I navigated the c:\blobstore directory to see if my file was there. Here's what I found:

Looks like it was successful. I went through this three times and it worked every time. I tried various sizes of files and various operations on those files. I added versions, deleted them, added metadata, indexed and searched for them. It all worked just fine.

I hope this blog post inspires some of you to install RBS and play with it yourself. I do have another blog post planned where I go a little more in depth with what I found in RBS.

Have fun,

tk

Comments

Question about deletion (maintainer with RBS)

Todd,

Great article, thanks. I have been trying to find information on how the maintainer and SharePoint delete operations work. In EBS it was full responsibility of the provider developer to find BLOBs no longer referenced in SharePoint and delete them from the storage. However the RBS documents and blogs refer to the maintainer doing the actual garbage collection work. Up to what point it will be internal to SharePoint/SQL to find deleted objects and when is that list passed to the provider? Does the provider have to go and find it or it might be a new RBS method or API, or something that will request delete of a list of BLOBs to the provider? IT was partially mentioned in the SharePoint conference, but I was expecting to find more concrete info in it. I have even found some blog references about a provider VETOING a delete operation or supporting updates.
Do you have any idea where I can find information about this?  
Please if you have some reference or info send me an e-mail to rodrimf@hotmail.com.
Thanks.
at 1/11/2010 12:52 PM

EBS for now RBS later

Hi Todd,

Right now our environments have MOSS 2007 and SQL 2005 back end so I'll need to either find an EBS method now.  or

This gives me more reason to look forward to SharePoint 2010 and helps make the case for SQL2008 as we get ready to make that move.

Thank you for the post.  Insightful and helpful as always.

at 2/25/2010 2:04 PM

Interesting

I like how this will be available for Foundation.  It opens the door to split a web app we have off of our Sharepoint farm that's hosting some large files in the DB and utilize the blob functions someplace else. 

eric

at 2/25/2010 2:19 PM

Filesystem Structure

Very useful post.  I'm going to be testing this right away.

In the meantime, could you possibly post some screenshots of what the content of the blobstore looks like when you upload files into it?  Is it GUID gobbledygook or is there some readability to it?

Thanks!
at 3/5/2010 12:01 PM

Re: Filesystem Structure

The last screenshot shows the structure. It's all GUIDs.

tk
Todd O. Klindt at 3/6/2010 5:15 PM

Great installation guide!

I'm one of the developers of RBS and wanted to drop in and thank you for putting together a great setup guide and to answer a couple of questions in the comments.

1. The RBS Maintainer process runs as a seperate process via a scheduled task and handles all the garbage collection algorithms. The provider simply needs to implement a delete blob and delete pool method. We also have an optional callback that can be used for provider specific maintenance tasks if that particular provider needs something custom done.

2. There's no plans to allow providers to support updates. Even though the underlying blob store might physically be able to support updates RBS has a requirement that blobs are immutable to ensure our consistency guarantees and to support metadata restores within the SLA time limit. There's more info on GC and the various settings at our blog, http://blogs.msdn.com/sqlrbs/.

3. The GUIDs used by the blobstore are the SQL Filestream implementation. This is wholly a provider specific path and unrelated to the general RBS architecture, although the Filestream provider is the default provider supported by Microsoft.

Once again, thanks for a great post and please let us know if you need any assistance or clarification in the future.

- Michael
at 3/16/2010 11:45 AM

Awesome guide ...but I have an issue

Hi Todd
Thanks for a wonderfull post. I followed your post exactly and was able to install the RBS bits on SharePoint 2010 server.
I verfied the log file and it did had the line
"MSI (s) (6C:B8) [22:11:46:374]: Product: SQL Remote Blob Storage -- Installation completed successfully."
But when I double checked the content database no tables were found for rbs?
Even after running the command $rbss.Installed() I got a False result

Any body faced similar issue?
- Steve

at 4/1/2010 12:56 AM

Re: Awesome guide ...but I have an issue

I guess I'd read through the whole rbs_install_log.txt file and verify everything went well and that you didn't specify the wrong database name or something.

tk
Todd O. Klindt at 4/1/2010 10:14 AM

Awesome guide ...but I have an issue

Thanks Todd

Finaly figured out the issue. While enabling FILESTREAM support for MS SQL Server in SQL Config. Manager, Windows share name for enabling I/O streaming was accidently misspeled. I had to reanable it and make it same as SQL instance name.
at 4/5/2010 2:38 AM

Re: Awesome guide ...but I have an issue

Thanks for the followup. I would have never guessed "did you mispell your share name?" :)

tk
Todd O. Klindt at 4/5/2010 3:35 PM

100GB Recommendation

The recommendation has always been to try and keep content databases under 100GB.  Does this virtually eliminate this concern?  If the only thing stored in the database is indexes and metadata, it seem highly unlikely anyone would ever reach 100GB.  Also, do recommend storing all content types in RBS or just large files like CAD drawings and videos?
at 4/29/2010 3:22 PM

Cannot enable RBS on more than one Content Db

Hi Todd,

I followed your post exactly and was able to install the RBS bits only on one Content Db. But I wanted to enable RBS on more than one Content Db.

In step 2, I changed the Content Db name and run the command. The rbs_install_log.txt file shows the phrase "configuration completed successfully" but the SQL query "select * from dbo.sysobjects where name like 'rbs%'" doesn't show any values and also the Installed() method on RemoteBlobStorageSettings object shows 'false'.

Is enabling RBS restricted to one Content Db in the SP 2010 farm?

-Imran

PS: I am using SharePoint 2010 RTM.
at 5/6/2010 5:39 AM

Re: Cannot enable RBS on more than one Content Db

I haven't enabled RBS on more than one Content DB, but everything I've read says it should be possible. I don't have a machine with RBS on it at the moment. I hope to have one set up in the next week or so. When I do I plan on going over these instructions and making sure they still work for RTM. I'll make sure and test multiple Content DBs when I do it.

tk
Todd O. Klindt at 5/11/2010 9:55 AM

Storing BLOBS on another server?

Great stuff guys. It worked well for me. I do have a question, when FILESTREAM sets up the share it always puts it on the C:\ driver, i.e. -c:\blobstore. Is it possible to have the blobstore on another server?
DavidH
at 5/18/2010 2:01 PM

Re: Storing BLOBS on another server?

You can't do that with the Filestream provider, it requires local storage on the SQL server. However, RBS is just a framework, so blob storage on another box might be possible with another RBS provider.

tk
Todd O. Klindt at 5/18/2010 11:01 PM

RE: Storing BLOBS on another server?

Are there other providers out there that are supported by MS for Sharepoint? or that are known to play well with SP2010? I did a quick google search but didn't come up with anything, just curious.

Thanks,
DavidH
at 5/19/2010 5:17 AM

Enabling RBS on more than one ContentDB

Hi David,

Were you able to enable RBS on more than one content DB? If yes, did u do anything different than what is given here? can you please share with us.

PS: I am trying to enable RBS on more than one content DB on SharePoint setup which has SharePoint 2010 RTM and SQL server 2008 R2 RTM server, I am not seeing any success yet.

TK, I am still waiting for your reply on this. Hope you do the test within in this week.

-Imran
at 5/19/2010 9:04 AM

Re: Enabling RBS on more than one ContentDB

You'll be happy to know that I've gone through all these steps for RBS on my RTM farm. I'll configure it on a second database hopefully today. I'll add that and some other gems to a new blog post. Stay tuned. :)

tk
Todd O. Klindt at 5/20/2010 9:15 AM

RE: Storing BLOBS on another server?

RBS is just an underlaying technology, so I'm not sure vendors will use that phrase when describing their products. I believe Avepoint has a product that uses RBS called DocAve Extender. It might do what you're looking for.

tk
Todd O. Klindt at 5/20/2010 9:26 AM

more than one blobstore?

Is it possible to have more than one blobstore? I am thinking I might like to have a blobstore for each content database, is this possible or do all the blobs have to be stored in the same place?
at 5/20/2010 9:41 AM

re: Enable RBS on more than one ContentDB

Imran: I have the same setup you do, i.e.-RTM, I was successful in associating different site collections with different content db's but I was not successful in trying to enable RBS on more than one contentDB.

tk: If you get this working, yes, do, please let us know how you did it.

thanks guys,
DavidH
at 5/20/2010 10:50 AM

Answering some questions from the comments...

Currently the Filestream provider is the only provider that Microsoft is shipping and supporting. A few other companies have providers or are building providers as well.

The Filestream provider setup has options to specify a Filestream filegroup to use, this is where you can configure the location of your blobs. The Filestream feature requires locally attached storage so you can't use a network connection for this.

The Filestream provider will determine where the best location is to store a blob based on it's size, so it will automatically store large files in Filestream and smaller files locally in varbinary(max).

It's possible to have multiple blob stores installed on a single RBS instance. In practice it's difficult with SharePoint to specify anything other than a default provider for each content database, so the recommended solution is to install the server component of RBS on each content db seperately. Different content databases can then have different providers.

Looking forward to future blog posts Todd.

- Michael
at 5/20/2010 11:02 AM

Re: Answering some questions from the comments...

Thanks for clearing some of that up Michael.

tk
Todd O. Klindt at 5/22/2010 10:46 AM

re: Enable RBS on more than one ContentDB

I experienced the same trouble you guys did enabling RBS on the second content database. The RBS_X64.msi installer isn't preparing the second database. I'm looking into it. Hopefully I'll have some more information next week.

tk
Todd O. Klindt at 5/22/2010 10:47 AM

re: Enable RBS on more than one ContentDB

Thanks Todd, it is consoling to know that I am at least not the only one who was running into problems.

Please do let us know what you find out.

Thanks guys,
DavidH
at 5/24/2010 9:46 AM

Named pipes

Hi Todd
One of your contribrtor - Steve - had a question as to why he was getting this:

"I verfied the log file and it did had the line
"MSI (s) (6C:B8) [22:11:46:374]: Product: SQL Remote Blob Storage -- Installation completed successfully."
But when I double checked the content database no tables were found for rbs?
Even after running the command $rbss.Installed() I got a False result

Any body faced similar issue?
- Steve

Answer - I too had the same problem and trawled through the yards of log data only to find an earlier error message suggesting it could not contact the SQL server at all (so how could the install be successfull?) it indicated that it needed Named Pipes to be enabled.  I duly followed this advice on the appropriate SQL 2008 R2 instance:
SQL Server Configuration Manager
SQL Server Network Configuration
Protocols for <instance name>
Right click Named Pipes
left click Enable

Then re-ran the install and the SQL database was duly configured.
at 6/9/2010 4:59 AM

Named Pipes - ps

btw
I should have signed off the last entry!

Good luck
Stephen Wilder
at 6/9/2010 5:00 AM

Filestream impersonation

Hi Todd
Thanks for a great post - I found it essential when setting up RBS for the first time.

I have just discovered another problem/issue not covered in your blog which I would like to share - (the problem and a possible solution!)

Having followed your instructions to the letter I got to the part where you try it out for real in a real world environment!! (eg separate web front end and SQL back end servers as a minimum)

I tried to upload a small (64kb) document to the Shared Document library - no problem it worked as it should - I tried to upload a larger (2Mb) document and it failed saying the document URL was invalid.

Trawling through the log files (btw the new correlation ID works a treat - well done MS) I came across a Filestream error message indicating an invalid impersonation.  Further Googling revealed a SQL server forum (http://social.msdn.microsoft.com/Forums/en/sqldatabaseengine/thread/50ea9248-6edd-4f6a-bbc3-b1d119849341) discussing this very point (but not related to SharePoint) and suggesting the issue could only be resolved by making sure all the servers involved in this dialog are using the same version of Windows server!  In my case I was using Windows 2008 standard x64 on the web front end server and Windows 2008 R2 Standard on the SQL back end server.

I have just gone through the laborious process of synchronising the 2 ;-(

....and the good news is that RBS now works!!!!

In the meantime has anyone else had similar problems and do our friends at MS have any comments/suggestions how to resolve this problem other than syncnig the OS versions?

Thanks for a great post Todd.

Rgrds
Stephen Wilder
at 6/9/2010 6:18 AM

What's the purpose of setting the DB Master encryption key?

Does the RBS provider specifically require it?
at 6/17/2010 1:31 PM

Documents are not Stored in the FileSystem

Hello,
I have follow all the steps given in this blog. It will Installed and Configure Successfully.

But when I upload the document it is not stored on the FileSystem 'c:\BlobStore'. It is stored in the ContentDB.

I execute these steps so many times but not able to achieve...

I have Win2k8 server 64 Bit R2 and SQL 2008 on the same machine.

Can you please help me to resolve the issue.
at 7/6/2010 2:40 AM

Re: What's the purpose of setting the DB Master encryption key?

I'm not sure why the encryption key is needed. I'll see if I can find out.

tk
Todd O. Klindt at 7/6/2010 9:55 PM

Re: Documents are not Stored in the FileSystem

There are a lot of steps, so this is really tough to troubleshoot via blog comments. Does your content DB have all the RBS tables? If you look at its properties does it show the RBS filegroup? Have you verified with PowerShell that RBS is enabled? How large is the file you're uploading? It should be over 1 MB.

tk
Todd O. Klindt at 7/6/2010 9:57 PM

Need tips on debugging

Hi Todd,

Do you have any tips to find out what the problem could be when the tables are not created after installation of the msi package?

log file shows it was successfull but clearly this was not the case.
Log file shows an unexpected error (code 2769) Custom Action CreateFilesNoUI did not close 21 MSIHANDLES.

Thx
at 7/14/2010 4:45 AM

e: Documents are not Stored in the FileSystem

H Todd,
I have verified the RBS using PowerShell, It is enabled.
I have specified the file more than 1MB.

I have found only one thing wrong...
When I open a SharePoint Site Administration. I have seen that the site is still pointing to old content DB (Default ContentDB created when we create a Site Collection). and not pointing to new ContentDB that is created by us using PowerShell. (WSS_Content_Blob)

Will this information help you to understand the problem..

Thanks,
TusharG
at 7/14/2010 12:46 PM

Store documents in file system instead of database in SharePoint document library along with the  meta data of the documents

Hi Todd,

I have to work on Store documents in file system instead of database in SharePoint document library along with the  meta data of the documents.

How can i achieve this task? I have implemented the ISPExternalBinaryProvider interface but it doesnt seems that its working fine and also not able to store the meta data of the document.

Please provide me the solution
at 7/20/2010 1:25 PM

Add Comment

Items on this list require content approval. Your submission will not appear in public views until approved by someone with proper rights. More information on content approval.

Title


Body *


Today's date *

Please enter today's date so I know you are a real person
Attachments