Russ Houberg
Senior Technical Architect, MCM
KnowledgeLake, Inc.
RBS Background
Implementation Guidance
Framework Components
Process Flow
Required Resources
Install & Configuration
Migrate()
Demo
Maintenance
Impact on Scalable Architecture
Remote BLOB Storage (RBS)
• By default SharePoint stores Binary Large Objects (BLOBs) in
the content database
• When enabled… Intercepts binary content (documents) and
sends them to a BLOB store
• SharePoint assemblies function as a client that implements a
base provider in the Microsoft.Data.SqlRemote assembly
• 3rd Party providers use the Microsoft.Data.SqlRemote
assembly to construct RBS providers that are “loosely”
invoked by SharePoint in order to store binary data in a BLOB
store
• Microsoft provides the “local” FILESTREAM provider to allow
for usage of the SQL Server local NTFS file system as a BLOB
store.
SharePoint
2003
SharePoint
2007
SharePoint
2010
Enabling RBS allows BLOB data to be stored on less
expensive storage
• Weigh the value of the storage against the administrative and
performance overhead of RBS. Enable RBS when:
• Content Database is expected to be larger than 500 (GB)
• BLOB data files are larger than 256KB on average
• BLOB data files are at least 80KB and the DB server I/O is a bottleneck
• Many small BLOBs can decrease performance
• Content Database size is likely to be more of a decision factor than
minor increases in retrieval latency
• RBS is most beneficial with very large files such as many page
PDF or TIF files or digital media.
• RBS is typically implemented in which large files are
infrequently accessed, such as an archive.
Enabling RBS allows a system that is specifically
designed to work with BLOB data to manage storage
• Write Once Read Many (WORM) device can raise an
exception in the RBS Provider that prevents the BLOB related
SharePoint metadata row from being deleted
• High end SANs are able to replicate BLOB content to another
(mirrored) device over a WAN using sophisticated bit
mapping (delta) technology. May be possible to combine
with log shipping or database mirroring to provide DR
solution.
• Maintainer is key to identify orphans in the database or the BLOB store
after failover
Consider the following when selecting an RBS Provider
•
•
•
•
•
•
Backup and restore capability
Tested disaster recovery
Deployment and data migration
Tested performance impact
Long-term administrative costs
V.Next SharePoint Upgrade Support
SQL RBS Add-in “Framework” (rbs_x64.msi)
• Provides “BlobStore” Abstract Class
RBS Provider Library
• Managed library that implements the BlobStore abstract class
in order to store and retrieve content in the BLOB store.
RBS Client Library
• Invokes the BlobStore class in the Enabled/Active Provider in
order to send to and request data from the BLOB store.
SQL Database RBS Objects (tables, stored procs, etc)
• Stores BlobId and BlobReference values and provides
resources for managing BLOB pointers
Application (SharePoint 2010)
• Invokes an RBS Client in order to send data to and request
data from the BLOB store
7. Back to User
2. Enforce
Business
Logic
1. Save Request
3. Save Blob
5. Return BLOB ID
6. Save Metadata &
BLOB ID
4. Write Blob
SharePoint Foundation 2010 (minimum)
SQL Server 2008 R2
• Any Version
• SQL Express R2 (Supports 10GB Database Size)
• NOTE: Microsoft SharePoint Foundation installs with SQL Server 2008
by default (not SQL Server 2008 R2)
FILESTREAM RBS Provider / Framework
• http://go.microsoft.com/fwlink/?LinkId=177388
The FILESTREAM provider is supported by SharePoint
Server 2010 when it is used with SQL Server 2008 R2 or
SQL Server 2008 R2 Express.
• Only “local commodity storage” (hard drive) is supported.
• Direct Attached Storage (DAS), Network Attached Storage
(NAS), and Storage Area Network (SAN) are all considered to
be “remote commodity storage” and are not supported by
SharePoint 2010.
Any other 3rd Party RBS Provider is considered to be a
“remote server” provider and SharePoint 2010
licensing requires that SQL Server 2008 R2 Enterprise
Edition be implemented.
NAS as BLOB Store storage target
• 20ms maximum latency to first byte on retrieval
The FILESTREAM provider…
•
•
•
•
•
Is limited to SQL Server local storage only
Does not support document encryption
Does not support document compression
Does not support SQL Server 2008
Pool sizes can only be configured by editing Machine.Config
Create master key in content database
• Used by RBS security model.
• Master Key should be BACKED UP and stored in secure, offsite location!
Install RBS_x64.msi on SQL Server and all WFEs
• Special command line parameters for FILESTREAM RBS
provider will execute RBS stored proc on content database
Prepare Content Database (other providers)
• C:\Program Files\Microsoft SQL Remote Blob Storage
10.5\Server\RemoteBlobEnable.sql
• Creates rbs_sp_enable_rbs stored proc
• Execute rbs_sp_enable_rbs stored proc (builds out all RBS resources in
content database)
If not using FILESTREAM provider
• Execute RBS Provider installer
Use PowerShell to activate provider
$site = get-spsite [http://siteurl]
$rbss = $site.ContentDatabase.RemoteBlobStorageSettings
$rbss.Installed()
True
$rbss.Enable()
$rbss.SetActiveProviderName($rbss.GetProviderNames()[0])
$rbss
Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties
----------- --------------------------- --------------------------------- ------------------------------------True FileStoreProvider_1
0 {}
Use PowerShell to EXTERNALIZE all content
$site = get-spsite [http://siteurl]
$rbss = $site.ContentDatabase.RemoteBlobStorageSettings
$rbss.Migrate()
[command completes when finished]
[can be stopped and restarted any time]
Use PowerShell to INTERNALIZE all content
$site = get-spsite [http://siteurl]
$rbss = $site.ContentDatabase.RemoteBlobStorageSettings
$rbss.SetActiveProviderName(“”)
$rbss.Migrate()
[command completes when finished]
[can be stopped and restarted any time]
$rbss.Disable()
$rbss
Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties
----------- --------------------------- --------------------------------- ------------------------------------False
0 {}
RBS Maintainer (Garbage Collection Tool)
• Provider Agnostic
• Reference Scan
• Compare BLOBIDs in application tables with BLOBIDs in RBS tables
• Finds BLOBs that are no longer referenced
• Delete Propagation
• Mark BLOBs for deletion
• Delete the BLOBs from the BLOB store
• Orphan Cleanup
• Enumerate BLOBs in the BLOB store
• Remove any BLOBs that are not known to RBS
RBS Maintainer (continued)
• Can be interrupted if necessary
• Defaults to 30 day delete window
• Can override for immediate delete with stored procedure call
• Sample maintainer command:
Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName
RBSMaintainerConnection -Operation GarbageCollection
ConsistencyCheck ConsistencyCheckForStores -GarbageCollectionPhases
rdo -ConsistencyCheckMode r -TimeLimit 120
RBS Installation and Configuration
Backup and Restore Order
• Backup
Backup Start
Both Backups Complete
SQL
BLOB Store
• Restore
Restore Start
Both Restores are Complete
BLOB Store
SQL
Library Size vs Content Database Size
• 10 Million to 30 Million Documents
• Item metadata ~5 – 7KB
1.5 Million
10 Million
20 Million
30 Million
DB Size @ 5KB
7GB
48GB
95GB
143GB
DB Size @ 6KB
9GB
57GB
114GB
172GB
DB Size @ 7KB
10GB
76GB
153GB
229GB
For archive scenarios, break up content by year or, if
necessary, month
• Leverage Content Organizer Rules to route content to the
right site collection
Microsoft has a solid long term solution to BLOBs in the
database
RBS Providers guarantee persistence
We can migrate existing content that is either in-line or
externalized with EBS.
Reduces Content DB size and facilitates large
repositories.