Getting Comfortable with the new hybrid Cloud Search Service in SharePoint 2013
At Ignite in May Microsoft took the lid off of one of the coolest new SharePoint features I’ve seen in a while, even cooler than FIM! It’s the Cloud Search Service Application, or CSSA. The CSSA is the next evolution of hybrid search between on-premises SharePoint 2013 (and 2016) and SharePoint Online. Before the CSSA we had the clumsy option of federating search results between the two, but they were separated, and it was confusing to users. You know what confused users are? Annoying, that’s what. The CSSA combines the two sets of search results in a seamless, non-user confusing way. And with your on-prem content in the Office 365 Index, it can be used in cloud only tools like Office Delve. I’ve been doing some friendly battle with the CSSA since it made its debut in August of 2015. In this blog post I’ll walk through the setup and configuration. More importantly I’ll show you all the places I screwed up, so you don’t have to, if you don’t want to.
In order to use the CSSA you have to have a few things in place. First, you need an Office 365 tenant, obviously. It has to be a Business tenant, a Personal one won’t do. If you don’t have an Office 365 Business tenant, or you don’t want to experiment with the one you have, (testing in Production is bad, mmkay?) you can get a trial Office 365 Business tenant. That will work fine for the tests, and the trial should last long enough for you play with the CSSA and learn a few things the hard way.
Next you need a SharePoint 2013 on-prem farm. It should not be a Production farm, and again, a trial farm will do. It needs to be patched to the August 2015 CU or later. A SharePoint 2016 farm will also work, though all of my testing has been on SharePoint 2013, so the SharePoint 2016 experience might be a little different than how I explain it here.
Finally, you need to have Directory Synchronization set up between your on-prem Windows AD and Azure AD. The tool for that is Azure AD Connect. You’ll need to install it and configure it. There are a few options on which users you sync to Azure AD. One thing to keep in mind is that every user that does queries against the CSSA must be synced to Azure AD and licensed in Office 365. There might be situations, like hybrid OneDrive for Business, where you choose to only sync and license a subset of users. That’s fine in those situations. For CSSA to work, the user has to be synced and licensed. If they aren’t, you won’t get any friendly messages reminding you of this necessity. Instead, those users just won’t get any search results, even though you’re super duper positive they should.
Creation and Configuration
After you have everything prepared, it’s time to create your CSSA and configure it. It’s important to note that you cannot convert an existing Search Service Application (SSA) to a CSSA, you must create a new one. In most cases you’ll have a single SSA in your farm. You’ll have to copy out any existing settings and content sources and delete it. The CSSA Connect site (request access here)has a script you can use to create your CSSA, but any existing PowerShell script should work, with one change. When you create the Search Service Application with New-SPEnterpriseSearchServiceApplication you need to use a new parameter and add –CloudIndex $True. Otherwise, it’s all the same. A SSA created in Central Admin will not be Cloud enabled, so it won’t work. You must create the CSSA with PowerShell.
You can use the following PowerShell to verify the CloudIndex bit was set correctly:
Get-SPEnterpriseSearchServiceApplication | Select CloudIndex
After the CSSA is created, Search still won’t work. The interface looks the same in Central Admin, and you can do a fair amount of configuration of it, but it can’t successfully crawl anything until it’s onboarded to Office 365. When you create a CSSA it stores its Index entirely in the Cloud, there is no provision for keeping the Index on-prem. The Azure plug-in is inserted into the Crawler, so it won’t even try to save the Index locally.
After the CSSA is created you need to connect your on-prem CSSA to your Office 365 tenant. The onboarding script requires the Microsoft Online Sign in Assistant and the Microsoft Azure AD PowerShell module. Download and install each of them. After they’re installed, run the onboarding script from the Connect site. You’ll need to pass it the URL of your Office 365 tenant. The command will look something like this:
.\OnboardHybrid-Search.ps1 –PortalUrl https://contoso.sharepoint.com
The script gives pretty good feedback, so you’ll know if everything was bolted together correctly. In the unlikely event that something does go sideways, you can fix the issue and just run the script again to onboard, it cleans up after itself pretty well.
After your CSSA is onboarded to Office 365 you need to create content sources for all that yummy SharePoint content you have. This process is exactly the same as it’s been for SharePoint for the last 10 years. The CSSA doesn’t change that at all. If you need to change your Content Crawl Account, now is the time to do that. After you’ve added your content sources go ahead and crawl them and make sure you don’t get any Errors or Top Level Errors.
Getting Some Results
Once you have some content in your index, your first instinct is to do a query and admire your handiwork. I can understand, these are exciting times and you want to make sure it’s all real. There’s two more steps to perform before you’ll get the satisfaction you’re looking for.
First, you need to make sure that all your users are licensed in Office 365. All queries are now going through Office 365, so before Office 365 will give them any information, they need to synced to Azure AD and licensed. Just so it’s clear, even if a Windows AD user is browsing an on-prem SharePoint site searching for on-prem content they have permission to, there will still be no results if the user is not licensed in Office 365. In theory you could work around this by having a second SSA and have non-licensed users search there. That sounds like a whole lot of work though.
It’s also important to note that any permissions that are assigned to Domain Users will not find their way to the Office 365 Index. This means that if you have content (SharePoint, file shares, whatever) where permissions are given only to Domain Users, that content will not show up in search results. This isn’t SharePoint’s fault. Azure AD Connect does not sync security principals that have the IsCriticalSystemObject attribute set to True. The Domain Users has that. Since the group is never synced up to Azure AD, the indexer can’t tell that a user has permission to it, so it won’t return it in search results. There are a couple of workarounds. You can create a new security group in Windows AD that has everyone in it and make sure it’s included in the Azure AD sync. You can also give Everyone or “All Authenticated Users” permission instead of Domain Users. This maps to “Everyone Except External users,” which while not ideal, should work.
Second, you need to change your on-prem Search centers to provide search results from a new Result Source. Even though there’s no local index to give results from, that’s what the CSSA will do without your intervention. All you really need to add is the URL to your Office 365 tenant. The new Result Source will look like this:
Here’s where it shows up in the Search Service app:
After the users and licensed and SSA is set to use Office 365 as a Result Source, you should be ready to go. Both Office 365 and SharePoint On-Prem use the same Index, so the results should be the same, but be sure to verify that.
The CSSA adds a new managed property, IsExternalContent, that you can use to differentiate between content crawled from on-prem sources and content from Office 365. It tells you which search results are External to Office 365. This can be used to create custom search scopes, and it can also be used for troubleshooting. Adding IsExternalContent:1 to a query, executed in SharePoint Online or an on-prem Search Center, will return only results from SharePoint on-prem content sources.
Setting that to 0 returns only results from Office 365. Here’s the same query, with IsExternalContent set to 0.
Hopefully this has helped you get the new Cloud Search Service Application installed and configured on your SharePoint 2013 or SharePoint 2016 farm. Here are a few articles I used to help me on my journey:
Cloud Hybrid Search Service Application
Implementing Next Generation SharePoint Hybrid Search with the Cloud Search Service Application
I would also like to give a shout-out to Neil Hodgkinson at Microsoft. He helped me with a couple of things I might have been screwing up. J