• Missing metadata issues
• Inconsistent metadata issues
• Cleansing of content
• Integration of semantic layers/Automatic tagging
• Integration with 3rd party service
• Merging data from other sources

Content Enrichment Web Service in SharePoint 2013 is a SOAP-based service within the content processing component that can be used to achieve this. The figure below shows a part of the process that takes place in the content processing component of SharePoint search.

Content Enrichment Web Service SharePoint 2013 combines the goodness of both FAST for SharePoint Search and SharePoint Search  to offer a whole new set of possibilities and has its own challenges. To see an implementation example, check the MSDN link which pretty much sums up the basic steps. In this post we are going to look at some of the advantages and challenges of CEWS coming from a FAST 2010 background:

1. CEWS is a service and you DON’T have to deploy it in your SharePoint environment: Perhaps this is the biggest architectural change  from the content processing perspective. What this means is that your code no longer runs in a sandbox environment within SharePoint Server. The webservice can be hosted anywhere outside your SharePoint server thus reducing deployment headaches and huge number of approvals required to deploy the executable files. I can see operations/infrastructure team/administrators smiling.

2.The web service processes and returns managed properties, not crawled properties: Managed properties correspond to what actually gets indexed and displayed in search results. So, this reduces some of the confusion as why I cant see the updated results( perhaps you had forgotten to map your crawled property to a managed property and wait you will have to index it AGAIN!). Nightmare!

3. You can define a trigger to limit the set of items that are processed by the web service: In FAST 2010, each item had to pass through the pipeline whether you wanted to process it or not. This check had to be done in the code. Trigger in 2013 will allow us to define this check outside the code so that only for selected content, web service is called. This will optimize the overall performance and improve crawling time, if you only want to process a subset of the content.

So far, so good! But.. there are certain challenges we need to look at and see how we can overcome it. In fact, this is the most important part when you are architecting your CEWS solution:

1. The content enrichment callout step can only be configured with a single web service endpoint : Now this sounds very limiting.  I have multiple search applications and earlier I maintained the logic in different solutions. Do I need to combine them all into a single service? What about the maintenance and change request? Well there are several possible technologies one could consider to solve this but what I did in my project was to create a WCF routing service and let the routing service handle my multiple web services based on filters. You could also use it to implement load-balancing and fault tolerance. Here in the following example, I have two content sources “xmlfile” and “EpiFileShare”. I want to have two different services “xmlsvc” and “episvc” to process these different sources. This is how I will configure the end points in my WCF Routing Service:   2. Only one condition can be configured for Trigger. Different search application will require different triggers: Now, this can again be solved by using WCF routers and filters and configuring separate endpoints for separate triggers. Here I am using default managed property “ContentSource” as a trigger/filter to determine my service endpoint. To summarize, I have shown some of the advantages and challenges of the new CEWS architecture in SharePoint 2013 search and how you can overcome it. Hope that now you want  to try this soon and share your experience with us.

Instant search is widely known as «the way Google do it» – in fact they were the ones who started this trend and now everyone are used to it. What if you could give your SharePoint users the same experience they are already familiar with?

To begin with you must have in mind that instant search will produce a lot more queries and that your search performance will get worse if you have too many users hammering your SSA (Search Service Application). So, yes – there is a risk. The trade-off is high if you can increase the search experience to your users, but there’s also a risk of damaging the search experience totally if things start to go slow and people end up getting zero-hits.

Anyways, for the proof of concept you don’t need to think about this now :-)

Before moving on please check that you can meet these prerequisites:

Prerequisites

• Administrator Access to a SharePoint 2013 site
• SharePoint Designer 2013 or a way of mapping your site’s directory
• A text editor such as Sublime Text or Notepad++ (doesn’t really matter, but I recommend an editor with some code highlighting / intellisense)
• Basic understanding of the concept regarding DisplayTemplate in SharePoint 2013 (You can read more about it here and here :-) )

What we are going to do is that we are simply going to create a new DisplayTemplate for the SearchBox Control based on the original DisplayTemplate and edit the OnKeyUpEvent.

Changing the DisplayTemplate

1. Open SharePoint Designer 2013 and open your site
2. Browse to http://SPSite/_catalogs/masterpage/Display Template/Search
3. Create a copy of the DisplayTemplate named Control_SearchBox.html and name it Control_InstantSearchBox.html and open it and edit the <title>-tag to something else (you’ll need it do identify the template later)
4.  Go to the javascript section in the template and create a function called  doInstantSearch(clientControl, value, event) that only executes the normal query request.

It should look something like this:

1. Edit the searchbox-control’s ‘onkeyup’-event by replacing the attribute’s value with: ‘doInstantSearch($getClientControl(this), this.value, event);’ and save your template. Now you can go to your SearchBox web part and change the template to your new one (look for the title you set in step 3.) and test it yourself.
2. Now you have a one-to-one relationship between each key you press and query that you send to your SSA. However, as you will probably notice, the whole thing feels a little bit slow and not so smooth. This is because you are hammering your SSA and re-rendering your template each time you press a key – even if it’s not a real character – not so funny right? Let’s fix it.
3. In order to fix this we need to add a character-filter and a timeout to our function. We will get the character from the keyCode and try to match the character using a regular expression. In this example we will operate with a RegEx matching all alphanumeric-characters including special norwegian characters like «æ ø å». If the character matches it we will reset the timeout function and start over with a new 300 ms delay. When we hit a non-alphanumeric character or the delay times out we will execute the search request.

This is what we end up with:

And now you have an instant search with a 300 ms delay for each legal character that the user enters in the search box.

Try it yourself :)

The thesaurus lookup is performed at query time, and the dictionary can be set up with support for various languages. The thesaurus must be deployed as a csv file using powershell, it has columns for key, synonym, and an optional language. The “key” column can be a phrase, and so can the synonym. ie. “Go fishing” can be a synonym for “hunt for fish”. To provide several synonyms for a word, simply add it several times. To make the synonym go both ways, add a second entry with the word and synonym switching place. (See Microsoft for further details )

The thesaurus is simple, but it works. Or, does it really? Testing synonyms fetched from the internet on a SharePoint search index populated with US Court records crawled from theinfo.org.  AKA is a common legal term synonymous with “also known as”. So let’s try that out.

Searching for “aka” yields some hundred results, searching for “also known as” yields a different set of results of roughly the same size. By adding synonyms we would expect the search results to combine the hits for both queries into one, so to speak.
So I create a thesaurus containing:

Key,Synonym,Language
aka,also known as

And upload it with the powershell command:

$searchApp = Get-SPEnterpriseSearchServiceApplication
Import-SPEnterpriseSearchThesaurus -SearchApplication $searchApp -Filename \\spbox\temp\thesaurus.csv

I wait for a few seconds, and search for “aka”.
Now, I would expect to find hits containing “also known as”. But where is it? I have to scroll and page down to the bottom of page 3 before I find it:

Why? Could the ULS logs provide any clues? Turning on verbose logging on the Search Query Processing, and search again. Now, in the ULS logs there are entries with “After thesaurus tree modification” (the indentation is mine, trying to make it look a little clearer, also abbreviated it some).

Microsoft.Office.Server.Search.Query.Pipeline.Executors.LinguisticQueryProcessingExecutor : After thesaurus tree modification:
	'AndNode(FirstChild=StringNode
(FirstChild=WordsNode(FirstChild=TokenNode(FirstChild=null,NextSibling=OnearNode(FirstChild=TokenNode(FirstChild=null,NextSibling=TokenNode
		(FirstChild=null,NextSibling=TokenNode
			(FirstChild=null,NextSibling=null,Length=1,Linguistics=True,Token=<strong>as</strong>,Weight=1),
			Length=1,Linguistics=True,Token=<strong>known</strong>,Weight=1),
			Length=1,Linguistics=True,Token=<strong>also</strong>,Weight=1),NextSibling=null
,ExtraTermsAllowed=0,<strong>Weight=0.2</strong>),
Length=1,Linguistics=True,Token=<strong>aka</strong>,Weight=1),NextSibling=null),

The synonyms are given a weight of 0.2. The original term has a weight of 1, theoretically that means the synonym has 20% percent weight of the original term. Perhaps we could make the synonyms show up by increasing the weight.  So, how can we do that? Apparently, there is no way.  Not unless we open up the magic box of the Ceres shell.

We begin by connecting to the Interactionengine:

Add-PsSnapin Microsoft.SharePoint.Powershell
& "C:\Program Files\Microsoft Office Servers\15.0\Search\Scripts\ceresshell.ps1"
Connect-System -Uri  (Get-SPEnterpriseSearchServiceApplication).SystemManagerLocations[0] -ServiceIdentity (Get-SPEnterpriseSearchService).ProcessIdentity
Connect-Engine -NodeTypes InterActionEngine

Now, let’s try to rip out the configurations of the SharePointSearchProvider flow:

$flowname = Microsoft.SharePointSearchProviderFlow
Get-Flow $flowname  > $flowname.txt

Reading the flow configurations for the SharePointSearchProviderFlow, we find an option named synonymWeight.

<Operator name="Linguistics" type="LinguisticQueryProcessing">
  lt;Property name="querySpellingCorrectionTokenLimit" value="10" />
...
<Property name="stemWeight" value="0.2" />

&lt;Property name=<strong>"synonymWeight" value="0.2" </strong>/&gt;

&lt;/Properties&gt;

&lt;/Operator&gt;

Strangely, it has the weight 0.2. Ring any bells, anyone? It is the same weight we saw in the ULS. Now, let us try to see what happens if we increase the weight here.
So, we set the synonymWeight to 1, and upload the file.

Remove-Flow $flowname
Get-Content $flowname.txt | Out-String | Add-Flow $flowname
Stop-Flow –FlowName $flowname –ForceAll

Now, when searching for “aka”, we get hits containing “also know as” on the first search results page.
Voila!