<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Search Nuggets &#187; Trond Øivind Eriksen</title>
	<atom:link href="http://blog.comperiosearch.com/blog/author/toeriksen/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.comperiosearch.com</link>
	<description>A blog about Search as THE solution</description>
	<lastBuildDate>Mon, 13 Jun 2016 08:59:45 +0000</lastBuildDate>
	<language>en-US</language>
		<sy:updatePeriod>hourly</sy:updatePeriod>
		<sy:updateFrequency>1</sy:updateFrequency>
	<generator>https://wordpress.org/?v=3.9.40</generator>
	<item>
		<title>Query Suggestions in FAST Search for SharePoint 2010 (FS4SP)</title>
		<link>http://blog.comperiosearch.com/blog/2011/05/03/query-suggestions-fast-search-4-sharepoint-2010-fs4sp/</link>
		<comments>http://blog.comperiosearch.com/blog/2011/05/03/query-suggestions-fast-search-4-sharepoint-2010-fs4sp/#comments</comments>
		<pubDate>Tue, 03 May 2011 09:10:58 +0000</pubDate>
		<dc:creator><![CDATA[Trond Øivind Eriksen]]></dc:creator>
				<category><![CDATA[Technology]]></category>
		<category><![CDATA[fast]]></category>
		<category><![CDATA[fs4sp]]></category>
		<category><![CDATA[query completion]]></category>
		<category><![CDATA[query suggestions]]></category>
		<category><![CDATA[search]]></category>
		<category><![CDATA[sharepoint]]></category>
		<category><![CDATA[sp2010]]></category>

		<guid isPermaLink="false">http://nuggets.comperiosearch.com/?p=409</guid>
		<description><![CDATA[Adding query suggestions to your search solution is a feature that can contribute to a better search experience for end-users. First of all, it makes us save time as we don&#8217;t have to type in the entire query. Secondly, it can help the user avoid potential spelling errors,  which in turn can reduce the quality of the [...]]]></description>
				<content:encoded><![CDATA[<p>Adding query suggestions to your search solution is a feature that can contribute to a better search experience for end-users. First of all, <em>it makes us save time</em> as we don&#8217;t have to type in the entire query. Secondly, it can help the user <em>avoid potential spelling errors</em>,  which in turn can reduce the quality of the results, and avoid that the user has to spend more time finding the information he/she is looking for. Also,  it can aid the user to do the first search more specific, by <em>adding information that may be pertinent to the search</em>.</p>
<p>With FS4SP,  these query suggestions are automatically created over a period of time based on how often the users search for a specific query within a certain timeframe. Sometimes, it can however make sense to modify the query suggestions manually &#8211; and to help us do that, we can always lean on PowerShell.</p>
<p>The script I have implemented, has the following parameters:</p><pre class="crayon-plain-tag">.\QuerySuggestions.ps1 -file [fileName] -action [add|delete] -queryssa [querySSAName]</pre><p>where <em>file</em> is the file with the query suggestion phrases (one per line), <em>action</em> decides if the query suggestions should be added or deleted, <em>queryssa</em> is the name of the Query SSA. And now for the actual code.</p><pre class="crayon-plain-tag">#Get input
param
(
    [string]$file,
    [string]$action,
    [string]$queryssa
)

function LoadQueryCompletion()
{
    # Read input file with stuff that you should add to the Query suggestion list
    if (test-path $file)
    {
        log VERBOSE &quot;Reading file: $file&quot;
	$input = Get-Content $file
    }
    else
    {
        log ERROR &quot;File noe found. Exiting script.&quot;
        exit 1
    }
    # Get SSA search application
    $searchapp = Get-SPEnterpriseSearchServiceApplication -Identity $queryssa

    # Populating a dictionary with the query suggestions that have already been added before
    $query_suggestions = Get-SPEnterpriseSearchLanguageResourcePhrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest
    $already_dict = @{}
    foreach ($term in $query_suggestions)
    {
        if (!$term){ continue }
        $already_dict.Add($term.Phrase,$term.Phrase)
    }

    if ($action -eq &quot;add&quot;)
	{
	    foreach ($entry in $input) {
	        if ($already_dict.ContainsKey($entry))
	        {
	            log WARNING &quot;Key already added: $entry&quot;
	        }
	        else
	        {

			try {
				log VERBOSE &quot;Adding entry ($entry) with language En-US&quot;
			    	$nooutput = new-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Name $entry
				$already_dict.Add($entry,$entry)
			}catch {
				log ERROR &quot;Could not add entry ($entry). Exiting script.&quot;
				exit 1
			}
	        }
	    }
	}
	elseif ($action -eq &quot;delete&quot;)
	{
		foreach ($entry in $input) {
			try {
				if ($already_dict.ContainsKey($entry)) {
					log VERBOSE &quot;Removing entry ($entry) with language En-US&quot;
			    	        $nooutput = remove-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Identity $entry -Confirm:$false
					$already_dict.Remove($entry)
				}else{
					log WARNING &quot;Entry ($entry) has not been added as a query suggestion&quot;
				}
			}catch {
				log ERROR &quot;Could not remove entry ($entry).&quot;
			}
	    }
	}
	else
	{
		log ERROR &quot;-action parameter contains an invalid value. Exiting script.&quot;
	}

    #Run timerjob manually to upload entries from DB
    Start-SPTimerJob -Identity &quot;prepare query suggestions&quot;

    log VERBOSE &quot;Finished loading query completion phrases.&quot;
}

#Main
$spInstalled = Get-PSSnapin | Select-String Sharepoint
if (!$spInstalled)
{
    Add-PSSnapin Microsoft.Sharepoint.PowerShell
}
LoadQueryCompletion</pre><p>If you want to include the logging output , add this to the top of the PowerShell script.</p><pre class="crayon-plain-tag"># Log level to color mapping
$lc = @{
	VERBOSE=&quot;green&quot;;
	WARNING=&quot;yellow&quot;;
	ERROR=&quot;red&quot;;
	MESSAGE=&quot;blue&quot;;
}
function log([string] $level, [string] $message) {
	$date = get-date -uformat &quot;%G-%m-%d %T&quot;
	write-host &quot;[$date] &quot; -nonewline
	write-host -foregroundcolor $lc[$level] $level.padright(7) -nonewline
	write-host &quot; $message&quot;
}</pre><p>If you&#8217;re really picky (like me), you should also consider having some kind of validation on the input parameters. For instance verifying that the file exists, that the action parameter contains a valid value, and that the Query SSA exists. You can check the last-mentioned by adding the following lines of code to the script.</p><pre class="crayon-plain-tag">$ssaExists = Get-SPEnterpriseSearchServiceApplication | Select-String $queryssa
if ($ssaExists -eq $null)
{
	log ERROR &quot;-queryssa is invalid. Search Service Application does not exist. Exiting script.&quot;
	$queryssas = Get-SPEnterpriseSearchServiceApplication | Where-Object {$_.DefaultSearchProvider -eq &quot;FASTSearch&quot;}
	foreach ($queryssa in $queryssas)
        {
		if ($queryssa)
    	    	{
    			$name = $queryssa.Name
    			log VERBOSE &quot;Found the following Query SSA: $name&quot;
    	    	}
        }
}</pre><p>After doing that, you should be ready to run the script on the SharePoint Admin server.</p>
<p>Afterwards, go to your FAST Search Center, and start typing one of the terms you have added. The result can look something like in the screenshot below.</p>
<p><img class="alignnone size-full wp-image-410" src="http://blog.comperiosearch.com/wp-content/uploads/2011/05/QuerySuggestions.png" alt="" width="428" height="246" /></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.comperiosearch.com/blog/2011/05/03/query-suggestions-fast-search-4-sharepoint-2010-fs4sp/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Property Extraction in FS4SP</title>
		<link>http://blog.comperiosearch.com/blog/2011/03/02/property-extraction-fs4sp/</link>
		<comments>http://blog.comperiosearch.com/blog/2011/03/02/property-extraction-fs4sp/#comments</comments>
		<pubDate>Wed, 02 Mar 2011 07:40:04 +0000</pubDate>
		<dc:creator><![CDATA[Trond Øivind Eriksen]]></dc:creator>
				<category><![CDATA[Technology]]></category>
		<category><![CDATA[enterprise search]]></category>
		<category><![CDATA[fs4sp]]></category>
		<category><![CDATA[property extraction]]></category>
		<category><![CDATA[sharepoint]]></category>

		<guid isPermaLink="false">http://nuggets.comperiosearch.com/?p=271</guid>
		<description><![CDATA[Property extraction (previously called entity extraction in FAST ESP) is a process that extracts information from the visible textual content of an item and stores that information as additional crawled properties for the document. In this blog post I will show how this can be automated in any given FAST Search for SharePoint installation. But [...]]]></description>
				<content:encoded><![CDATA[<p>Property extraction (previously called <em>entity extraction</em> in FAST ESP) is a process that extracts information from the visible textual content of an item and stores that information as additional crawled properties for the document.</p>
<p>In this blog post I will show how this can be automated in any given FAST Search for SharePoint installation. But first, just a short introduction to the extractors that we have out of the box:</p>
<ul>
<li><strong>Companies</strong> &#8211; extracts company names based on a generic dictionary.</li>
<li><strong>Locations</strong> &#8211; extracts names of geographical locations based on a generic dictionary.</li>
<li><strong>Person names</strong> &#8211; extracts names of persons based on a generic dictionary.</li>
</ul>
<p>In most cases, you will have companies, locations and person names that are specific to your company or organization. However, you may want to modify the built-in property extractors by adding inclusion lists and exclusion lists to improve the quality of these extractors. Typically you can use customer lists from your CRM system, employee information from your ERP system, and product listings you might have.</p>
<p>In order to accomplish this in an easy manner, we need some PowerShell magic.</p>
<p>First, just a quick overview of the input paramenters to the script:</p><pre class="crayon-plain-tag">.\PropertyExtraction.ps1 -file [fileName] -type [companies|personnames|locations] -addto [include|exclude]</pre><p>where the <em>file</em> parameter is the list of properties to extract, the <em>type</em> parameter specifies which property we&#8217;re dealing with and the <em>addto</em> parameter sets if the properties will be added to the include list or the exclude list.</p>
<p>Below is a snippet of where the fun takes place in code.</p><pre class="crayon-plain-tag">function ImportEntities()
{
    #Setting term entity dictionary to be &quot;companies&quot;, &quot;locations&quot; or &quot;personnames&quot;
    $entityExtractorContext = New-Object -TypeName Microsoft.SharePoint.Search.Extended.Administration.EntityExtractorContext
    $entityExtractors = $entityExtractorContext.TermEntityExtractors
    foreach ($extractor in $entityExtractors)
    {
        if ($extractor.Name -eq $type)
        {
            $entityExtractor = $extractor
            log VERBOSE &quot;Setting extractor to: $type&quot;
        }
    }

	log VERBOSE &quot;Reading file: $file&quot;
	try{ $input = Get-Content $file }
	catch{ log ERROR &quot;Failed to read file: $file&quot;}

    #Iterating over properties dictionary and adding them if they haven't been added before
    $input = Get-Content $file
    foreach($entity in $input)
    {
        if ($addto -eq &quot;include&quot;)
		{
			if ( $entityExtractor.Inclusions.Contains($entity) )
	        {
	            log WARNING &quot;Entity already added ($type): $entity&quot;
	            continue
	        }
	        else
	        {
	            try {
					$entityExtractor.Inclusions.Add($entity)
					log VERBOSE &quot;Added entity ($type): $entity&quot;
				} catch {
					log ERROR &quot;Failed to add entity ($type): $entity&quot;
				}
			}
		}
		elseif ($addto -eq &quot;exclude&quot;)
		{
			if ( $entityExtractor.Exclusions.Contains($entity) )
	        {
	            log WARNING &quot;Entity already excluded ($type): $entity&quot;
	            continue
	        }
	        else
	        {
	            try {
					$entityExtractor.Exclusions.Add($entity)
					log VERBOSE &quot;Excluded entity ($type): $entity&quot;
				}catch {
					log ERROR &quot;Failed to exclude entity ($type): $entity&quot;
				}
	        }
		}
    }
    log VERBOSE &quot;Finished loading $type properties.&quot;
}</pre><p>An example company list file could look like:</p><pre class="crayon-plain-tag">Apple
Comperio
Microsoft</pre><p><a href="http://blog.comperiosearch.com/wp-content/uploads/2011/03/Companies1.png"><img src="http://blog.comperiosearch.com/wp-content/uploads/2011/03/Companies1.png" alt="" width="492" height="207" /></a></p>
<p>If you did a crawl before starting on your white and black lists, you can use any noise in your navigators as input to the black list, and also entities missing in the white lists. An easy way to get all navigator values is to search for “#”, which will do a blank search showing all data you have access to.</p>
<p>Now that you have added the new white and/or black lists, you should schedule a new crawl and the quality of the entity refiners should be improved. Voila!</p>
<p><a href="http://blog.comperiosearch.com/wp-content/uploads/2011/03/SearchForComperio.png"><img class="alignnone size-full wp-image-311" src="http://blog.comperiosearch.com/wp-content/uploads/2011/03/SearchForComperio.png" alt="" width="666" height="390" /></a></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.comperiosearch.com/blog/2011/03/02/property-extraction-fs4sp/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
	</channel>
</rss>
