• Missing metadata issues
• Inconsistent metadata issues
• Cleansing of content
• Integration of semantic layers/Automatic tagging
• Integration with 3rd party service
• Merging data from other sources

Content Enrichment Web Service in SharePoint 2013 is a SOAP-based service within the content processing component that can be used to achieve this. The figure below shows a part of the process that takes place in the content processing component of SharePoint search.

Content Enrichment Web Service SharePoint 2013 combines the goodness of both FAST for SharePoint Search and SharePoint Search  to offer a whole new set of possibilities and has its own challenges. To see an implementation example, check the MSDN link which pretty much sums up the basic steps. In this post we are going to look at some of the advantages and challenges of CEWS coming from a FAST 2010 background:

1. CEWS is a service and you DON’T have to deploy it in your SharePoint environment: Perhaps this is the biggest architectural change  from the content processing perspective. What this means is that your code no longer runs in a sandbox environment within SharePoint Server. The webservice can be hosted anywhere outside your SharePoint server thus reducing deployment headaches and huge number of approvals required to deploy the executable files. I can see operations/infrastructure team/administrators smiling.

2.The web service processes and returns managed properties, not crawled properties: Managed properties correspond to what actually gets indexed and displayed in search results. So, this reduces some of the confusion as why I cant see the updated results( perhaps you had forgotten to map your crawled property to a managed property and wait you will have to index it AGAIN!). Nightmare!

3. You can define a trigger to limit the set of items that are processed by the web service: In FAST 2010, each item had to pass through the pipeline whether you wanted to process it or not. This check had to be done in the code. Trigger in 2013 will allow us to define this check outside the code so that only for selected content, web service is called. This will optimize the overall performance and improve crawling time, if you only want to process a subset of the content.

So far, so good! But.. there are certain challenges we need to look at and see how we can overcome it. In fact, this is the most important part when you are architecting your CEWS solution:

1. The content enrichment callout step can only be configured with a single web service endpoint : Now this sounds very limiting.  I have multiple search applications and earlier I maintained the logic in different solutions. Do I need to combine them all into a single service? What about the maintenance and change request? Well there are several possible technologies one could consider to solve this but what I did in my project was to create a WCF routing service and let the routing service handle my multiple web services based on filters. You could also use it to implement load-balancing and fault tolerance. Here in the following example, I have two content sources “xmlfile” and “EpiFileShare”. I want to have two different services “xmlsvc” and “episvc” to process these different sources. This is how I will configure the end points in my WCF Routing Service:   2. Only one condition can be configured for Trigger. Different search application will require different triggers: Now, this can again be solved by using WCF routers and filters and configuring separate endpoints for separate triggers. Here I am using default managed property “ContentSource” as a trigger/filter to determine my service endpoint. To summarize, I have shown some of the advantages and challenges of the new CEWS architecture in SharePoint 2013 search and how you can overcome it. Hope that now you want  to try this soon and share your experience with us.

FAST ESP was, and still is, an incredibly flexible and scalable enterprise search platform and is used by customers with sophisticated search needs and a significant requirement for updating the frequency, relevance adjustments, scalability and stability. The platform has very rich semantic and linguistic characteristics, and a wide range of connectors to the underlying systems, and connection points which can search and present relevant content adapted to the users context and device. These features generate revenue and/ or reduce costs for more than 2,600 customers worldwide that use or have used the FAST ESP platform.

FAST Search and Transfer was acquired by Microsoft in 2007. Following the acquisition, it became clear that FAST ESP will not be continuing as an independent cross-platform search engine. FAST ESP will however, exist in mainstream support until 2013 (2015 for customers with a License Grant), and extended support on all OS platforms until 2018 (2020 for License Grant customers).

During this period it is possible and safe to use and even develop new services on the FAST ESP search engine. Our team of Application Management Services (AMS) consultants have extensive experience in the FAST and FAST ESP platform – many of them formerly worked in consultancy, support or services in Fast Search and Transfer / Microsoft.

While Comperio’s AMS services can customise and sustain stable operation of customers on the FAST ESP platform, we also actively advise on search strategy, design, solution architecture and new search technology, as needs and technology evolve.

Our long-term plan with Comperio Front has been making customers less dependent on the underlying search engine, and the situation with FAST ESP shows that this has been a good strategy. For our customers with Comperio Front, we are now able to provide effective transitions to new technology platforms, as well as new corporate business models, and the search engine’s position as an information hub are preserved in Comperio Front.

When it comes to transitions to new technologies post the FAST ESP era, more choices and opportunities are opening up. For many traditional FAST ESP customers in e-commerce, catalog, classified ads and media, FAST for SharePoint is not necessarily a natural choice straight away, especially if they use anything other than Microsoft technology. Comperio has experience and dialogue with several customers on the transition to open source platforms and cloud services such as Solr, Elasticsearch, Amazon Cloud Search and also Google Custom Search – when called for.

That said, Microsoft plans to further use the advanced capabilities of FAST in their products, both in Office/SharePoint/Exchange/Office365, as well as in its search engine on the internet – Bing. Microsoft’s release of FAST Search for SharePoint 2010 is an example of this and the next generation of Microsoft products will show this more clearly. For many FAST ESP customers in the business market, it is natural to follow this path, as long as it is possible to build the solutions they want. And this is also where Comperio Front comes in, as a building block to make it possible to customize the search based solutions in SharePoint in a flexible manner.

Microsoft’s development team for search, or FAST R&D department, is based in Torggata, Oslo, and works closely with both the Bing and the Microsoft Office Division development teams.
With Comperio’s close affiliation from a geographical, historical and technical perspective, we continue to follow the FAST / Microsoft team in their development (and vice-versa) with technical and strategic dialogue constantly held.

But above anything else, we want to help our clients to great search experiences!

You can learn more about FAST ESP at http://www.comperiosearch.com/products/fast-esp/ and go to www.comperiosearch.com to see other products and services offered by Comperio.

FAST ESP var og er fremdeles en utrolig fleksibel og skalerbar enterprise search platform og blir brukt av kunder med avanserte søkebehov og ekstreme behov for oppdateringshyppighet, relevansjusteringer, skalerbarhet og stabilitet. Plattformen har meget rike semantiske og lingvistiske egenskaper, og et stort utvalg av connectorer til underliggende systemer, samt tilkoblingspunkter for å kunne søke og presentere relevant innhold tilpasset brukerens kontekst og device. Disse egenskapene er inntektskilde og kostnadsreduserende faktor for mer enn 2600 kunder verden over som benytter eller har benyttet FAST ESP plattformen.

FAST Search and Transfer ble kjøpt av Microsoft I 2007. Etter oppkjøpet er det etter hvert blitt klart at FAST ESP ikke blir videreført som selvstendig crossplattform søkemotor. FAST ESP vil imidlertid eksistere i mainstream support fram til 2013 (2015 for kunder med en Licence Grant), og extended support på alle OS plattformer fram til 2018 (2020 for License Grant kunder).

I denne perioden er det fullt mulig og trygt å benytte og til og med utvikle nye tjenester på FAST ESP søkemotor. Vårt team av Application Management Services  (AMS) konsulenter har lang fartstid fra FAST og med FAST ESP plattformen., og har blant annet jobbet med konsulenttjenster, AMS og support internt hos Fast Search and Transfer/Microsoft. Samtidig som vi med våre AMS tjenester kan oppretteholde stabil drift av din FAST ESP plattform, vil vi være aktive rådgivere og løsningsarktekter over på en ny søketeknologi ettersom behovene melder seg.

Vår langsiktige plan med Comperio Front har vært å gjøre våre kunder mindre avhengige av underliggende søkemotor, og situasjonen med FAST ESP viser at dette har vært en god anbefaling. For våre kunder med Comperio Front er vi nå i stand til å tilby effektive  overganger til nye teknologiplattformer, samtidig som bedriftens forretningslogikk og søkemotorens posisjon som  informasjonsnav er ivaretatt i Comperio Front.

Når det gjelder overgangene til nye teknologier i en etter FAST ESP- era er det flere valg og muligheter som åpner seg. For mange tradisjonelle FAST ESP kunder innen ehandel, katalog, rubrikk og media er ikke FAST for Sharepoint nødvendigvis er et naturlig valg for øyeblikket, spesielt ikke dersom man ellers benytter noe annet enn Microsoft teknologi. Comperio har erfaring og dialog med flere kunder om overgang til opensource plattformer og Cloudtjenester som Solr, Elasticsearch og Amazon Cloudsearch, og også Google Custom Search, alt etter behov.

Når dette er sagt er Microsoft’s plan framover å bruke de avanserte egenskapene til FAST i sine produkter videre, både for kontorstøtte i Office/Sharepoint/Exchange/Office365, så vel som i sin søkemotor på internett – Bing. Microsofts release av FAST Search for Sharepoint 2010 er et eksempel på dette, og neste generasjons Microsoft produkter vil vise dette enda tydeligere. For mange FAST ESP kunder i bedriftsmarkedet er det naturlig å følge dette sporet, så lenge det er mulig å bygge de løsningene man ønsker. Og også her er kommer Comperio Front inn som en byggestein for å gjøre det mulig å tilpasse søkebaserte løsninger i Sharepoint på en fleksibel måte.

Microsofts utviklingsteam for søk er lokalisert med basis i FAST R&D avdeling i Torggata Oslo, og samarbeider tett med så vel Bing som Microsoft Office sine utviklingsteam. Med vår nære tilknytning, geografisk, historisk og teknisk fortsetter Comperio å følge FAST/Microsoft i deres videre utvikling, og vårt Microsoft team er hele tiden i Front med det nye som leveres fra Microsoft.

Men først av alt ønsker vi å hjelpe våre kunder til gode søkeopplevelser!

Here’s an interesting article from Beyond Search which looks further into this topic:

Comperio Provides a Safety Net at SharePoint

Entering a Misspelled word during internet search can lead to a few moments of irritation, but in the business world things become even more complicated. Typing one wrong letter can often send users spiraling downward towards a promotional disaster. Fortunately, some providers are hanging up a few safety nets to prevent the fall.

Microsoft just revamped Bing to recognize common errors during internet search.  According to, ‘Bing reveals efforts to help with human search errors Bing will remember key words and phrases most frequently used. To correct issues the search feature itself was altered as:

“We used to show synonyms as part of our recourse links and this would open up some surface area for showing alterations. The query “define interesting” highlights an example where the recourse link was unnecessary.  In this case, showing the Recourse Link didn’t enhance the experience. We’ve removed the Recourse Links in cases where we are very confident that they add little value or distract users.”

Basically, Microsoft created a net to catch errors in advance and Comperio can implement similar functions within SharePoint using Fast technology. The safety net Comperio provides can help catch users before they fall into the uncomfortable realm of presentation faux pas.  They can customize the programming towards the specific needs of the business to increase efficiency.

For more information about Comperio, visit the company website at http://www.comperiosearch.com/.

via Jennifer Shockley, June 27, 2012

Beyond Search

Users find laundry lists of results a necessary but sometimes hard to use way to pinpoint needed information. Users looking for a PowerPoint presentation want a way to spot presentations without browsing, opening, scanning, and repeating the process. One feature of SharePoint is its document thumbnail and preview function. Instead of a list of text results, SharePoint can display search results with a thumbnail image of the document. Users can quickly identify a document type, which allows a research task to be accomplished more quickly.

There is, however, one challenge in some SharePoint installations. According to the document Office Web App & FAST Search Document Thumbnail and Preview scenarios, many users found document previews and thumbnails to not show up in FAST search results for SharePoint 2010. Microsoft acknowledges:

“Document Previews do not work with Claims Based Authentication and is a known limitation with the Product.”

Microsoft’s knowledge base article provides a number of ways to resolve the problem. But what does a SharePoint administrator do when a third party application is part of the mix? The SharePoint licensee needs immediate access to deep expertise with both SharePoint and FAST search to ensure that system performance and functionality are maintained at a high level.

Comperio, one of the world’s leading firms in FAST search engineering and consulting, can resolve preview issues quickly. Comperio’s engineers have in-depth experience with both SharePoint and FAST search. If you want to tap document previews using Microsoft’s native functions or employ third party software from firms such as BA Insight (www.bainsight.com), Comperio delivers. Comperio combines experience and technical expertise for leveraging FAST search within SharePoint. For more information about Comperio, visit our website at www.comperiosearch.com.

via Jennifer Shockley, June 22, 2012

Beyond Search