The empty search box by Google

The problem with an empty search box

So, what’s the biggest problem with the empty search box? In my mind it doesn’t give a hint about what’s the possible outcome of asking a question to the search engine. Think about these five questions being different search boxes:

1. Soooo…?
2. What would you like?
3. What would you like for desert?
4. Do you like ice cream for desert?
5. Do you like pistachio ice cream for desert?

The first question is the most open ended, and is the equivalent of the empty search box in a general search engine. The one with all of the worlds knowledge at hand. Second one hints that the answers of your request lies within your likings. Third, the questions get semi-concrete, and fourth and fifth, you’re asked a yes/no-question.

A search without search box working quite well

The recipe app asks you if you like any of these recipes based on two variables (time of year + place you harvest the food). We know the date and make an assumption about place and tell you that we’ve chosen these variables. Then we give you a couple of extra variables to play with to refine the search a bit more.

So far, the content seems to trigger peoples imagination and the swipe interaction is easy enough to do many times, although not very well communicated so far. When I normally give people a working search prototype, they do 2 – 5 search queries. Now I see between 5 – 15. That’s great stuff, and maybe a search without search box is actually a good idea?

A screenshot of the Recipe App search solution without a search box

Known bugs and weaknesses

• URL stays the same
  No way of sharing a specific search (month+place+filters)
• Buggy visual relevance
  For desktop and pad, all recipes with match on three ingredients or more should have full width result view.
• Not all recipes indexed
  HTML for the recipes has changed. We didn’t have time to figure out all the new characteristics of the new HTML, so a lot of recipes were not indexed.
• Visual snag on time navigator
  When selecting “short”, “medium” or “long” time to prepare recipe it should collapse as with the type navigator.
• Swipe hangs every now and then
  The swipe library is either not tuned perfectly or a little fragile. Easy to get into a state where it stops working.
• All ingredients equally important
  “Oregano” and “Chicken wings” equally important. That results in some not-so-desired search results.
• Google Analytics and single page app not fixed
  We log one pageview per user since it’s a single page app. With a rewriting of the URL this can be easily fixed.
• Design and UX
  It’s just a makeshift design to communicate the idea. It should better explain interaction and season+place.

Any comments? We’d love to get your input! Check out the solution, the actual recipe app, or you can check out the other blog posts about the Recipe App. There’s a lot of both search domain-, tech- and UX-stuff.

Recipe app link: recipe.comperiosearch.com

Responsive design test done in HTML5, CSS3 and JavaScript. A discovery engine in the making?

We have gotten a lot of knowledge, both tech and UX. Latest of many findings was the custom scoring function in elasticsearch. Hopefully, we’ll learn something within the business of food recipes, and we’ve received a lot of info from Christopher Sjuve at Klikk.no, the site we crawl for content to our search index. All blog posts on the app available under the “recipe app” tag.

Takeaways

The main intent of the app, when the idea was born back in 2011 or 2010, was to discover recipes or more general: To make a real content discovery engine based on search. Recipes that you didn’t know of or think of when planning a meal. And recipes that uses ingredients in season, thus making it relevant. In other words, the best, cheapest and most tasteful ingredients. While we continue to discover a lot of issues with how we thought the app should work technically, it really seems it can work for the end user.

A general and common issue with search solved

Mostly, with search, you get no real discovery. You rediscover stuff you already knew where there, or you find stuff you anticipated where there. But what with all the content unknown or not anticipated by the user? Nowhere in that process will regular search find stuff suitable for you that you initially didn’t think of. That’s a problem because you never touch a vast majority of possibly valuable content in your search index.

A real content discovery engine

So, what does this have to do with our recipe app? By picking the app, we know two facts upfront: The user wants food recipes, or make/serve food, to be more specific. And we know the time of year. This puts us in the position to make do a real instant search. The first screen you see is a search query already executed. The questions (search query) asked by the app to the index are then “recipes with the most in season ingredients for this month found on i.e. a farm”. We choose the farm since this is what resembles a supermarket the most. After that we let the user play with known constraints like type of meals and preparation time in addition to change place where ingredients are found and time of year.

More playfull approach. People swipe (when they understand that they can swipe, yes one of the UX issues) a lot from what we have seen. We’ll add some Google Analytics to the app soon, so we can track actual use.

Testing on a Sony Xperia Tab Z. A lot of small bugs and a bit noisy.

We’d love your feedback!

So, what do you think of our recipe discovery engine? Is it just that, a discovery engine? Are we on to something? Do you think it would work? What should we change? Don’t be shy, give us a real uppercut feedback in the comment field below!

Warning! Dynamic scripting has been disabled by default in  elasticsearch version 1.4.3. Using the technique in this article now requires some extra steps. Details on the Elastic blog.

Our design explains that we need results to be sorted by number of ingredients, and then by date, with the most recent recipes on top, scoring recipes from 2008 at bottom. The original attempt involved a simple “terms” query.

Investigating the results for July, in the garden, gave us recipes for jam at the top, with the count of matching ingredients being only 3 for the top hit. The list of ingredients in season for July in the garden is quite long, but all you need to know is that it contains “rips“, a little red sour berry, and “poteter”, that is potatoes.

You can find the queries used in this post at http://sense.qbox.io

For this query, we were some what surprised that none of the recipes on top contained potatoes. Using the highlight function, it is easy to see that the number of ingredients returned for the top hits should have been higher. Then it dawned upon us: TF-IDF! You’re messing up again! What we see is just the normal relevancy, promoting the least commonly used terms to the top. This works well for natural language queries, but that’s not really what we are doing here.

Luckily, elasticsearch doesn’t leave us stuck in a rut. We implemented a custom scoring function using the function score query.

For this scoring, we don’t need any points from the query terms, we just want to replace the default scoring with our custom one (boost_mode = replace).
The scoring function has two parts, one where we add up the number of ingredients, and one part to add some boost to the most recent posts.

The function to sum up term frequencies looks like this:

The tf() function returns the term frequency  for a term in this field. There are a number of functions you can use to perform your own calculations based on index properties. The functions available are documented in the text scoring in scripts and the scripting module.

To the term frequency we add the date scoring:

We are using a linear function, but we could also have used gauss or exponential curves.

The “scale” parameter decides the point on the graph where the value specified in “decay” should be found. Setting it to 700 days allows the scoring to reach 0 for recipes dated in 2008, which was one of the requirements.

The number of ingredients will always be whole numbers, while the date scoring is normalized to values from 1 to 0.

To allow the scoring from both functions to add up, we use the parameter score_mode=sum.

Elasticsearch is an extremely powerful toolbox for search, information retrieval, analytics, big data, you name it. The possibilites are endless.

If you want to learn more about custom scoring in elasticsearch, there are some nice videos you can watch:

“Scoring for human beings” by Britta Weber, Berlin Buzzwords 2014

http://www.elasticsearch.org/videos/introducing-custom-scoring-functions/

Main recipe app user interface

First off I’ll just make a clickable test, doing some modulo stuff so that the edge-cases will wrap (December -> January). Then try Awwwards Touch Swipe library. It has the ability to register both horizontal and vertical swipe motion. They also have a image gallery swipe demo. It only shows horizontal swiping, but I’m guessing it will be possible to expand that to vertical swiping as well.

When we have a swipe navigation that works good enough it’s time to build that in to our Angular app and use it to change the search query. Then we have to do some front end code on the actual search results and search result items.

That’s all for today. Happy 17th of May. Sounds nice? This is work in progress, so check back every now and then for new blog posts.

Any comments? Feedback on the recipe app user interface is more than welcome!

Recipe App User Experience on Ipads and Androids

How to sort the result set

The relevance model is now down to two variables:

1. Amount of in-season ingredients in a recipe
2. Date (year) that recipe was published.

Date will only be used to rank recipes with same amount of in-season ingredients.

Filters and drill-down possibilities

Main filters will be:

1. Light meal
2. Dinner
3. Desert

On these three filters you will be able to do a second filtering based on how short the recipe preparation time is:

1. Quick (40 minutes or less)
2. Slow/Thorough (more than 40 minutes)

Auto selected filters

The actual search is done without the user needing to do or type anything. An OR search for the current months ingredients is done when you enter the page. Default place will be “The Farm”, since that’s the closest thing to what you find in the supermarket.

The filters will also be automatically picked. During work days the “Dinner” and “Quick” filters will be set. Late Friday the meal type filter will change to “Light meal” and “Slow/Thorough”. Sometime after 12 o’clock on Saturday, the meal type will change to “Dinner”.

Based on the ideas for visualization of the search result, this is how I imagine the UX for our Seasonal Food Recipe App. To the left you have a search result for “The Sea” in May, to the right “The Sea” in January. We’ll have a new prototype running soon. Next task will be to make the navigation sprite with a combination of the five places and 12 months (sun in different position for each month). Then HTML-prototype the three different search result types: Large (full width), medium and small.

Thanks to Qbox for letting us use one of their Elasticsearch instances!

The recipe app is work in progress. Check back every now and then for new blog posts on the subject.

For the recipe app I want to do something slightly different, showing the absolute search result quality for each result. In other words: If the best search result in a result set is not very good, make it smaller than if it has higher value, thus showing an absolute value for each result. The dialogue equivalent comparing i.e. two movies would be to define the best of them better than the other, but not the best you’d seen.

Absolute search result quality

aka. Absolute visual relevance hierarchy

How do we then measure absolute quality? In an earlier post I described what would be our relevancy hierarchy. The more in-season ingredients in a recipe, the better quality.

I’ve defined three quality groups for absolute search result quality so far (number of ingredients: n):

1. n >= 4
2. 1 < n < 4
3. n = 1

Here are three different examples on search results set. First has two results with four or more in-season ingredients and second has only one. Third has none, typically something that would happen during the winter months in Norway:

The recipe app is work in progress. Check back every now and then for new blog posts on the subject.

Search templates is a brand new feature  added to elasticsearch version 1.1. 0. It allows potentially complex search logic to be stored as templates. The user sends a query with the parameters that matters for him, along with the name of a search template, and the supplied parameters will be inserted at the proper place on execution.  The template format is moustache, a widely used javascript templating tool.

Cool stuff, you say. How can I use it? And does it really work?

In the current implentation of search templates, you create a JSON request specifying both the template and the parameters.

You can save the templates in the config/scripts folder of your elasticsearch installation, and use the saved query.
The template is saved as config/scripts/ingredientsquery.moustache. This is how my template looked like:

And here is a sample query using this template

To make changes to the template effective, you need to restart elasticsearch.

You would perhaps expect it to be possible to upload a search template using the REST API.  Unfortunately,  that hasn’t been implemented yet.  At the moment, you have to store the search templates in the  config/scripts folder in your elasticsearch node.  There is an issue registered to add this feature, but in the mean time, our friends over at qbox are helpful as always and offer to manually place the templates where they belong while we wait for elasticsearch version 1.1.1 (or something).

Using this awesome feature, the only reason we still need some middleware between our elasticsearch installation and our browser is to protect the search service from malicious requests. As it so happens, Qbox have recently added a feature that enables public read only access to the elasticsearch query _search endpoint.

This enables us to use the Qbox elasticsearch service directly from the angularjs client for elasticsearch.

Thus we eliminate the need for the searchclient in ruby, and one less server component to care about.

Sounds nice? This is work in progress, so check back every now and then for new blog posts.

Luckily we got some help in our decision process. Elasticsearch presented itself to us as self-evident. qbox.io helped us out with hosting (Thank you guys!).  We needed something  at least equally cool to create the front end stuff, somewhere to flesh out the design and user interface. We chose AngularJS, perhaps out of ignorance and old habits, it hasn’t failed us yet.

Now we had the most important parts of the tech stack figured out, we had to figure out how to enable communication between the two. We imagined that somehow the user would “do something” in the web part, and his/her actions would be “sort of transferred” back into elasticsearch, which in turn would perform some tricks to find out the most relevant information, and return it back to AngularJS. We soon realized that we didn’t want to communicate directly with elasticsearch from the browser, although it is possible (using for instance elasticsearch-angular.js). Putting an elasticsearch instance directly available on the internet is very easy. Unfortunately, it leaves your node freely available to anyone.  Elasticsearch is controlled by issuing calls to the REST API, such as

curl -XDELETE 'http://localhost:9200/indexname/'

Which will, yes you guessed it, delete the index named indexname. While we truly believe in empowering the user, we also believe there has to be limits.  There are several options for locking down the service. One very common option is to block port 9200 in the firewall, and set up a web server/proxy such as nginx that can route web traffic to a selected address like, say, /search to a specific address on the server running on port 9200 inside the box.  That way you can protect elasticsearch, and open just the parts you want to allow traffic on.

In the end we actually did use nginx as a webserver and proxy, but we didn’t pass the queries straight to elasticsearch.

We made ourselves a tiny search client in the middle. Some of the reasons were:

1. Separation of concerns
2. Security
3. Testability

The search architecture

The recipe web-app takes care of getting some explicit input from the user. In addition to that, we may gather some implicit information to use as “hints”, for instance  the time of day, the day of the week, the month of the year, geo-location,  browser/mobile/os and so on. The web-app sends parameters to the search client as a simple GET query i.e. /search?ingredients=potet and receives in return the unmodified elasticsearch JSON response.

By separating the search logic from the user interface, we can easily test the results of modifying query parameters such as what fields to search in and what weight to give to different fields. Modifying details of the elasticsearch query can be done without fear of accidentally breaking something in the front end. We could even swap out the entire search engine without having the front end notice.

The search client is implemented in ruby, using grape,  a tool to build lightweight REST- like  APIS.  Connecting and talking to elasticsearch is taken care of by the official elasticsearch ruby client.

The skeleton code for the search client is available on github. We have stripped away the “special” queries, leaving only the most generic functionality needed to start creating a search solution. It can receive query parameters, issue queries to elasticsearch and return the unmodified response, and log it. In actual implementations we have added complexity to the queries, functions that retrieve only facets, or query in different subsets of the index.

Feel free to fork the project and tailor it to your own needs. If you have suggestions for improvements, don’t be shy about it, create a pull request.

Sounds nice? This is work in progress, so check back every now and then for new blog posts.

One generic tool for getting the right results on top is the “term frequency–inverse document frequency“, or tf-idf for short. It’s a combination of how often a term is mentioned in a document compared to how often it’s mentioned in all of your documents in the index. So, rare terms within the whole index used often in one document makes it a good search result when searching for that term. But most likely, not good enough. You need to figure out what’s the characteristics of your content, and what are the most characteristic use cases and user stories for your users. Only then can you achieve great relevancy, …  I mean get the right result on top of your search result.

Model for relevance tuning

We’ll use our Recipe app as an example…

So, for our food recipe app, we have some obvious content characteristics:

• The more ingredients in-season for one recipe is good. We’re doing an OR-search on all ingredients in-season so this comes out-of-the-box … almost.
• Quite a lot of recipes doesn’t stand the test of time. We know that most of the recipes at oppskrift.klikk.no from 2008 or newer are quite good and have nice photos.
• We’re not sure if we need this, but we know whom of the writers to trust. This may be an overkill when we already have a boosting on newer recipes.

And we know a lot about our users as well:

• Most grown up people in Norway have a job, thus limited time to prepare a meal. This means that recipes that takes shorter preparations should be boosted from Monday through Thursday. The verdict on Friday is still not decided.
• During the weekend people have more time to make dinner. The recipes that takes a short time to prepare most probably cut some corners, and are not that good compared to recipes that takes a little longer time. So for the weekends, we should do a demotion of really quick recipes, at least for dinners.

This is the info we’re going to use to sort our search result. But we have more knowledge about our users that we can use to auto-set filters for certain times of the day:

• Most work days, people don’t plan a breakfast meal or lunch. The whole day we can auto-set the main “course” filter.
• During the weekend, people may also plan a lunch. We’ve decided to auto-set the “light meal”-filter during weekends up until lunch time. After that the “main course” filter is auto-set. We’ll log if the first thing our users do is to set another filter.
• On Friday and Saturday a lot of Norwegians drink beer, wine or liquor. After some hours of drinking, they get hungry. Maybe we should have an “afterparty, quick and greasy and tasty-meal”-filter auto-set for late Fridays and Saturdays?

So, what’s the filters we’ve decided on:

• Light meals
• Starters
• Main courses
• Deserts
• … and maybe the Afterparty-thingy

What do you think?

Sounds nice? This is work in progress, so check back every now and then for new blog posts.

Tapping into the mental model

Each variable combination will do an OR-search containing a lot of ingredients for that particular place and time of year. Our relevancy model so far:

1. Recipes with the highest amount of ingredients hits
2. Newest recipes
3. Recipes written by Christopher Sjuve

Number one on the list is given, but why number two and three? A lot of the older recipes doesn’t stand the test of time, and we know we trust Christopher Sjuve’s recipes.

The places we’ve chosen:

1. The sea
2. The farm
3. The garden
4. The forest
5. The mountain

It’s an odd bunch of places, but so far we think it will work. Logs and usability testing will tell us later if we’re hitting the target or not. The farm doesn’t fit that well with the others, since you don’t normally enter a farm and steal a cow or some potatoes. But it will be what’s closest in content to your average supermarket, and will be the default choice. Almost all of the places will have some overlapping ingredients. Each search is a combination of a place and a month. 5 places x 12 months means a sprite of 60 images where you swipe horizontal to select a place. Month will be selected for you, but to open up for exploration we think it will be valuable to  have a vertical swipe to select month.

Here’s the first wireframes on the UX concept.

First version of the query matrix. So far not organized by places, but types of ingredients.

The search is already up and running, but lack every sign of graphical user interface.

Thanks to Qbox.io for letting us use a Hosted Elasticsearch instance for this project!

Sounds nice? This is work in progress, so check back every now and then for new blog posts.