Christoffer was pacing angrily back and forth, Nøgne Ø IPA in his left hand, phone in the other. I was looking at the the long list of cancellations, including our connecting flight to Arlanda on the way to SF. The Norwegian strike was hitting hard with nearly no planes flying in Europe.

Arlanda airport is fairly described as the butthole of the world. Filled up with angry swedes and featuring the worlds slowest transfer security gate. We had booked the flight being fully aware of the pain and the risks and now the plane wasn’t even going to land there.

“Listen up” Christoffer was growling between gulps of strong IPA, “There’s no way we’ll be planted in your crap airport for over 12 hours bullshit strike or not.”

“Calm down” I looked up at the furious bearded giant. “You’re rattling my nerves.”

“Besides we still have CEOs credit card. Get some other plane and upgrade our tickets to business class while you’re at it. I’ll need some rest and proper legspace after all this crap.”

…

Out of breath after a mad dash to the airport service desk with Christoffers 60 pound america suitcase in tow, we were swaying dangerously with a Nøgne Ø beer heavy on our breath.

“We need tickets for the next plane to Arlanda. It’s of the utmost importance that we reach Arlanda as quickly as possible!”

The man at the counter tried his best to ignore us. Maybe we skipped line, I don’t know.

“Listen up, we are going to Elasticon in SF. The most degenerate collection of search professionals in the world. It’s a nasty assignment but somebody has to do it. We need to catch our flight from Arlanda and all this strike bullshit has left me with no patience!”

The man at the desk was sensíng an ugly scene. Two mean IT bums with their CEOs card and shot nerves. He ignored the shouting and commotion behind us and got to work.

…

Feet up in bclass seats on our way to SF we finally got some rest. Christoffer scanning the bclass microbrewery selection with a critical eye. “I need an imperial stout! In case of turbulence you know, need the extra weight. Maybe two or three even.” “Suit yourself” I said nipping a light lager while Christoffer was waving and hollering at the flight attendants. “Turbulence can be heavy shit for sure.”

…

Coming down in SF we knew we had to tighten up in front of the nasty customs procedure awaiting us. Christoffers america suitcase was another concern, but we were banking his large collection of home grown hops and rare yeasts not triggering any bomb sensors or something. The trick is to stomp your toe into something right before approaching the customs official. This will give you the steely stare and tight grimace needed to pass muster at the in front of the customs official asking why you’ld ever come here and if you have the means to butt yourself out before you become too much of a nuisance. “It’s a conference” I’ve wheezed through my gritting teeth. “But it might as well be a madhouse. These people are serious business”. The man behind the counter wasn’t really satisfied but didn’t press any further, and we hobbled on to pick up Christoffers gigantic trunk at arrivals.

…

Eleven hours of dry recycled air and flat beers can break any man, and stumbling out of Oakland airport we knew we were in a particularly bad shape. After his half a dozen heavy stouts Christoffer had entered a nasty beer induced coma and snored like an annoyed elephant for nine consecutive hours. Now he was wide awake of course ready for action while the rest of us hadn’t slept for 28 hours or so. Jetlag was about to do a double trick on us and I knew we had to take action.

There is only two remedies available to a man facing the disorientation and confusion of a serious jetlag and that is either to desperately stay awake long enough or induce a serious comatose sleep by an means necessary.
“If we hurry we can get into some dive before it’s too late. Load you up and then get you back to the hotel so I can get some proper rest.” I said. A plan I deeply regretted trying to keep myself together in a dingy Liars Dice den in Chinatown watching Christoffer enjoying a Sierra Nevada pale ale. The sharp rattling of dice cups among huge piles of dollar bills on the bar counter was jangling my nerves and leaving me with a sense that we had embarked something quite different than what we had signed up for …​

There is a Facebook  group dedicated to the task of spreading the knowledge, using images showing the difference between for instance “underbukser” (under wear) and “under bukser” (positioned below trousers).

Underwear  or under wear. Not the same thing!
Photo: André Ulveseter

Elasticsearch offers a wide range of analysing options. The ngram tokenizer splits a string into a series of continuous letters. For instance “underbukser”, with a size two ngram would split the word into “un” “nd” “de” “er “rb” “bu “uk” “ks” “se” “er”.  Elasticsearch will use the same analyzer  when querying the field, so if we search for “bukser” it will be split into “bu”, “uk”, and so on, and matches will result.

Well, enough chit chat, time for some code. Using the excellent Play tool created by Elasticsearch experts found.no we can even test it all out in our browser, no need for a server, you can do it at home on your ipad, chromebook, or even on a windows phone.

myAnalyzer in Action, showing how the word is split into ngrams

Here is a simple demo showing the ngram tokenizer in action.
The demo has three documents indexed, all containing the field foo with respective values “arbeidsmiljø”, “arbeid”, and “arbeidsmiljøloven”.
It uses an analyzer aptly called “myAnalyzer”,  this analyzer is using a custom tokenizer called “my_toknizer”, where the actual ngram tokenization is taking place.  The  ngrams for this sample are created with sizes ranging from 2 to 3. Testing it out on found.no/play, you can see how the various stages of the analyzer modifies the text. Neat!

The mapping enables the “myAnalyzer” to be used for  the “foo” field. Finally, I create a query, for the term “miljø”, which I expect to be found in the middle of documents nr. 2 and 3. Pressing the “run” button executes the setup, displaying the search results at the bottom right.

If you are really interested to learn more about analyzers, try the elasticsearch guide on languages, which is getting better day by day, or the articles on qbox on autocomplete using ngrams  or  found.no on language analyzers