GETTING STARTED WITH
ELASTICSEARCH ON WINDOWS
AND .NET WITH NEST
A short introduction
Oslo/NNUG Meetup
Tomas Jansson
29/01/2014
THIS IS ME
Tomas Jansson
Manager & Group Lead .NET
BEKK Oslo
@TomasJansson
tomas.jansson@bekk.no
github.com/mastoj
blog.tomasjansson.com
TL;DR;
https://github.com/mastoj/NestDemo
AUDIENCE
N00b
Expert
N00b
Expert
BACKGROUND
This is the data and we need this new application
THE MASTERPLAN
WHAT I WANT TO SHOW YOU IS...
Elasticsearch is awesome
Indexing using NEST
Querying using NEST
... not about advanced elasticsearch hosting
INSTALLATION
Great news, install as a service added in 0.90.5
Powershell to
the rescue
NEST
Abstraction
over
Elasticsearch
There is an low level abstraction as well called RawElasticClient
NEST
Abstraction
Fluent &
Strongly
over
Elasticsearch
typed
Functional C#
FUNC DEMO
C:\Dev\git> scriptcs
scriptcs (ctrl-c or blank to exit)
> Func<int, int, int> add = (x, y) => x + y;
> add(1, 3)
4
Func  executable
SIMPLE EXPRESSION DEMO
> using System.Linq.Expressions;
> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;
> addExpr(1, 3)
Expression  ”function description”
(1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method.
> var binExpr = addExpr.Body as BinaryExpression;
> Console.WriteLine(binExpr);
(x + y)
> var add2 = addExpr.Compile();
> add2(3, 1);
4
MORE COMPLEX EXPRESSION DEMO
> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
MORE COMPLEX EXPRESSION DEMO
> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
Enables us to translate from one domain to another in an ”easy” manner
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
Show me the
code!
ELASTICSEARCH CONNECTION
public class ElasticClientWrapper : ElasticClient
{
private static string _connectionString = Settings.ElasticSearchServer;
private static ConnectionSettings _settings =
new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200
.SetDefaultIndex(Settings.Alias) //"customer_product_mapping"
.UsePrettyResponses();
public ElasticClientWrapper()
: base(_settings)
{
}
}
//usage
var client = new ElasticClientWrapper();
MAPPING
public class Product
{
public double UnitPrice { get; set; }
public int TotalQuantity { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string ProductName { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string CategoryName { get; set; }
}
public class Customer
{
public string CustomerID { get; set; }
public string CompanyName { get; set; }
public string Address { get; set; }
public string City { get; set; }
public string Country { get; set; }
[ElasticProperty(Type = FieldType.nested)]
public Product[] Products { get; set; }
}
MAPPING & INDEXING
_client = new ElasticClientWrapper();
Mapping created from attributes
_client.CreateIndex("indexName", s =>
s.AddMapping<Customer>(m => m.MapFromAttributes()));
var customers = _customerRepo.GetCustomers();
_client.IndexMany(customers, "indexName");
Indexing will use the mapping
for the specified index
There is async versions of the
methods
ALIAS
_client = new ElasticClientWrapper();
_client.Alias("indexName", "aliasName");
Alias
Index_01
SWAPPING
_client = new ElasticClientWrapper();
_client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } );
1.
Create new index
2.
Swap
3.
Delete old index
Alias
Alias
Index_01
Index_02
MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES)
public class SearchModel
{
private int? _numberToTake;
public string Query { get; set; }
public Dictionary<string, IEnumerable<string>> Filter { get; set; }
public int? NumberToTake
{
get { return _numberToTake.HasValue ? _numberToTake.Value : 25; }
set { _numberToTake = value; }
}
}
QUERYING
Elasticsearch
NEST
{
_client.Search<Customer>(sd => sd
.QueryString(Input.Query));
"query": {
"query_string": {
"query": "tomas"
}
}
}
FUZZY
Elasticsearch
NEST
{
_client.Search<Customer>(sd => sd
.Query(q => q
.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query))));
"query": {
"fuzzy": {
"_all": {
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}
}
Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH
{
"query": {
"bool": {
"should": [{
"match": {
"_all": {
"query": "tomas"
}
}
},
{
"fuzzy": {
"_all": {
"boost": 0.1,
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}]
}
}
}
FUZZY IMPROVED (USING BOOL QUERY) - NEST
_client.Search<Customer>(sd => sd
.Query(q => q
.Bool(b => b
.Should(new Func<QueryDescriptor<Customer>, BaseQuery>[]
{
_ => _.Match(m => m
.OnField("_all")
.QueryString(Input.Query)),
_ => _.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query)
.Boost(0.1))
}))));
HIGHLIGHT RESULT - ELASTICSEARCH
{
"query": {
// see previous example
},
"highlight": {
"pre_tags": [
"<span class='highlight'>"
],
"post_tags": [
"</span>"
],
"fields": {
"companyName": {
"fragment_size": 100,
"number_of_fragments": 1
}
}
}
}
HIGHLIGHT RESULT - NEST
_client.Search<Customer>(sd => sd
.Query( /* See previous example */ )
.Highlight(h => h
.PreTags("<span class='highlight'>")
.PostTags("</span>")
.OnFields(new Action<HighlightFieldDescriptor<Customer>>[]
{
_ => _
.OnField(c => c.CompanyName)
.NumberOfFragments(1)
.FragmentSize(100)
})));
FACETS - ELASTICSEARCH
{
"query": { /* See previous example */ },
"highlight": { /* See previous example */ },
"facets": {
"products.productName": {
"nested": "products",
"terms": { "field": "products.productName", "size": 1000 }
},
"products.categoryName": {
"nested": "products",
"terms": { "field": "products.categoryName", "size": 1000 }
},
"country": {
"terms": { "field": "country", "size": 1000 }
}
}
}
FACETS - NEST
_client.Search<Customer>(sd => sd
.Query( /* See previous example */ )
.Highlight( /* See previous example */ )
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].ProductName)
.Size(1000))
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].CategoryName)
.Size(1000))
.FacetTerm(f => f
.OnField(c => c.Country)
.Size(1000)));
http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html
FILTERS - ELASTICSEARCH
{
"query": {
"filtered": {
"query": { /* See previous example */ },
"filter": {
"bool": {
"must": [
{
"terms": { "country": ["usa"] }
},
{
"nested": {
"query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } },
"path": "products"
}
},
{
"nested": {
"query": { "terms": { "products.productName": ["Chai"] } },
"path": "products"
}
}
]
}
}
}
},
"facets": { /* See previous example */},
"highlight": { /* See previous example */ }
}
FILTERS – NEST – PART 1, THE CUSTOMERS FILTER
private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}
FILTERS – NEST – PART 1, THE PRODUCTS FILTER
private static BaseFilter AddProductsFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}
FILTERS – NEST – PART 1, THE MAGIC DICTIONARY
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};
FILTERS – NEST – PART 1, ALL THE HELPERS
private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}
private static BaseFilter AddProductsFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};
FILTERS – NEST – PART 2, THE QUERY
_client.Search<Customer>(sd => sd
.Query(q => q
.Filtered(fq =>
{
fq.Query(qs =>
{
if (!string.IsNullOrEmpty(Input.Query))
{
qs.Bool( /* See previous example */ ));
}
else
{
qs.MatchAll();
}
return qs;
});
if (Input.Filter.Count > 0)
{
var filters =
Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray();
fq.Filter(fs => fs.Bool(bf => bf.Must(filters)));
}
}))
.Highlight( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ );
SUMMARY
Elasticsearch
NEST
Easy installation
Strongly typed client
Awesome search engine
Fluent
Abstraction over Elasticsearch
RESOURCES
Demo code: https://github.com/mastoj/NestDemo
Nest documentation: http://nest.azurewebsites.net/
Nest source code: https://github.com/Mpdreamz/NEST
Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net
Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense
Questions?
Thank you!
@TomasJansson