It’s a new year! And fresh from a short break from the blog, to focus on presenting at the xDB training course in New Orleans and Sitecore User Group in Cardiff, I’m back. So let’s get to it!

I talk a lot about the value of extending xDB to store data relevant to the client and how to surface that data for Content Editors to use. I now want to cover how to use that data on a grand scale. From accessing that data for all your contacts in a performant way. Then using that data to create lists of similar contacts based on their interactions. Finally allowing you to target groups of individuals with relevant email communications, via EXM, or synchronize with a CRM (more on that in a future blog).

Note : 
This functionality is only available in Sitecore 8.1 Update 3 and upwards

Indexing xDB Facet Data

We’re dealing with potentially hundereds of thousands of Contacts stored in xDB, each one having a plethora of data stored against it. Computing that quantity of data at run time each and every time isn’t going to cut it. Fortunately, this is not a new problem. We deal with this challenge when searching for Content. So we can use indexes to compute the large amount of data at a deferred time and indexing only what we need.

Fortunately, the Sitecore Analaytics Index is already indexing Contact data stored in the default xDB Facets and from Sitecore 8.1 Update 3 upwards we can now tell it to index our custom facets.

As expected, there is a pipeline for us to hijack and get it to do our bidding; contactindexable.loadfields. We’ll use this pipeline to access data in xDB Facets of the Contact currently being indexed and then return the data we care about in a number of indexable data fields. First the patch for the pipeline.

Now that we’re hooked into the pipeline for getting the indexable fields lets create a class which inherits from ContactIndexableLoadFieldsProcessor to get the data. The overridable GetFields method accepts ContactIndexableLoadFieldsPipelineArgs which exposes the Contact. From there you can call GetFacet method to retrieve your custom Facet.

If you are familiar with my previous blogs the example I use is a client’s site allows customers to order samples of products. I store contextual information about that sample order in xDB.  I want to index all skus of the products that have been ordered by the Contact as well as their favourite (most ordered) product type.

In the code below, once I have calculated the skus and favourite type I create a new IndexableDataField for each, defining the field name, the value, and the type. For the favourite product type I simply want to store the type’s name as a string. Whereas for the products ordered I want to store each product’s sku in an array of strings.

I recommend you create a processor for each type of data you want to index. In my example I’m only indexing one Element in one of my custom Facets.

Now that we have done the hard work of retrieving the values and creating indexable fields we need to define those fields on the Sitecore Analytics Index. As always I recommend using a patch. Add new fields within the Field section defining the FieldName property as the name you gave it in the processor.

Here is the field definitions for Lucene.

Here is the definition for your next indexable fields if you are using Solr.

Using Luke for Lucene you can see that the data of the custom Facet for this Contact has been indexed.

Quick bit of help – Once the new index patches are added to your site it will likely begin indexing the Contacts. If not, try visiting as a new contact and push the session to xDB. Reminder, you can’t rebuild the Sitecore_Analytics_Index like you can other indexes.

Now that we have an efficient method of selecting any number of contacts based on xDB data lets put it to use.

List Manager – Custom Segmentation Rules

Let’s say I want to group every contact based on their favourite type of product or I want to email promotional materials to all Contacts who have ordered samples from a particular range, then I am segmenting Contacts. Sitecore’s List Manager is made for this purpose. However the List Manager is not aware of our custom xDB data now indexed in the Analytics index. Let’s make it aware.

The List Manager lets you select Contacts based on a number of rules, this is the same Rules Engine that is used for personalisation etc. However Segmentation rules can’t be used for personalisation and vice versa.

To select the contacts in the above scenarios I will need to create a custom Rule Condition class. The class must inherit from the base class TypedQueryStringOperatorCondition of the IndexedContact type and a generic type where the generic type inherits from VisitorRuleContext. This exposes the IndexedContact allowing retrieval of the various IndexedDataFields including the ones we’ve created. As we are inheriting from TypedQueryStringOperatorCondition we take a value defined by the Content Editor and run compare expressions against the value we have indexed.

Before we can run the rule we need to create an definition Item in Sitecore. Insert a new Item at the path /sitecore/system/Settings/Rules/Definitions/Elements/Segment Builder/ using the Condition template. Enter the namespace for the type Field and then define the text of the rule.

Upon selecting the rule the Content Editor is prompted to chose a comparison type and the value to compare. I recommend writing your rules such that they support the built-in operators but if your indexed data is more complex I recommend hiding the comparison types away from the users.

Upon clicking OK the list is populated with with Contacts matching the conditions and all contacts will be visible within the List Manager.

And that’s it!

Segmenting millions of contacts in meaningful ways done simply, elegantly and no nasty load on the server. You’ll be free to use all that great data you are capturing in new and helpful ways.

The lists themselves are used by Sitecore’s EXM so you are facilitated tailored mailshots right off the bat. From the List Manager you can access each Contact’s Experience Profile, which makes the List Manager the most effective way to find particular Contacts right now.

The lists a Contact belongs is also stored against the Contact record in xDB so they can be easily accessed for other uses. Such as sending particular Contacts to a CRM I’ll be covering this in a future blog.