How to extend Sitecore’s Experience Database

It’s about time we start doing with Experience Database. Out of the box, xDB is awesome for collecting general data such as name, address, contact details etc and that’s really useful. But how about we take it to another level; store specific information to the website, the client and the interactions of the visitor.

So lets visualise this with an example so its easier to apply to the real world. We have a website. Users can order samples through a purchase path. Download a swatch image of a product. Watch promotional videos.

Each of those interactions have a varying degree of value to the company. Being able to see and use which samples have been ordered by individual site visitors (called Contacts) will allow the company to target that user uniquely. Prompting them purchase the full product, review the sample, recommend similar and so on.

In conjunction with seeing which product ranges they have shown interest in by watching videos or downloading a swatch the company will be able to achieve one-to-one personalisation – improving user experience and hopefully more sales.

How to extend xDB

Okay, so we have the type of data, the interactions, we want to store in xDB and some idea of its attributes. We need to convert that into a format that xDB understands and line with MongoDB best practices on which it is based.

Contacts are stored in MongoDB in a Collection, essentially a list. Each Contact contains a number of Facets, typically a grouping of data by theme e.g. Addresses. Each Facet contains a number of Elements, essentially table of data e.g. Delivery Address. Then an Element contains Attributes, the data itself e.g. Address line 1.

In our example, we are storing website key interactions completed by a user. So we’ll create a Facet called KeyInteractions. We are storing samples ordered by the user, videos they viewed and swatch images downloaded so we’ll create an Element for each. This is how all that will look.

Sitecore Contact in xDB

How to create xDB Facets

Fortunately to achieve this all we need to work with is the Sitecore API. First we define an interface describing the Facet, then the containing Facet itself. The Facet class will implement that interface, Sitecore.Analytics.Model.Framework.Facet and define the elements it contains.


public interface IKeyInteractionsFacet : IFacet
{
IElementCollection<ISwatchDownloadedElement> SwatchesDownloaded { get; }
IElementCollection<IVideoPlayedElement> VideosPlayed { get; }
IElementCollection<ISampleOrderElement> SampleOrders { get; }
}


[Serializable]
public class KeyInteractionsFacet : Facet, IKeyInteractionsFacet
{
public static readonly string FacetName = "KeyInteractions";
public const string SwatchesDownloadedName = "SwatchesDownloaded";
public const string VideosPlayedName = "VideosPlayed";
public const string SampleOrderName = "SampleOrder";
public KeyInteractionsFacet()
{
EnsureCollection<ISwatchDownloadedElement>(SwatchesDownloadedName);
EnsureCollection<IVideoPlayedElement>(VideosPlayedName);
EnsureCollection<ISampleOrderElement>(SampleOrderName);
}
public IElementCollection<ISwatchDownloadedElement> SwatchesDownloaded
{
get { return GetCollection<ISwatchDownloadedElement>(SwatchesDownloadedName); }
}
public IElementCollection<IVideoPlayedElement> VideosPlayed
{
get { return GetCollection<IVideoPlayedElement>(VideosPlayedName); }
}
public IElementCollection<ISampleOrderElement> SampleOrders
{
get { return GetCollection<ISampleOrderElement>(SampleOrderName); }
}

I recommend storing the facet name as a public property here so you can easily reference it in future. In the constructor we ensure that a Collection can be created for each of our interactions. As each interaction can be completed multiple times the Get methods use GetCollection returning a collection of xDB Elements. Finally add the Serializable tag to the class so it can be transmitted to for creation in Mongo.

How to create xDB Elements

Elements are constructed similarly to Facets. An interface describing the Element and a concrete class that implements that interface and Sitecore.Analytics.Model.Framework.Element.

The Elements for Sample Ordered and Swatch Downloaded will both hold two attributes, Range Id and Decor Id, that can be used by both Content Editor and system to find the Sample ordered or swatch downloaded by the user. The VideoPlayed Element will contain the Id of the Sitecore item containing the video and the video Url.


[Serializable]
public class SampleOrderElement : Element, ISampleOrderElement
{
private const string DecorIdName = "DecorId";
private const string RangeIdName = "RangeId";
public SampleOrderElement()
{
EnsureAttribute<string>(DecorIdName);
EnsureAttribute<string>(RangeIdName);
}
public string DecorId
{
get { return GetAttribute<string>(DecorIdName); }
set { SetAttribute(DecorIdName, value); }
}
public string RangeId
{
get { return GetAttribute<string>(RangeIdName); }
set { SetAttribute(RangeIdName, value); }
}
}

As with Facets the constructor of the Element should ensure our attributes can be created. For the setter of each Attribute we use the SetAttribute method to handle storing the name of the Attribute and its value in Mongo. The getter uses GetAttribute to retrieve the value.

For all this to be wired up we need to define our Facets and Elements in config. The default settings are within Sitecore.Analytics.Model.config file so I recommend patching them in as with all Sitecore configs. The config works in the same way those in Inversion of Control containers – define the interface and then the implementation. Our example settings look like this.


<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"&gt;
<sitecore>
<model>
<elements>
<element interface="Sitecore.Analytics.Model.Entities.IContactAddresses, Sitecore.Analytics.Model" implementation="Sitecore.Analytics.Model.Generated.ContactAddresses, Sitecore.Analytics.Model"/>
<element interface="ISlayTitans.CMS.xDB.Interfaces.Facets.IKeyInteractionsFacet, ISlayTitans.CMS" implementation="ISlayTitans.CMS.xDB.Facets.KeyInteractionsFacet, ISlayTitans.CMS" />
<element interface="ISlayTitans.CMS.xDB.Interfaces.Elements.ISwatchDownloadedElement, ISlayTitans.CMS" implementation="ISlayTitans.CMS.xDB.Elements.SwatchDownloadedElement, ISlayTitans.CMS" />
<element interface="ISlayTitans.CMS.xDB.Interfaces.Elements.IVideoPlayedElement, ISlayTitans.CMS" implementation="ISlayTitans.CMS.xDB.Elements.VideoPlayedElement, ISlayTitans.CMS" />
<element interface="ISlayTitans.CMS.xDB.Interfaces.Elements.ISampleOrderElement, ISlayTitans.CMS" implementation="ISlayTitans.CMS.xDB.Elements.SampleOrderElement, ISlayTitans.CMS" />
</elements>
<entities>
<contact>
<factory type="Sitecore.Analytics.Data.ContactFactory, Sitecore.Analytics" singleInstance="true" />
<template type="Sitecore.Analytics.Data.ContactTemplateFactory, Sitecore.Analytics" singleInstance="true" />
<facets>
<facet name="Personal" contract="Sitecore.Analytics.Model.Entities.IContactPersonalInfo, Sitecore.Analytics.Model" />
<facet name="KeyInteractions" contract="ISlayTitans.CMS.xDB.Interfaces.Facets.IKeyInteractionsFacet, ISlayTitans.CMS" />
</facets>
</contact>
</entities>
</model>
</sitecore>
</configuration>

Storing custom data in xDB

We’re now set up to store custom data. So lets write code to pass data to those Facets and Elements and also retrieve it. The Repository Pattern is perfect for this. A single place responsible for accessing and writing your custom data and is easily swappable for testing etc.

I have created a Write method in the Repo that accepts the Contact and a model of the interactions that can be performed on the site. From the Contact we can retrieve the KeyInteractions Facet to begin writing to each individual Element.


public class KeyInteractionsRepository : IKeyInteractionsRepository
{
public void Write(Contact contact, KeyInteractionsModel model)
{
IKeyInteractionsFacet facet = contact.GetFacet<IKeyInteractionsFacet>(KeyInteractionsFacet.FacetName);
if (model.SwatchesDownloaded != null && model.SwatchesDownloaded.Any())
{
foreach (KeyValuePair<string, string> keyValuePair in model.SwatchesDownloaded)
{
var swatch = facet.SwatchesDownloaded.Create();
swatch.DecorId = keyValuePair.Value;
swatch.RangeId = keyValuePair.Key;
}
}
if (model.VideosPlayed != null && model.VideosPlayed.Any())
{
foreach (KeyValuePair<string, string> videoPlayed in model.VideosPlayed)
{
var video = facet.VideosPlayed.Create();
video.VideoId = videoPlayed.Key;
video.VideoUrl = videoPlayed.Value;
}
}
if (model.SampleOrders != null && model.SampleOrders.Any())
{
foreach (var sampleOrder in model.SampleOrders)
{
var sampleOrdered = facet.SampleOrders.Create();
sampleOrdered.DecorId = sampleOrder.Key;
sampleOrdered.RangeId = sampleOrder.Value;
}
}
}
}

The code above shows looping through each interaction contained in the model. If one is present, Create is called against the Element to create a new item in the list and then set it’s Attributes with the model data. Writing to xDB is as simple as that!

Kinda. How can we get that data from the interactions made by the User on the front-end of the site into the Repository to save it?

The best approach I have come up with is to create a MVC controller and for each interaction JavaScript takes the HTML 5 data attributes of the markup and Posts to the Controller. Those details are passed through the layers from the Controller to the Repository for writing.

While storing interactions. We may as well register Sitecore goals as the same time. It’s pretty straight forward, we will get some data showing in Sitecore Analytics and start assigning value to visitors of the site.


<asp:HyperLink runat="server" ID="lnkDownloadSwatch" data-rangecode="SOLOHOME" data-decorcode="00271"
data-goalid="{08030449-A811-428B-95F0-59FCD42B8DEB}" data-goaldescription="SOLOHOME-00271">

The markup of the button to download a swatch is above. The Range Code and Decor Code of the product in the image is used for the interaction. For the goal, the download a swatch goal Id and a description are defined.

On the button click event JavaScript reads the data attributes and passes them to the methods in the MVC Controller via a HTTP Post.

The PageEvent controller, implementing Sitecore.Web.Mvc.Controller, has a generic method for registering all goals. However, each interaction has its own method which accepts the interaction data, creates an entity and passes it to the Key Interactions repository.


public class PageEventController : Controller
{
private IKeyInteractionsRepository KeyInteractionsRepository = new KeyInteractionsRepository();
[System.Web.Mvc.HttpPost]
public JsonResult RegisterGoal(string goalId, string goalDescription)
{
Item eventItem = Sitecore.Context.Database.GetItem(goalId);
var goal = new PageEventItem(eventItem);
if (!Tracker.IsActive)
Tracker.StartTracking();
if (!Tracker.IsActive || Tracker.CurrentPage == null || goal == null)
return Json(new PageEventRequestResult()
{
Success = false,
Message = "Sitecore is unable to track this goal",
});
Sitecore.Analytics.Model.PageEventData eventData = Tracker.Current.CurrentPage.Register(goal);
eventData.Data = goal["Description"] + " " + goalDescription;
Tracker.Current.Interaction.AcceptModifications();
return Json(new PageEventRequestResult()
{
Success = true,
Message = "Successfully registered goal",
});
}
[System.Web.Mvc.HttpPost]
public JsonResult RegisterSwatchDownload(string rangeCode, string decorCode)
{
if (!Tracker.IsActive)
Tracker.StartTracking();
if (!Tracker.IsActive || Tracker.Current.Contact == null)
return Json(new PageEventRequestResult()
{
Success = false,
Message = "Sitecore is unable to track at this time",
});
KeyInteractionsRepository.Write(Tracker.Current.Contact, new KeyInteractionsModel()
{
SwatchesDownloaded = new List<KeyValuePair<string, string>>()
{
new KeyValuePair<string, string>(rangeCode, decorCode)
}
});
return Json(new PageEventRequestResult()
{
Success = true,
Message = "Successfully registered interaction"
});
}
}

Retrieve custom data from xDB

Our Repository is looking a bit one sided, we are only storing data, so lets add some code to retrieve it and put it to use.

Earlier we defined Get methods in our Facet and Elements that handle the retrieval from MongoDB, so that’s most of the work done. So to finish off the repository our Read method will utilise those and construct a model that can be used throughout the solution.


public KeyInteractionsModel Get(Contact contact)
{
var keyInteractionsModel = new KeyInteractionsModel();
keyInteractionsModel.SwatchesDownloaded =
GetSwatchesDownloaded(contact)
.Select(s => new KeyValuePair<string, string>(s.DecorId, s.RangeId));
keyInteractionsModel.VideosPlayed =
GetVideosPlayed(contact)
.Select(v => new KeyValuePair<string, string>(v.VideoId, v.VideoUrl));
keyInteractionsModel.SampleOrders = GetSampleOrders(contact)
.Select(s => new KeyValuePair<string, string>(s.DecorId, s.RangeId));
return keyInteractionsModel;
}
public IElementCollection<ISwatchDownloadedElement> GetSwatchesDownloaded(Contact contact)
{
IKeyInteractionsFacet facet = contact.GetFacet<IKeyInteractionsFacet>(KeyInteractionsFacet.FacetName);
return facet.SwatchesDownloaded;
}
public IElementCollection<IVideoPlayedElement> GetVideosPlayed(Contact contact)
{
IKeyInteractionsFacet facet = contact.GetFacet<IKeyInteractionsFacet>(KeyInteractionsFacet.FacetName);
return facet.VideosPlayed;
}
public IElementCollection<ISampleOrderElement> GetSampleOrders(Contact contact)
{
IKeyInteractionsFacet facet = contact.GetFacet<IKeyInteractionsFacet>(KeyInteractionsFacet.FacetName);
return facet.SampleOrders;
}

The main Get method uses smaller individual retrieval methods for the Elements to creates the return type. Those retrieval methods use the Sitecore API to get the Facet for the Contact and return the appropriate Element.

You can also retrieve your custom data via Web API using the following Url – http://{sitecoreInstanceName}/sitecore/api/ao/v1/contacts/{contactId}/facets/{facetName}

And that’s it

How to extend Sitecore’s Experience Database to store custom data – creating custom Facets, Elements and Attributes. But not just storing any old crap, actually useful information as to how each user interacts with the Website.

But wouldn’t it be nice if we could show all that highly valuable data to the Content Editor in a pretty and usable interface so they could it for personalisation, analytics and marketing? So my next blog will be on how to extend Sitecore’s Experience Profile.

10 thoughts on “How to extend Sitecore’s Experience Database

  1. Can you explain to me what the heck the Downloads node (Behavior > Assets > Downloads) is to display on the Experience Analytics page? I have items tagged with the Downloads Page Event, but why don’t they display on this dashboard? ugh

    Like

    • I can actually! On media items, there is an assets field (or something to that effect) set a value in that field, it’s a pick list, publish. Then when downloaded, i.e. viewed via a general link it will be registered in the analytics. Be sure to wait for the session flush to xdb otherwise it won’t appear yet

      Liked by 1 person

  2. Pingback: Index custom xDB Facets to Segment Contacts in Sitecore’s List Manager | Exercising Sitecore

  3. Pingback: Display the Context of a Sitecore Goal in the Experience Profile | Exercising Sitecore

  4. Pingback: How to Identify and Merge Contacts in Sitecore xDB | Exercising Sitecore

  5. Pingback: How to Update Contacts in Sitecore xDB | Exercising Sitecore

Leave a reply to sandyfoley Cancel reply