At a recent Sitecore UK User Group Wesley Lomax and I presented on how to resolve technical debts in Sitecore implementations. With the release of the Helix principles last year a lot of preexisting Sitecore implementations are now facing the challenge of moving to these principles but are hindered by a load of technical debt.

For enterprise level implementations, it’s going to feel like an insurmountable challenge –  refactoring and correcting a colossal codebase and it’s Information Architecture (IA). It’s all probably been in development for years by multiple developers with multiple design patterns. On top of that add technical and business constraints, maintaining Business-as-usual, and still delivering projects? Fixing and updating legacy implementation soon seems like an impossible task.

I recently moved from agency to client-side this year and I have tackled this exact problem so I wanted to share some of my experiences to help others out.

First off, why should I make work for myself?

Here are a few benefits of solving your technical debt;

• Raise the quality limit – if the foundations aren’t sound, what’s build on top can only be so good.
• Code decay – the code’s environment changes and progresses (KPIs, infrastructure, audience, etc). If the code doesn’t change with it, it becomes less suitable, eventually a hindrance and ultimately becoming legacy code.
• Short term pains for long term gains – the upfront work is going to be significant but will get more reasonable as time goes on.
• Preempt serious rework – I’ve seen legacy code block entire projects let alone make development tasks longer.
• Align with the direction of the industry – you’re going to be in a better position to take advantage of what Sitecore offers.
• Improve as a developer – probably the most interesting to you, learn some new technologies, concepts, ways of working.
• Undeveloping is fun! – ‘undeveloping’ some badly implemented code is favourite part of being a dev

The thing is, clients (usually) don’t want to pay to resolve technical debt. They see technical debit as just that, technical debt. It’s the developers remit. The more digitally mature clients will understand the cost of ownership; longer to implement changes, features more prone to bugs, performance issues etc.

However, just because the client doesn’t want to pay or devote time to technical debt it doesn’t mean it shouldn’t be resolved. So it’s down to you. You’re the one working with this thing right?

You’ll want to figure out what’s right for you

Don’t go blindly following Helix.

There is far more to building and updating a Sitecore implementation than implementing Helix principles. Make a few notes about your team, the client, and the existing implementation before deciding what you want to achieve. Here’s somethings you might want to consider;

• Think about your team – we have a number of backend developers, frontend developers and architects with varying skill-levels. While we wanted the implementation to use the latest and greatest. The higher priority was to ensure all can work on the solution effectively. Simple solutions for complex problems.
• Keep the Content Editors in mind – we have a Digital Team which are the site administrators and around hundred Content Editors. Some of whom use Sitecore everyday and others once every few months. So the IA changes need to be to intuitive but still familiar.
• Whats the scope of your implementation – developing for an international commerce site requires completely different development considerations than a brochureware site. Whats a good technical approach for one could be disastrous for the other. Our implementation caters for half a million paying members reaching a hundred-thousand unique visitors each week. Performance and resilience is number one concern where personalisation and sales funneling might be for a brochureware site.
• Be mindful of what the client actually does – while what the client sells or the services it provides may not have direct impact on what you’re going to fix first, it can give you an idea of what projects and features your implementation may have in the future. I have the complete re-platform of the My Account functionality coming so I needed to ensure the solution and infrastructure an handle heavy Microsoft Dynamics integration.

Agree as a team where you want to be

We just wanted to make development great again. Enjoying the day-to-day and not feel like we are fighting the code base to get something done.

As a team decide how you want the solution to look in three or six months time. What problems do you guys face daily? How pain-free is creating releases? What issues are always raised in support? Here a few things you might want to solve;

• Daily pains – headaches you have to deal with while going about your day-to-day development.
• Best practices – areas where the solution falls short of .NET and Sitecore best recommendations.
• Solution structure – should the solution be easier to navigate or simple to utilise.
• Development blockers – code or design choices that you are forced to work with or work around.
• Methodologies – what design patterns or principles do you want to follow to improve the quality and ease of development.
• Reoccurring problems – issues keep getting swatted only to reappear.
• Logs – use the Sitecore Log Analyser to find the most frequent problems in your code base.
• Resolve To Dos – all solutions have them.
• Helix – complete a gap analysis of the solution against the Helix documentation to give you a shopping list of things to do. Do this for both Code and IA.

To give you an idea of the tech debt we wanted to solve;

• Harden the foundations – ensure stability and performance of the site. Address the poor code implementation of MVC lowering the quality limit and enforce object-oriented design throughout.
• Remove N Tier Architecture – N Tier was forcing us to dev in a particular way, building upon bad practice for consistency sake. Circular references was forcing bad design decisions. Finally we didn’t want monolithic architecture as we didn’t know where to start correcting. We wanted to move to Modular architecture.
• Solve daily headaches – it was incredibly frustrating that the folder structure of Rendering Items was completely different from the Views. The repeated hunt for the razor files was infuriating.
• Rewrite the bad – critical parts of the implementation were written by developers not quite familiar with Sitecore. We wanted to fix that. A lot of customisation simply wasn’t needed so we wanted it removed.
• Enable future phases – restructure the solution ready for integrating with Dynamics to facilitate Membership Self-service. There is massive drive in the company to make use of all of Sitecore’s features and a lot of Sitecore’s contextual marking functionality didn’t work due to custom code or poor configuration.
• Domain driven development – we wanted to be able to talk to our project owners in terms they understand. Meaning logical parts of the solution to be named and modeled after logical parts of the company and its services. Previously, all the projects were named after the technology it used.

How to go about it

1. Settle on technologies and design patterns

You know the problems you want to solve and how the solution should look. You probably have an idea of what technologies and approaches you can use to help you get there. If not, have a look at how Habitat handles tooling and I recommend SOLID Principles, clean coding, and the Stairway Pattern.  You want the code to be consistent, readable and easy to work with. Selecting principles can help you get there, especially with larger teams.

You’ll want a good Branching Strategy. The internet recommends so many different workflows but I don’t think you can go wrong with Git Flow. This should ensure your work in progress doesn’t some how make it to Prod.

Most of your time will likely be renaming, restructuring, removing, and refactoring code. Save yourself a headache. Get yourselves Resharper if you haven’t already.

Like us, you probably don’t have the time or the resource to regression test after each change. So I recommend you write some Spec Tests to cover the functionality, look at how Habitat has implemented them. This way you can ensure the User Story is still met while changing the underlying code. I don’t need to sell to you the value of Unit Tests.

If you’re going down the Modular route you’ll need to consider how you are going to publish all the dlls, views, and configs to your webroot. A Task runner like Grunt or Gulp (or some other random mouth noise) can collect those files up and place them in your local site. Modularisation of the code will affect your deployment packages too. So make sure you are adding the Projects as Build Dependencies of your Web App and reference them in your deployment package configs.

2. Rewrite development guidelines

It’s a good place to start.

Most companies have development guidelines that haven’t been looked at in years let alone updated. But what good is redeveloping existing code in a particular way if new code doesn’t follow suit? You just came up with the content of the development guidelines, take the time to draft up some paragraphs and examples.

You’re going to have a much easier time aligning everyone in the team. And newbies will have a handy guide on how to work on this thing.

Chances are, this will be the first time your jumble of ideas and approaches have been collected in one place. Writing it all down should show that your plan all fits together. But if it doesn’t, or there are some gaps, no worries! This is the perfect time to find something that works. Finding out after months of work is not.

Don’t just go copying the principles of Helix as your dev standards. You can state that your development guidelines also include Sitecore best practices. No point regurgitating.

3. Review your solution

By now you should have a pretty decent list of things you want to do. And with your Development Guidelines you have something to review the existing code against.

It may sound like a lot of work but it doesn’t need to be. You know how it should all be. So spending an hour or two making notes of things that need to be actioned. I recommend you pay special attention for things that can block your plans.

Ultimately this step is going to give you an idea of how the code matches up where you want it to be.

4. Review your Sitecore Items

This being a Sitecore implementation, the Sitecore Items are as important as the code files. Here are a few things I recommend doing.

Complete a Content Inventory. Sounds boring as holy hell. However, its good to understand what Templates are present, the data they model, the fields they have, and how the templates relate to each other. Chances are there are a number of templates / fields that aren’t required any more or can be merged.

It’s not just content though, chances are there are a number of Placeholder and Rendering Items that can be thrown away or need some rethinking. This work should actually help making code changes easier.

One thing that bugged me in my solution was that each of the 20-odd sites were structured differently. Where you’d find a datasource folder in one site, it would be located completely different in another. No concept of separating data that is shared across all sites or specific to one. So decide on the hierarchy of a Site. Map it out in Excel or something so you have a structure to work towards.

For this part, I found reviewing the IA against Helix Section’s 2.5 and 2.6 really helpful. Complete a gap analysis of how you stack up against it and aim to implement some of its conventions. I really liked the concept of an ‘Interface’ template. A template that doesn’t have any fields but including it an Item’s base Templates shows its of a particular type. For example, _Home Interface template added to the base templates of all Home Items is massively helpful to develop against if the various Home pages of your sites use different Templates.

5. Create a backlog

Your reviews would have given you a list of things to do. Basically you’ve got yourself a backlog. I’m not saying start running sprints and treating it like a project. But add the tasks to a Kanban board, I recommend Trello so you can share it easily. Pick up the tasks as and when you’re able.

You probably have a lot to do so keep tracking the work simple as possible. Three columns, To-do, Doing, Done. At the very least you’ll know how the solution is progressing.

One thing I will say is to prioritise the tasks which block others. We couldn’t do a lot of what we wanted until we started breaking down the Business Layer into Feature Projects.

6. Finally, pay your debts in installments

You’re probably thinking this all well and good but I don’t have three months to spend just doing this. Neither did I. What I found best is to work on areas when you come into contact with them.

You’re never going to get this done as a Big Bang approach – all in one go after months of dev. Clients won’t pay for it. They won’t accept a halt to delivery. And neither do you want to deploy all this massive refactoring all one go, far too risky. Imagine the conflicts you’ll have on merge.

What you want is phased approach. Roll out these changes a little bit each release. We use the Boy Scout Rule – leave the code in a better state than you found it. As you come into code leave it more in line with your development guidelines than how you found it.

Delivering new functionality around registration? Spend ten minutes moving the logic from the Controller to the Accounts Feature. Creating a new listing? Take the time to split the existing indexes into Feature-centric ones. Or whatever may be in your backlog, you get the point.

It may not sound like much, but a handful of developers doing the Boy Scout Rule as they build over three months can really turn a solution around.

Benefits of our hard work

We collected some metrics before we started on our Tech Debt Backlog. We also took another set of readings about two months of completing tasks via the Boy Scout Rule. We used NDepends to help get some figures – its a tool that can estimate Technical Debt based on attributes of the solution. We didn’t take the results as gospel but it was nice to have an independent opinion.

The key take away is that before we started our backlog our solution was estimated at having 77 days of technical debt. 77 days of purely fixing problems. On the later reading NDepends quoted us 14 days of technical debt. I’m sure there are diminishing returns here on out but that is a massive jump for moving to a modular architecture, enforcing SOLID principles, and undeveloping.

For actually working on the solution. Yeah, there are still times where I’m holding my head in my hands in dismay. But those are becoming less frequent. The business layer is shrinking fast and being replaced with Feature and Foundation projects. Current count at sixty.

We actually know where we to start fixing things instead of fighting the web of project dependencies. The SOLID principles and Stairway pattern has helped a lot with that. We’re no longer building upon bad for consistency sake. We can even find Razor files based on the location of their Rendering Item! Templates actually make sense now and were making progress changing the IA so that each site is consistent, has its own content as well as a share of the global content.

Code weeds are becoming less common. We’re not trying to solve a simple support ticket only to loose three hours of development time and submit humongous pull requests to each other (…on a Friday).

I’m not saying we’re done by any means. We still have a lot of work to do. I still need to nuke every helper class from existence. But we’re closer to where we wanted to be.

And that’s it!

So, get your team onboard. Figure out whats right for you. Decide how you want the solution to look in six months. Choose the tech and principles. Review the solution. Load up the backlog in a Kanban board. And leave your code in a better state than you found it.

What I can’t stress enough is not to tackle this on your own. You need to your Team’s buy-in. Without Wes, we wouldn’t have achieved a faction of what we have so far.

It does feel like we’re making development great again.

 

Wes has kindly published the full presentation slides on his blog.