Welcome to the July edition of the Hutch tech blog! Senior Hutch engineer, Dave talks tech stack migration using his own experience with one of our most used products in the TNT team.

A tech stack migration is probably one of the worst things you should bring up to your boss, as it comes with all kinds of risk, none more so that putting you, or even worse your entire team, on a non-profit (at least in the short term) project with a potentially ever extending finish line (we like to keep things car-related at Hutch). So you really want to do your due diligence before you poke your head round that corner.

Well, guess what I want to do in one of my upcoming Sprints??? Yip you guessed it, I want to migrate the tech stack for one of our most used products in the TNT team from Knockout JS and Java, to Angular and SAM. I thought I would share with you my thought process and potential plan as I do a little research before hitting my boss Chris with the big proposal.

Disclaimer: This is not a post on how to do a tech migration, but more my thoughts out loud on our specific requirements. There are mountains of posts out there on how to do a tech stack migration, as well as specialist companies who can help you, but hopefully you find some of my thinking useful. So, let's get to it.

WHY?

The why is probably the most important part of the migration process. Not why you used this stack in the first place (in my case I had just started at Hutch and wanted to impress the team, so I used Knockout as it was what I was most comfortable with), but why are we going to migrate? What is the reward for doing this migration and is it worth it? We really need to understand the why before we start ripping into code, because you might actually be better off not even bringing it up to your boss. If the risk is greater than the reward, then a migration may just not be worth the pain.

Now, this migration is not at the scale of say Etsy's recent migration, our product is a pretty small piece of a bigger complex puzzle that is our games and services. This means that the risk of doing a migration, while not minor, is not going to put the company at risk. 

There are usually two primary questions your boss will ask you, and as part of this process we want to answer these questions and put everyone at ease:

How long will it take?
How much will it cost?

Most developers on the other hand will start the why with pros and cons of your existing tech stack against your proposed new stack. (We are just going to assume here that you have done your research on which stack you intend to migrate to, that is an entire process in itself) I prefer to identify what my main areas of pain are with the current stack, and will my new stack resolve those problems. In my case I have the following issues using my current Java Stack (Jetty, Jersey, Jackson, Knockout, FreeMarker).

1. Authentication. Because of the data injection model provided by FreeMarker, I can't implement a Google sign on solution.
2. Cost. We are maintaining 2 AWS EC2 instances for what is a small website with multiple endpoints.
3. Complexity. There are not only a lot of moving parts for a simple product, but Knockout JS has become really complicated because of the amount of functionality on the page.

Now that I have identified the pain points, does my proposed solution of Angular and SAM solve these problems?

1. Authentication. Yes 

Angular provides more flexibility with integrating OAuth2, and we can integrate the AWS API Gateway with SAM.

2. Cost. Yes

Our Angular application can be hosted via S3 website hosting, so we only pay for data transfer.

3. Complexity. Yes

No servers need to be maintained with SAM, and Angular's component system makes our UI easier to break down into smaller chunks.

So now that I know my proposed new tech stack will solve my main pain points, can we answer our bosses questions?

How long will it take? - NO
How much will it cost? - NO

We are still not in a position to answer, so we need to move on and figure out exactly what it is we are going to migrate.

WHAT?

The answer to what, or scope, should be pretty simple, just tell your boss you are going to rewrite every line of code right? Wrong, you want to figure out exactly what you are moving so that if a full rewrite is not needed, you don't go creating a beast of a project. 

With any migration you pretty much have 3 areas to figure out:

    1. Front End Technology Migration
    2. Back End Technology Migration (+ Hosting Server Migration)
    3. Database Migration

There is potentially a 4th area which is Hosting Server Migration, but since we are going to be sticking with AWS regardless of the outcome, we will include this in our backend migration decision.

Front End Migration

Our front end is currently made up of Knockout JS, Foundation and Freemarker. 

One of our biggest issue with this solution is that it's complicated. As it stands our page has 3042 lines of code, as all of our JavaScript is on the page, merged with Freemarker for data injection. The attractive thing about moving to Angular is being able to write smaller components. For example, we have a dynamic condition builder, and this would be perfect to write as a component, not only because it would reduce the amount of code on this single page, but it would also allow us to use the condition builder on other pages. The support in the Angular community is also a huge factor, as finding solutions to Knockout can be tricky because it is older and less popular, especially amongst the Hutch Dev team. 

I'm also keen to move away from Foundation to use good old fashioned Bootstrap. It's not that there is really anything wrong with foundation, and when the project was started I thought that it actually had a nicer look and feel out of the box. However, as time has gone by the support for Bootstrap has just grown exponentially, and finding solutions to various issues is just so much easier to find with Bootstrap. 

Angular's template and data injection models also mean that we could introduce Google Sign on. This is something we have been implementing in our other products, and this one would make total sense as it is only used by internal users. This removes the need for us to do user management and permissions, so it's definitely a massive incentive to migrate.

Should we migrate - YES

Back End Migration

Right now we are using AWS EC2 instances with a Java server. We use Jetty and Jersey for our api architecture, and to be honest it's pretty awesome. With the help of Jackson we can easily serialize our data into concrete objects. The community is also huge and as I said in a previous post, the tech moves slow enough where you are never really out of date with Java.  

The main issue as highlighted earlier, is that it all seems like overkill for what is essentially a simple website with multiple endpoints. In our latest projects we have used SAM, which is a serverless model on AWS, and it's really just a lot easier to manage. The cost is also substantially less as we can host our website code in an S3 bucket with Angular. If you have never used SAM before, I would recommend it as there is definitely an element of there being a magic box. You pretty much just create a YAML configuration file and off you go.

So the question here for me is not simple because our current Java stack is not actually creating any major problems. We could save some money on hosting fees, and our releases and infrastructure management would become easier, but what is the trade off rewriting all of our Java into probably C#? Mmmmmm...

Should we migrate - NO

Database Migration

Some people might argue with me here, but I would suggest that migrating your database and subsequently your data, is by far the most complicated part of any tech stack migration. There is more risk involved with your data, not only because of data rules and standards, but syncing your data and ensuring you have no down time (probably impossible) is risky business.

Our current solution uses Mongo Atlas, a really nice cloud solution for storing documents. If you are unfamiliar with Mongo, it's a NoSQL database, and because it's not relational, you can pretty much do whatever you like with your data. 

{
   "id": 7,
   "name": "John Doe",
   "age": 22,
   "hobbies": {
     "indoor": [
       "Chess"
     ],
     "outdoor": [
       "BasketballStand-up Comedy"
     ]
   }
}

In the example above, if you were using a relational database you would probably have a table for users, a table for hobbies and a table to link user hobbies. With Mongo you can just construct a json object containing any data you like.

My main consideration for our migration is whether or not we move to Postgres, a relational database. The primary reason for a migration would be to put the product in line with our others, which use Postgres, but for me that is not a strong enough reason to go through the pain.

A relational database is perfect when things are very defined and expected to exist and require more thought into what needs to be stored and how. In our case, our data could be different for each use case and therefore a NoSQL solution is a good fit as we would not want to be updating the database with new fields, as well as our class objects with new properties every time we used the system.

Should we migrate - NO

So now that I know what I am going to migrate, can we answer our boss's questions?

How long will it take? - NO
How much will it cost? - NO

Again, we are still not in a position to answer, so we need to move on and figure out how long it will take to migrate.

WHEN?

Now that we are clear on why we are migrating, and what we are migrating, we can more accurately put some timings on our tasks before we lay it on the boss. Out of my 3 possible migrations I'm only proposing we do a front end migration, so I am in a really good position, but if I had said yes to all three then the answer to the 'when' would have to be spot on as we would be asking for a big commitment of time and cost. You really want to make sure you have done your research and considered all of your options before proposing a migration of that scale.

At Hutch we do use Agile with 2 weeks sprints, so for me I want to be able to give our project manager Julianna a fairly accurate estimate of how many sprints I will need to complete this migration. If that estimate comes in at 20 sprints that is ok, because it gives the team the right information to make a decision (I'm fairly sure that would be a no).

When using Agile we want our code to be complete, tested and ready for release when we are finished our sprint, so in this case I am going to propose that I would need 4 weeks, so 2 sprints to complete this front end migration.

So now that I have a clear idea of how many sprints I need, can we answer our boss's questions?

How long will it take? - YES
How much will it cost? - YES

Whoop whoop! This puts me in a much better position to move forward and propose we move ahead with our migration.

ROUNDING THINGS UP

This exercise is invaluable before you start any migration. Clearly you can go into a lot more detail and research depending on the project. You might also want to create an actual plan with a list of to-dos and such like, but that was not really necessary for me. All I wanted to achieve with this exercise was to be able to establish if it was worth migrating to a full new stack, and I got my answer. I was then able to give my boss a firm idea of the time it would take to complete the front end migration, which I think would be beneficial as we introduce more features to the product.  

Look out for more tech blog posts coming soon! Or bookmark the tech blog here: hutch.io/blog/tech/

And for our latest tech job vacancies, head here: hutch.io/careers. We'd love to hear from you!

Image credits:

Flying colhereiros - unsplash.com

Brainstorming over paper - unsplash.com

You've got your work cut out - unsplash.com

Eventually everything hits the bottom - unsplash.com