A Belated Post-Mortem of an Ember Rewrite and Other Thoughts on Ember

Everything that has a beginning has an end, or ‘the short version’

The Matrix

Bidvine was founded in 2014 on the idea of connecting customers with local service professionals in a way that provides successful outcomes for all involved. A simple enough idea, but would it work? The answer was yes, and better than we thought it would. A prototype website was flash-built in late 2014, and by early 2015, it was clear that this prototype website was not going to last.

Bidvine is a marketplace, and needs to have efficient user acquisition on both sides on the marketplace, but also provide an efficient communications layer that underpins the core interaction, and also that forms the core utility of the platform.

Being a website built on acquiring users, we are by necessity built on acquiring data. We resolved on Ruby on Rails for our prototype website and it could not cope with the data. We also needed to build scalable, intuitive user interfaces as part of our role as this communication layer, and our launch day Ruby on Rails site had no frontend at all.

Fortunately, we were a(n) (even) young(-er) startup then and had the flexibility and mobility to shape our future: as a business, and as developers.

After using this Ruby on Rails site as our MVP through our Techstars London Demo Day launch in February 2015, we knew that we needed to build out a proper front end, and also extract the API into a serviceable and scalable backend.

We considered Angular, Ember, and React for the frontend, and, for the most part, we didn’t consider anything but Node.js for the API.

Eventually – some days later, and after many ‘are you sure?’ moments – the decision was made and we switched to Ember and Node.js/Express.js.  

The process of rewriting our website was not a simple undertaking. We submit this document as proof that it can be done, as a guide to those who would do the same, and as a thank you letter to the Ember community.

The decision

The Decision

We faced a dilemma: refactor our existing site built with Rails, or rewrite and improve our site using a different framework? The more pressing question, as its answer would inform the answer to the first question, was what framework should we be rewriting to?

Based on our collective experience as web developers, we surveyed the landscape of the web and tried to identify toward where web development was trending to make our decision. Immediately, several choices were obvious:

  1. Stay with Ruby on Rails because it seemed to work for many others,
  2. Use ASP.NET MVC, or
  3. Use a front-end JavaScript framework, specifically Ember, Angular, or React + some routing library.

Looking at the options, any of them could have worked for us, all proven solutions, and we had some level of experience using each language. However, .NET, in general, is a language largely ignored by startups due to the cost of committing to a Microsoft development environment so we ignored it as well. We didn’t have any objections to using Ruby on Rails, other than the fact that it wasn’t working well for us at the time, but we preferred what we saw with the front-end JavaScript frameworks.

In consideration of our ability to hire against each decision, the talent pool for each was somewhat different. The popularity of Rails meant that developers were both in demand but also very open to acquiring the skills and learning the language. The .NET crowd was generally coming from a different experience-set and demographic to the startup-oriented developer and therefore it was potentially more difficult to find the right stage and culture fit. The Javascript frameworks lent themselves more by stage and fit with a tolerance for learning and uncertainty inherent in the people exploring and interested in these newer frameworks.

In particular, what we saw was the web moving in the direction of these JavaScript frameworks. Angular was a well-entrenched framework by early 2015, but Ember and React were much more in their infancy. React had recently revolutionised JavaScript frameworks by introducing “DOM diffing”, which in layman terms means React was able to manipulate a website much more efficiently. In response to this, Ember was introducing its “Glimmer engine” and committing to a much more sustainable framework structure going forward. At the same time, Angular was being completely rewritten in TypeScript.

While Angular was committing to an Angular 2 upgrade that would require anyone using Angular 1 to largely rewrite their whole website, Ember was committing to a smooth upgrade path with a steady stream of deprecations, and a clear path forward. Angular was, thus, out of consideration if we were to have to rewrite our application to Angular 2 just after we finished rewriting it to Angular 1. We were left with two choices: Ember, or React. We ultimately chose Ember because Ember is a complete framework and React is not. When we made the decision, React had a relatively small community behind it compared to what Ember had. The same decision would be a much more difficult one now, given React’s recent growth in popularity, but this was early-to-mid 2015, and we had our way forward.

The alpha

Alpha Wolf

Now that we knew our destination was a full Ember website, we had to figure out how to get there. To test this decision, we chose to try out Ember first on our internal admin users. The initial release was to be simple: basic listings and details views for our two primary data types. The implementation of this simple release was decidedly less so.

We had purchased a pre-built theme, complete with fancy charts, widgets, collapsible content, and much more, that came with all the necessary JavaScript and CSS we would need. What we didn’t know, was that this type of setup wouldn’t just plug-and-play with Ember. Since we were new to Ember, something as simple as getting the active tab to be highlighted took us days to implement. To allow for easier hosting, we also decided we would host this Ember application on a different domain. This meant we would have to use cross-domain cookies and, for our local development, we had to run NGINX and instead of using localhost as developers usually do, use a self-resolving domain (like lvh.me).

Overall, this might not seem too bad, but we went from one domain (www.bidvine.com) to two subdomains (www, and an admin domain). Keep that number in mind, because it will get worse throughout the process. And, as every developer knows, when you overcomplicate your development environment, things inevitably go wrong. With three things now required to run our site (and more to come), a lot of time got wasted dealing with local environment issues. Not to mention, we had to expose an API for Ember at this point, which meant we had duplicated logic within our Rails code that we had to maintain for two different applications.

All said and done, the alpha test went well for all involved. The admins were happy to have a fancy, responsive new website to use that didn’t crash all the time, and we had a proof-of-concept that we could build our site using Ember.

The beta (goes wrong)

Beta

Confident in our understanding of Ember, we endeavoured to move a piece of user functionality to Ember next. Our user onboarding procedure on the supply side of the marketplace needed an overhaul and we wanted to add public user profiles, so we started there. Again, the plan was simple, but it ballooned the complexity of our setup in a way that we didn’t expect. Because our admin Ember application was using a full theme, and Ember didn’t, and doesn’t yet, have an option to conditionally include resources, we created another Ember application.

With another Ember application came a third subdomain (pro.bidvine.com, which no longer exists). We also had to now start redirecting from one subdomain to another within all the applications themselves, which introduced complexities in maintaining proper user flows throughout the site. The added complexity of these new applications also caused our staging environment to fall into irrelevance as we did not keep it up to date with the new changes.

Needless to say, we messed up. Big time. The initial release did not go well and we got called on it. Without a proper staging environment, we had a plethora of issues with session cookies being lost across subdomains, mixing HTTP and HTTPS, and redirects not working as we’d intended. The bugs were trivial and easily fixed, but we learned a valuable lesson about quality assurance.

Apart from the initial troubles of this release, we had further proven that Ember was the way to go. However, as a startup, we couldn’t justify writing off the next 3 months to rewrite the whole platform. Yes, we needed to continue to move functionality to Ember, but we also needed to overhaul it as we did. Our platform itself was a proof-of-concept and we needed to become a fully actualized platform for connecting customers with local service professionals.

The Big Boss!

The Big Boss

Rather than port all the existing functionality over at once, we chose to move the user-centric functionality, which meant the admin pages and things tied to authentication stayed on Rails. As a result, most of our website would now exist on a secondary subdomain, and the primary subdomain would have to frequently redirect out of itself. While not an ideal solution, we weren’t yet concerned with search engine optimisation (SEO), and we couldn’t justify porting it all at once.

Remember that count of subdomains? We were at two, and then three between two Ember applications and the Rails application. Well, we decided to throw a fourth into the mix at this point too: a Node.js/Express.js API. It might seem excessive, but we were planning for the future and our Rails API was minimal so it was a now-or-never decision. The complexity of the new API was minimal as well, but we did have to pipe certain requests made to the new API through to the old one for authentication purposes. Again, it sounds trivial, but when hosting with a DNS service that tries to protect you, and it blocks piped requests within a domain, unexpected complexities can foil a release.

We dubbed this release our “Big Boss” release because it was going to be a great accomplishment for us in terms of moving away from our MVP of user functionality and be the first release that would include full on-site communication tools between users. We added a number of new features, greatly improved a number of existing features, and generally made the user experience look and feel like a proper website.

The complexity of this release can’t be overstated. Mixing two APIs and three front-ends together was a nightmare from a dev-ops perspective, and there were hiccups during the release that caused a rollback, a one day delay, and a lot of lost sleep.

We triumphed, though, and the now we had an exciting new way for our users to interact. User reception of the changes made was overwhelmingly positive, and the website performance improved significantly. At this point, the only remaining step was to go full Ember + Node and cut the cord with Rails.

Going off the rails on a crazy train

Crazy Train

With experience comes wisdom, and we knew what we were doing now. We moved the remaining admin and authentication functionality from Rails to Ember + Node without anything of particular importance to note. Decisions were made, such as what type of authentication to use and how to implement user sessions, but from a development perspective, this final length of the journey went relatively smoothly compared to the previous hills and valleys.

Our initial angst about splitting our site into two Ember applications proved to be unjustified. In retrospect, having these two platforms exist separately is a huge benefit because the two applications perform very different functions and use the data models in very different ways. We were, finally, a stable platform once again. Although this might seem like the end of the journey, other considerations persisted, and continue to persist, which must always be accounted for.

Ember knows best

Big Brother

As every developer knows, there are many correct ways to write code and many more wrong ways. Some programming languages have more bad ways than good ways, and JavaScript is one of those languages. The great thing about Ember, though, is that it shows you the right way much like a good parent would. But, like with any parent, the rules change over time. Where once a certain piece of code was allowed, it becomes expressly forbidden. Where once some code worked one way, the same code now works an entirely different way. And, when such changes occur, all of the open source add-ons you’re using change too.

Upgrading from Ember 1 to Ember 2 (on two Ember applications) was a huge task! Every little trick, every private API, every loophole you found to shortcut the way Ember really wanted you to code, gets deprecated eventually by the Ember team. Every minor version upgrade of Ember introduces some new deprecations or bugs to our applications that must be resolved. This wasn’t just a one-time deal either: Ember is always changing to make sure it stays relevant. Such is the appeal of Ember, and also the bane of an Ember developer’s existence.

How (not) to SEO

SEO???

Great, so we need to care about SEO now. Search engine optimisation is an important aspect of any website, especially one such as Bidvine that depends on users being able to find us when they need something done. Well, it’s too bad that Google (and every other webcrawler) doesn’t know how to deal with our fancy new JavaScript website! Google thinks our site has no content because it’s all generated by JavaScript. We’re doomed, right? No – mostly.

Luckily, some smart people built tools to get around this glaring problem. These tools will intercept requests from crawlers to our website and render the page before sending it back to the crawler. That way, Google sees a full HTML response and can index our website properly. Prerender is such a tool, and it works great for us. It does, however, cost money. Additionally, Google has changed its stance on such tools, preferring they aren’t used now. Tools like this could be used to “cloak” (disguise) your website and Google says they say they can handle JavaScript-based websites now. At the time of this writing, that is not true – at least not that we have been able to make work – and SEO with Ember (or any JavaScript framework for that matter) is a constant battle. We still rely on Prerender to make us visible to the SEO world.

What Now?

Ember

Looking back over a year and a half of using Ember, we are struck by how helpful the Ember community has been. The emberjs.com guides, all the Ember add-ons, all the StackOverflow questions and answers, and all the Ember GitHub boards are invaluable resources to an Ember developer. If we had to make the same decision again about what framework to build our site on, we would still choose Ember.

We continue to add new features, enhance existing features, and generally improve the overall stability and accessibility of our website. We contribute back to the Ember community where we can and we thank the wonderful Ember Core Team and community for being awesome at what they do. What’s next? We’ll be sure to keep up to date on Ember so that we can continue to release exciting new features to the users of Bidvine!

Leave a Reply

Your email address will not be published. Required fields are marked *