An Unexpected Journey to the Summit of Mt. Chromium.

A blog post in response to Station Desktop 1.0: the technical back story post by Hugo Mano.

This week, Hugo from Station shared an excellent technical back story on some of the experiences they had building and developing their desktop app.  His blog instantly struck a chord with us, as we saw many parallels between their journey and ours - particularly their experiences with Electron. So, it's inspired us to share some insights into how we developed Wavebox, and why we made our important move from Electron to Chromium in Autumn last year.

For those of you who don't know, Station offered a similar service to Wavebox, but announced their decision to pivot this week and craft a new version of their product as a Chrome extension. We would like to wish the Station team all the best with this new endeavor - and also say a sad goodbye to a project that technically achieved so much in such a short space of time.

Station and the original version of Wavebox (Wmail) were both based on Electron, which is a commonly used framework that embeds parts of Chromium code and NodeJS runtimes, allowing you to write cross-platform desktop applications. Electron is not a browser, but it enables you to build feature-rich desktop applications using familiar web technologies.

It's worth taking a few moments to note that Electron is a brilliant framework that's easy to use and has a huge amount of community support around it. Popular apps such as VSCode use it, and use it well.

When we started building Wavebox we had an excellent run with Electron. But, as we started to push Electron to the limit, just like Station, we encountered technical limitations and problems which meant we were constantly spinning plates rather than developing new features. As a product lead company, this wasn't where we wanted to be, and so after some difficult decisions and a summer of very intense development, we announced in November 2019 that Wavebox was leaving Electron to be natively built on Chromium.

So why did we choose Electron originally? Well, it's temptingly easy to write a web app in Electron. If you clone the quick-start project and change the URL to something on the world wide web, theoretically, you can be browsing in less time than what it would take you to install Chrome. Although tempting, this lulls you into a false sense of security. Things start to get more complicated, very quickly. In fact, if you take time to read through the Electron documentation, they are quite clear on their stance regarding browsers...

"When working with Electron, it is important to understand that Electron is not a web browser."

However the fact is that when we started Wavebox, we weren't building a browser at all. Our original app was just a Gmail and Google Inbox viewer, with limited access to the web. But as our user base grew (quickly) and requests starting pouring in for browser-like features such as spell-check, right-click menus, address bar, tabs, bookmarks, in no time it became painfully clear that we had inadvertently started building a new browser....in Electron.

At first, all looked good. We stuck a website into a page and it worked just fine. But then we started to find things that didn't work. These were things that worked on the web, but not in Electron. Why? Because Electron takes all the browsery stuff out of Chromium, leaving you to add it all back in again - if you want to that is. And we needed to because our users were crying out for it.

To give you some idea of the kind of problems we encountered, here are a few examples of browser-type things we had big trouble with. Bear in mind that these are exactly the same problems Station (and similar apps) were having too:

1. Opening windows
Take the following perfectly valid code that you'd see on the web:

window.open().document.writeln('<h1>Hello world</h1>')

In web browsers and as per the web spec, this opens the new window in the same process and allows the creating window to access the new one and manipulate it as needed (in the above case to write Hello world in the new window). On the other hand, Electron places each window in a different process, which means it doesn't work. There is a proxy object which tries to polyfill this behavior, but there are plenty of instances where this doesn't work.

We could use the nativeWindowOpener setting, but it has its downsides, such as being unable to add UI to the opened window. To combat this, we developed a hugely complex matching engine (see WindowOpenEngine for a scale of the code) that picked one of three window types to use, giving us maximum compatibility when needed or maximum features whenever possible. Users noticed, and it wasn't perfect 😢.

2. PDFs
Opening PDFs on the web is a standard feature for most browsers. When we were building Wavebox, this was part of Electron, but there were some caveats, namely, it didn't work in iframes, and print support was missing. This broke several sites. We did managed to add printing in the end, but never managed to add support for iframes - instead, we had to resort to ghastly hacks to get a reasonable experience. Take a look at our Trello patch if you're prepared to skip your lunch! 🤢

3. Chromium version lag
The internet is a constantly changing beast, and the mainstream browsers release frequently. When we were using Electron, there was quite a lag in staying up-to-date with Chromium.This lag landed us with many problems in providing support for sites and some sites prompting users to update their browser. Although we worked hard to stay up to date with Electron and brought forward several patches from Chromium, it was too much of a technical burden to keep the whole framework up to date ourselves.  (Note that Electron is now much better at staying up-to-date with Chromium than it was back then.)

4. Electron updates
Updating Electron had its own problems. We constantly found ourselves using the more secure and niche combination of features from the framework. These weren't generally tested by others and meant that an Electron update resulted in a stream of bugs. These would often only manifest themselves in edge cases and on less-often used web apps, and we just didn't have the resources to test everything. For us, these releases allowed us to keep the core engine up-to-date, but for our users, it usually meant a new set of bugs to disrupt their day-to-day work 🐛🐛.. Not good.

5. Sandboxing
We always tried to build the best Wavebox we could. But during this time this sometimes brought its own problems. Take this one for example - a new version of macOS shipped and it stopped form controls from rendering. After some investigation, we found that this was down to the sandboxing feature (an important security feature). Behind the scenes, Chromium had rewritten the sandbox, but Electron had yet to update to it. Most Electron developers found themselves unaffected because they'd restyled their form controls, or enabling sandboxing wasn't a feature they needed to run their local code. We, on the other hand, faced a dilemma to either disable sandboxing (not an option because of security) or update the sandbox ourselves (and we just didn't have the technical firepower or expertise at the time to do this).

We didn't know at the time, but it would take three months for this issue to be fixed upstream. Luckily we managed to find a workaround for the problem. It wasn't nice, but we discovered that offsetting the zoom level by 0.001 made it work -  not the fix we wanted to use, but the only option we had at the time to keep our users working.

6. Extensions
Extension support was our most requested feature, and we put a huge effort into this, as did Station. We both had different implementations, both with differing levels of support, and we can truly appreciate the hard work Station put into this - we we're experiencing exactly the same pain! Although the Chrome Extension API is pretty stable and well documented there are nuances that are difficult to reproduce and figure out. For example, knowing that a tab-created event should fire before the call to create it completes is undocumented. For us, these kinds of things would cause issues and would manifest themselves as weird bugs in some extensions. It took several hours to reproduce these and eventually track them down, often trying to reverse engineer minified code to figure out what's going wrong.

One day, the constant plate spinning became too much, and something had to be done. We can only imagine that Station must have had similar conversations regarding constant change and how impossible it was to stay up-to-date and keep everything working. For us, the decision came after a particularly difficult week when everything threatened to go pear-shaped. To try and get across our past pain, here are the issues we saw in that one week, and our internal time estimates for fixing them....

• A macOS update breaks form controls. We find that updating to the new Electron beta fixes this (yay!), but we can't update to this version because it breaks some other things. PDFs for example. 5-6 days, possibly more, depending on what else we find.
• An extension stops working and we have to figure out the root cause of the issue by reverse engineering a bunch of minified code. It takes us 2 days to figure out the cause and an additional day to add a fix and re-test all the other extensions. 3 days in total.
• A new bug report comes in about how a website unexpectedly stopped working because we do something in a slightly non-standard way. Again we have to reverse engineer what's happening and decide a full patch isn't feasible in Electron. We end up creating a bug fix specifically for that website. 2 days.
• Users with certain hardware configurations start reporting that when Wavebox launches the window just renders white with no content. Resizing the window fixes the problem. We can't reproduce the issue ourselves and go about trying to resize the window on startup and blindly tweaking Electron preferences that may fix this problem. 4 days. (We never actually found the cause or fix for this in Electron, it just fixed itself! Possibly a graphics driver issue).

We prevented things going completely pear shaped by spending days fixing sporadic bugs like these, when we should have been shipping new features. Hugo describes this exact same problem in his blog:

"While the acquired technical expertise about making a browser on Electron made us proud, our velocity for shipping user-facing features slowed down significantly."

There were so many fixes we should have done, but couldn't due to lack of time and resources and we always ended up trying to fix the broken plates and with each fix came additional technical debt.

Although it wasn't in our original plan, we ended up misusing our underlying tech stack. But we had a great product, thousands of supportive users, and a strong vision for the future. And so around April 2019, we decided to do something about it, and there were only a few options available to us:

• Get more technical firepower and code our way out of the hole
• Cross our fingers, hope Electron would add more of the features we needed and stabilize, keep calm and carry on?
• Take the bull by the horns, and build a better browser.

Of course, we decided to go with option 3, and immediately started planning what a new version of Wavebox would look like - we'd take everything that we learned from our Electron experience, and start building a scalable browser for the future.

At about this time, Brave had just made the successful jump from Electron to Chromium, and it looked very promising - our users could have all the browser features they wanted, and we would no longer be at the mercy of Electron bugs and updates. We would also have a solid foundation upon which to built Wavebox's unique features for seamless app working - such as cookie sandboxing, app sleeping, CPU management, notification engine and unified search. There was only one problem....the switch to Chromium was a HUGE mountain to climb.

If Electron is Mount Snowdon, then Chromium is Everest, and you've been promoted from capable rambler to world-class explorer overnight.

It would take many months, and some inspirational coding, to build our browser vision into Chromium. Here's what we learned about Chromium during this period of development:

• You'll need big beefy machines. Starting your first Electron app takes just seconds on a modern laptop. To build Chromium takes hours, even on the fastest box. You're also going to need a machine for each target platform and OS.
• Prepare to be on your own. There's a strong community around Electron and just a quick web search will bring up handy Stackoverflow results. Not as much with Chromium, erm, you have to just figure a lot of it out.
• Don't throw away 99% of your code, throw it ALL away. Yep, you're starting from scratch. The architecture of Chromium differs from Electron significantly, and you need to be prepared to write a lot of C++ as opposed to JavaScript.
• Documentation? Forget it! There's no tutorial for this hike. You're going to spend a lot of time reading a lot of code. When you come to add new features, half the battle is finding where to add it!
• Write your own build tools and set up your own distribution and updating system. Someone else has done all the hard work for you with Electron, again, not so with Chromium.
• Prepare to implement browser features. Although Chromium is a fully-fledged browser, it is missing some key components. These are added by vendors such as Google in Chrome or Microsoft in Edge. You're going to have to do this work too and there's no shortcut. This is something that came as a bit of a shock to us at the time.

It took our technical team months of hard work to ship the first Wavebox 10 alpha, and that was just the beginning. More months of coding followed, plus all the project planning and marketing campaigns needed to migrate existing users to the new platform. Luckily, everything went smoothly, and with the lure of full extension support, better efficiency, and fewer bugs, it didn't take much to persuade our users to make the switch with us to Chromium.

For us, the move to Chromium made absolute sense. It took a huge effort from our team, but it's paid off, and we're now in such an exciting place.

Everything is faster. All those frustrating browser bugs are a thing of the past. Extensions work brilliantly. Chromium is easily and regularly updated, and each release comes with new features and improvements. In the past few months we've released some amazing new features like Connect, an in-browser remote-working service for teams (chat, voice, video, screen-share) and Brainbox, a creative writing tool that links our browser to OpenAI's GPT-3 artificial intelligence API.

We've learned a lot from the setbacks we had with Electron, which are clear with hindsight. That said, we wouldn't have got to where we are today without it—Electron played an essential part in our journey. Station may no longer be in the new productivity-browser race, but they certainly pushed the envelope and kept us on our toes! We look forward to seeing what their extension becomes—and hopefully, you will be able to use it in Wavebox sometime soon, and enjoy the best of both! ❤️.

👋👋  Wavebox welcomes Station 1.0 users! Try the Wavebox 7 Day Free Trial, and use the following 40% discount code at checkout station-40 (valid until Sept 20th).

Photo by Ben Lowe on Unsplash