Webpack JavaScript SDK

This article details the evolution of one particular aspect of the design for Jexia’s Javascript SDK. I will explain our goal, the problem that we needed to solve, the few intermediate designs we came up with along the way and, finally, the solution that we implemented in the end.

First the context: Jexia is a company focused on building technology that helps developers deliver their products faster, with higher quality and security. The Javascript SDK is built to help developers easily access a subset of functionalities that our platform provides:

– Storing and working with data

– Storing and working with files

– Interacting with the integrated authentication/authorisation system

– Subscribing to and consuming real-time notifications for operations executed on data/files/users

The SDK first of all needs to cover all of these functionalities, but at the same time needs to expose a friendly interface that can be integrated effortlessly in our client’s codebase. Our goal is to find the perfect balance between ease of use and feature richness.

Jexia’s JavaScript SDK: the challenge

As we had a small team at our disposal when building the SDK, we definitely had to make sure we would obtain the biggest impact with the least effort. We wanted to support as many different environments and frameworks as possible, so we had to intensively optimize our development process. Building separate browser-specific and node-specific SDKs was out of the question, we needed it to be built once and run everywhere. So the main design decision we made was to have a single Javascript codebase that can cover all platforms that we are targeting: browser, Node.JS and mobile Javascript.

One of the main issues that we ran into when starting to build the Javascript SDK was how we can handle the dependencies of the SDK library (HTTP requests, Websocket connections, Promises, etc.). The problem here is that we want the SDK library to behave in the same way on all platforms, however there are small differences between these environments that can have an impact on that behavior. For example, all browsers have native implementations of fetch or Websocket, however Node.JS does not, which meant we would have to rely on third party libraries to offer some of the functionality to us.

How do we build an SDK that works on both Node.js and browser, without containing Node.JS or browser specific code?

I imagine that the initial temptation for most Javascript developers would be to go with the obvious solution: lace the codebase with statements like “if we’re running on _this_ platform, do _that_” . It was the first idea we passed around the team and then promptly dropped.

The pitfall of this kind of approach is that it doesn’t scale, as functionality and dependencies are brought in. It also tends to lead to a breakable implementation that instead of failing for corner cases, fails for any use case that wasn’t taken into consideration at design time.

And it introduces branching paths in your code that over time evolve and make it harder to follow the logic without actually running and debugging. Remember, we had a small team and we needed to be smart about how we use our time.

Our Solution

The first idea we tried to implement was adding static dependencies to the SDK project and bundle them along with the SDK. The argument was that we need to ensure the SDK has the same behavior in all environments. Relying on native implementations might bring specific coding and testing overhead.

We chose some popular libraries that handle HTTP requests and the rest of the things we needed and started implementing. After a very short while, we realised we were forcing Node users to clutter their environment and Node process with all these libraries that they didn’t directly need. We were also forcing browser developers to increase their app’s footprint by bringing along all of these libraries.

Intermediate problem 1: could work on both, but has some serious drawbacks.

The second step was to remove dependencies for the browser, because browser apps are the most affected by an increased download size. We took the decision to rely on standards and standard-compliant implementations, to minimize issues that might pop up in different environments. We chose the fetch standard, Websocket standard API and Promises standard.

So we switched to more light-weight libraries for Node.JS that offer the bare minimum of what we needed and that were compliant to these standards that had native implementations on the browser side. This way browsers use the native implementations while the Node SDK still brings in dependencies, but more “universal” and lightweight.

Intermediate problem 2: how can we implement this in such a way that the code dynamically chooses if it should use a native implementation or a third party dependency, without knowing what platform it is running on?

This can be solved in two different moments: at bundling time or at run-time. Run-time was pretty tricky, there is a proposal for dynamic imports in Javascript, but it is not implemented yet. We went with the first option, solving it at bundling time, so we integrated Webpack (which we planned to use anyway for bundling) and started doing research.

The fuzzy idea was to apply local imports in the modules that needed dependencies and to use Webpack to choose the correct platform-specific dependency at bundling time. We were thinking of having static imports that point to specific targets, but have these targets point to different dependencies based on bundling time decisions.

Intermediate problem 3: we couldn’t make it work. We didn’t find a technical solution to switch the target for the imports at bundling time.

We took a step back and decided to use a different mechanism compared to the Javascript import approach we had. We started using global variables to hold the dependencies.

Our Webpack configuration was built as follows: we had two entry points and two output bundles, one for browser apps and one for Node.JS. The Webpack entry points are Javascript files that import the SDK internals and the dependencies for that platform, define the global variables that the SDK knows to use for each dependency and then export the SDK’s API functions/classes.

By removing import statements from the SDK, moving them into the Webpack entry point files and using global variables we finally achieved our initial goals: to have a single code base that can run on both platforms, without bringing unnecessary dependencies to browser apps and without writing any platform-specific code (except for the Webpack entry point files, which are basically part of the development environment and not part of the SDK per-se).

There was also an element of customization to the SDK now, because the clients could redefine the global variables and use the SDK with other dependencies than what it came with. It was a win-win all around. Except that…

Intermediate problem 4: the Node.JS SDK’s dependencies are pretty much hardcoded and the client might not want polluting the development environment. Also, we’re using globals. Globals, man.

So next we set out to remove dependencies for the Node bundle and avoid bringing a bunch of baggage into people’s development environments. Ideally we would allow the user to choose what libraries the SDK uses for its dependencies. So naturally we found a solution through dependency injection. We removed the reliance on global variables, instead using local variables that were passed as parameters from above in the call stack.

Thus, we uncoupled our low-level logic from these dependencies and elevated the dependency management to a higher level in the SDK structure. We went further and injected these dependencies from the outside into the SDK itself, which meant that the main SDK class (we call it the Jexia Client) would now ask for these dependencies to be given to it on initialization. This brings the dependencies to the front and makes them visible to our end user. Nothing shady happening under the hood here :).

This also means that for Node applications it’s easy to customize and adapt the SDK to the stuff you’re already using?—as long as the dependencies comply with the standards, they will work.

Intermediate problem 5: asking the client to inject dependencies when building a browser app doesn’t really make sense, as they would use the default native implementations anyway.

This was already after we solved our core problems and started paying more attention to detail. This was solved quite naturally, by leveraging our Webpack entry points to hide the whole dependency injection thing on the browser side. By writing factory methods that use browser native implementations by default, we simplify the SDK usage on browsers by removing the entire dependency injection solution from the equation. The SDK’s API is slightly different, simplified and customized for the browser development experience. And all this without using ifs and while providing a 23kb bundle to the browser developers 🙂

The End
I should note that I purposefully left some details out of the post in order to make it easier to follow. There are some elements that complicated the whole situation, like for example the fact that we use Typescript for the development of the SDK, but deliver it as a Javascript focused SDK. I might have also skipped some intermediate steps while trying to make it look like a neater process than it actually was.

I’d like to close this with a kind of fairy tale ‘they lived happily ever after’ conclusion, but that’s not exactly what usually happens in software development. Right now we are happy with the decisions we took and the design we came up with, however we are perfectly aware that we will make changes in the future, due to developers’ changing needs. We do have plans about how we can extend this solution in order to allow Node.JS devs to use dependencies that don’t comply with the standards, but it all depends on how devs are going to want to use our SDK.

I hope this article helped give you an insight into the design process of a particularly hairy problem and how iterating on the design while sticking to a few values like removal of complexity, striving for continuous improvement and not compromising on the small details can actually lead you to a much cleaner and better solution in the long run.

Tags: , , ,

Categorised in:

This post was written by Editorial Team

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.