NodeJS in the enterprise world - the building infrastructure


I have decided to launch a series of blog posts to share the experience we gathered in the past years about NodeJS and its introduction to the enterprise world, which is far beyond what you meet when reading todo tutorials in regard to design, orchestration and service complexity.

Our showcase is a real-time financial transactional system we implemented and put in production for some time now. I hope you find it worthy to read.

Is NodeJS strong or matured or stable enough to be found on?

Paypal and Strongloop reduced this question to the religious dilemma of IT fossils actually. 

The article “Node.js is taking over the Enterprise – whether you like it or not”. has also discussed this matter.

The infographic by Strongloop is not a fairy tale at all, NodeJS is as good as “stunned” you will be while looking over it. :)
It was a great adventure for us to dive into no man’s land, where at that time, Java and .Net and even older platforms were considered as king. Or - to be more precise - we are just talking about the conservative and cautious nature of enterprise world, when the need for changes rises.

In this series, I try my best to share our success and failures discussing architectural concepts (in Node.JS) and their application (in enterprise environment).

The problem

A financial system has to deal with transactions fragmented in space and time involving multiple protocols and data-exchange formats. You relay transfers between financial institutes and payment providers across the globe, taking banking days, cultural differences, multiple currencies and a wide range of transfer types and processes into account. In contrast with general belief, financial processes require strict human interaction and control at several points. It is a natural consequence of when it is money we have to care about.
Sounds nasty, but do not let complexity scares you. The success of taming such wild animal makes you forget all scratches and bites you suffered from. :)

Our job was to design and implement a new transactional management system from the ground, put in production with a minimal feature set and expand it continuously as we pass forward in time.
In the first phase of the project, interoperability had to be provided between payment providers like Earthport, GCEN, Trustpay and the internal transaction management system.
Payment providers allows you to connect to distant financial players placed in a country with "orthogonal" financial or legal system.
Notwithstanding that SWIFT is a very matured standard for financial activities used by most of the banks we know, payment providers are using their own formats and interfaces.
Supporting the manual decision points, an administration and management UI also had to be designed.

Building (the project)

The alpha of everything. 

  • how to orchestrate the project, the files?

  • how to build up the development processes and how to support them?

  • how to ensure code and process quality?

Let me focus on the client side first: it involves multiple tasks, processes and roles to care about.
Of course, the culture of your company weights a lot when it is about development process design and task distribution, but mainly you have 3 main aspects:

  • styles

  • templates

  • services

In this article you might find that I gave serious weight to the UI and UX. I have done it for a reason. I think, they are mission critical areas to be managed properly, even if in EE world, the tradition to create great UX does not really exist.

“Skymap” is the best word to describe the JS and the Web world, where “diversity over monochromacity” is the main rule, where "stars" are shining for only months or quarters then they begin to fall. Independently from the technology we used, I would like to be as generic as possible, not whispering that any of the chosen solutions would be THE tool for you. In the end, always the characteristics of your project shows the way in the jungle of stuffs and never the most heated dispute or hype.

Anyway, we decided to use a CSS preprocessor and an HTML templating and to keep the code of the server- and client-side in harmony, CommonJS as module format. I know, data-binding always gives a twist to the story, I won't forget about it, just let me go forth.

Do not let this diagram scares you, I just wanted to show which tasks I would like to include in my building process.

The workflow of the CSS preprocessing is simple:

  • source files have to compiled
  • linting has to be performed for quality measurements
  • rules have to be prefixed
  • results has to be minified

(Note: That workflow might involve external styles and files.)

The processing of template files follows similar way: files need to be compiled and minified accumulating external dependencies and configurations.
Even if we are working with dynamic pages evaluated only when a GET request comes in, we have to execute the same building process to check syntax and other measurements during development time.

The really swingy part is the JS of course. You might

  • define own services
  • use an MV* library or framework which brings in its own style and conventions
  • use external JS functions, libraries or plugins

Following the logic of SPA, you have to produce one single entity at the end, which bundles all resources you need.

The static content is very straightforward. Minor steps like image compression, copy, small transforms must be performed.

One can implement such process faster than drawing a diagram presented above, believe me. :)

We tried to keep the building process as dry and simple as possible, but real-life always thought us a lesson about being flexible. Even in the strictest corporate culture, you might end up with dealing styles or replacing images as a developer or changing typos in the page templates as a designer. Flexibility is the direction we found the most precious to follow.

Our project is not a simple SPA at all, we have to develop several pages, however we have to spend some time to understand our problem to be solved.
Several circumstances have to be respected:
the financial company already has an own identity: styles, images, colours, etc.
Even if the purpose and services of the pages are different, at technical level, they share an enormous subset of styles and services and even templates.

Following that logic, let me introduce an “abstract page”, which is a union of such considerations and which is a basis of every page you have to design and develop during the project as the diagram below shows:

Orchestration always requires you to observe and sense as many details and abstraction layers as you can handle and even more. Never forget to design your IT system for tomorrow and not today. Building is not just about executing tasks for deployment, but to aid development processes of the developers too. Please, find the diagram below:

As you can see, the black build tasks are very needed to handle the activities I mentioned above, but not enough to make the work comfortable to the developers.
Yes, I’m talking about live reload. :)
The tiny devil what gives the last detail to our build design.

"Watch" activities must be defined to monitor if a file you are working with, has been updated and then a related build process has to be launched in the background and reload the browser automatically. Of course, respecting the already existing structure of building tasks.

This concept gives you the freedom to seize the building support for your own activities. You can build and watch:

  • the style or templates or services of a given page
  • a complete page
  • the whole UI
  • the server-side
  • the complete project at once

This granularity can really make the work of the developers comfortable.

Design is ready and is waiting to be implemented…

Next time...

In the next blog post, I show you how to implement such building system I tried to introduce above and how you can refine it to serve your needs.

Part 2 has been published.

Feel free to share your thoughts in comments below, I am willing to exchange thoughts with you.
If you consider to start a new project soon or to move your stack to NodeJS and do not mind some support and consulting, please let us know.
We are open for business at Upwards Motion.

comments powered by Disqus