I recently gave a talk at SoCal Code Camp that I had titled (humorously, but in retrospect, inaccurately) “Node Diggity, Node Doubt: Designing Lasting APIs With Confidence”. I realized as the event date approached that I really wanted to only talk about two topics that were lessons learned from two recent projects that we undertook at AktaryTech: proper separation of concerns and sizing/securing data I/O. The talk was well-received by the largely beginner-level (or Node-curious) audience, but I feel that these lessons are important enough to reiterate here in the hopes that my real target audience — people who have built a few MeanJS  or MeanIO apps and are finding that their little MVP has some legs and needs to grow — will find them and get some value.

Separation of Concerns and Dependency Injection

MeanJS and MeanIO do a great job of scaffolding out some basic code for you, but they can’t help you once that scaffold is there… at that point, it’s up to you to make proper engineering decisions (I should point out here that my experience with the generators is much more focused on MeanJS, so that is the one I’m going to use in my examples.) The specific issue we’re going to deal with here is the fact that by default, you have to know that you likely may need to create another layer to your API code that isn’t event hinted at in the generator scaffold — a layer that will separate your concerns and give you the ability to leverage dependency injection. By default, MeanJS will create the following server-side folder structure: Version 0.3.0:

Version 0.4.2:

This would lead many rookies to think that the generator has given all of the pieces of the puzzle and that they just need to fill in the blanks in those folders. More experienced developers will know that APIs don’t exist in a vacuum (especially not these days — often APIs need to work with 3rd party APIs or other micro services) so they will add a layer of abstraction or two to work with these resources.

This is the missing piece: the idea that we will use outside resources and the code for resources should not be intermingled with our controller code. We use two folders for this: resources and providers. For example, if we have an API that makes calls to Stripe for payments, we’ll set up the specific bits that interact with the Stripe API in a resource, then abstract that into a provider. This gives us multiple benefits:

• much, much DRYer code
• if Stripe changes their API, we only have to update our code in one place
• we can now test all of the Stripe (and indeed, any payment) calls without having to do anything with the request or response objects (that we would have otherwise had to mock if we left this code in the controller)
• if we need to replace Stripe with Venmo or some other provider, we can do it in phases at the resource level and slowly migrate code over through providers — the controllers will have to barely change… if at all.

There are likely plenty of Node modules out there to help you achieve this, but one that we like to use is CRaP — a framework that allows you to declaratively state dependencies and that will auto-inject them for you at runtime.

It should be noted that this can be achieved without any frameworks too… we just choose to use one.

Sizing and securing data I/O

Another common mistake with the MeanJS generator is not realizing how much or what kind of data is actually going out the door. Isomorphic design is a benefit of full-stack javascript, but it can also be quite dangerous in production if you’re not careful.

It’s not uncommon to create a Mongoose model with many attributes that point to other documents, some arrays, or a combination thereof. If this happens, you need to be aware that in a worst-case scenario, an array of populated subdocuments (possibly with their own subdocuments) can get very large very fast and you won’t notice it until you start getting 500 errors in production.

A common (stylized for simplicity) example might be the following:

This all looks very simple and innocent, doesn’t it?

This gives us objects that looks something like this (I’m omitting the _id property for brevity):

Let’s consider the scenario that you want to list articles that people named Bob can edit. How would you do that? One way would be to do the following:

Seems ok, right? But what is actually being returned? Let’s take a look at a single record being returned in the filtered array:

Do you really want your cost data going out the door with simple article content? How about users passwords?

I suspect not. What happens if Users get hundreds of Departments added to their array? How big will this result get just because of a subdocument we don’t even care about?

The astute observer will say that we can simply use the more advanced form of Query.populate that looks like this:

… to solve this problem, and they would be right — in this very simple, contrived example.

BUT what they miss is that doing so is not a DRY approach. You may save yourself from this one time, but will you remember EVERY time you want to return an article or list articles?

What if you DO want to list costs for some users, but not for others? A better approach is to declare a policy for articles.

For our solution to this, we chose the Propex module. This allows us to declare a policy in a px.js file in the root of our project, and let propex handle everything that comes into and goes out of our API.

In our example, px.js might look like this:

Also, to make this work, we have to update our controller to modify the return array using the px we defined:

This gives us the ability to split public and protected data and maintain a single policy for every article that goes out the door. Now that we’ve implemented this, let’s take a look at the output object (as a public user):

I’d say that’s a bit cleaner and more secure, wouldn’t you?