HttpClient and cold Observables

TL;DR: use obs.toPromise() with the Observable returned by the HttpClient library if you condiser the end-consumer will attach callbacks to the Promise before the API answers; otherwise, use .shareReplay(1) and subscribe directly a first time to the Observable.

HttpClient library, whether in NestJS or Angular, uses cold Observables.

A cold Observable will only “activate” (here: do the HTTP call) when it is subscribed to, for the simple reason that when it emits (here: when we receive response from the call), we want at least someone to listen, so that the result don’t go unheard straight to the void.

Yet, I find that in a lot of usecases involving API calls, the user which want to make the call will or will not deal with the result, depending on the situation (it could be updating a stock, which returns the resulting inventory, which we want to store somewhere or not). I’m talking about a situation where you’re developping a module to place calls, used by consumers services.

consumer <-> HTTP module <-> remote API

My first trial involved subscribing to the Observable directly in my HTTP module, before handing the Observable to the consumer.

const obs = httpClient.get(url);
obs.subscribe( () => {}, () => {} );
return obs;

Note that I provide two empty callbacks in .subscribe to avoid nuclear mole bomb (see previous post) ?

This works, but it works too much: if you observe API calls in the Network tab of your dev tools, you’ll see 2 API calls at once. Why that? Because the way it’s done, the cold Observable given by HttpClient will activate n times if you subscribe n times to it. So here, it’s fine if the consumer does not subscribe it, but possibly problematic if it does.

For that matter, there’s a nice Rxjs operator: .share(). (Btw, don’t let people who say it’s equivalent to .publish().refCount(): it’s not.)

const obs = httpClient.get(url).pipe( share() );
obs.subscribe( () => {}, () => {} );
return obs;

This operator will share the result of one call with all subscribers at the time of result coming: nice! But actually not enough. If the call is real fast, and the consumer takes time to subscribe, it will miss the first result and another call will be made (resulting again in two calls, which we really want to avoid).

So there’s another Rxjs operator for that: .shareReplay(1).

const obs = httpClient.get(url).pipe( shareReplay(1) );
obs.subscribe( () => {}, () => {} );
return obs;

This operator will share the last result it got (and HttpClient will only ever emit one result, so we’re fine with that) with any present or future subscribers. Awesome !

Now all that seems a bit far-fetched. There’s a way simpler solution, but note that it’s not appropriate if you think the consumer will attach callbacks later on, after the API has sent results (in which case, error-handling seems to be simply impossible). It’s to use the Observable .toPromise() function. Why that? Because due to the nature of the 2 structures (Observable and Promise), .toPromise() has to subscribe to the Observable, activating it in the same time, so we don’t have to take care of forcing the call anymore. Then, you handle a Promise to the consumer. This Promise will receive the first emission of the Observable (and there will be maximum one, so that’s fine), and keep it to show it to any callback the consumer will attach to .then().

const obs = httpClient.get(url);
return obs.toPromise();

Only caveat: if the API call gets into an error, and if the consumer does not handle error on the Promise before that, then you’re doomed, and the error is likely to bubble up and crash the process. At the time of writing, I found no way to handle this situation, except by interecepting the error with the Observable .catchError() operator and providing a default replacement value, which is probably not the best on the consumer side…

Own NPM repository for shared code

Splitting our services into microservices had led us to wonder: what do we do with the shared code? Especially code for basic functions, such as logging or error handling.

Because the code of each microservice lives in its own project folder, and we obviously can’t replicate our shared code in each project folder, we have to place it in a separate shared/common/utils folder outside of any project folder. But how to access an external directory from our projects then? Solutions with npm –link proved to be uneasy to set up, and were causing problems with namespace aliases. Plus, if your common code evolves with even a minor breaking-change, and all your projects are linked to it that way, you have to make the required adaptations in all your projects before you can go on working on them. Not so flexible.

In the end, we decided to go the long, but righteous way: to store such common code in a special project of its own, which we would build as versioned npm package, store on a private repository, and access from any of our projects. It was actually pretty easy to do. Let’s check the few steps to get there:

Host a private NPM repository

Seems like all the thing seen from afar, but it can actually be a breeze, provided you have a server at hand which can run Docker.

We used Verdaccio, which requires no setup at all. Check out https://hub.docker.com/r/verdaccio/verdaccio/ for details. Publishing restrictions on the generated repository are not covered here. Simply make sure you mount a volume to store published packages, as in docker-compose.yml:

volumes:
- /path/to/storage:/verdaccio/storage

Create the shared code library project

We use a NestJs project, compiled with Typescript. Your shared code should reside in /src as usual. Now for the configuration of the library building and publishing:

Notable options within tsconfig.json:

{
"compilerOptions": {
"module": "commonjs",
"declaration": true,
...
"outDir": "./dist",
"rootDir": "./src",
"baseUrl": "./",
"noLib": false
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules"]
}

tconfig.build.json

{
"extends": "./tsconfig.json",
"exclude": ["node_modules", "test", "dist", "**/*spec.ts"]
}

package.json

{
"name": "@ceneau/backoffice",
"version": "1.0.0",
...
"main": "dist/index.js",
"files": [
"dist/**/*",
"*.md"
],
"scripts": {
"build": "tsc -p tsconfig.build.json",
...
},
"publishConfig": {
"access": "restricted"
},
...
}

index.ts should be a barrel file at the root of src/ whose goal is to re-export every objects you want to export in the package. index.ts will be transpiled and become dist/index.js, the root of your exported objects.

With that, the library is ready for building and publishing, but we still have to tell it where to store the generated package, as well as few optional behaviors to follow on each build.

.npmrc (at project’s root) – even if you use yarn

@ceneau:registry=http://url.to.private.npm.repository/

.yarnrc (at project’s root) – obviously if you use yarn to build

version-commit-hooks false
version-git-tag false
version-git-message "Ceneau backoffice common - v%s"

These options handles the generation of tags for each build, and associated git message. Check documentation for more details.

Now, to publish a new version of the shared code:

npm adduser --registry http://url.to.private.npm.repository/

This, if you created your npm repository with no publishing restriction, allows you to create a user within it. You’ll be ask for details and password.

yarn publish

Builds and publishes the package. Hooray !

Use your common library in projects

Now that we’re here, the only remaining trick is to tell your project where to search for your common packages, in addition to the other default npm repositories.

.npmrc (at project’s root) – even if you use yarn

@ceneau:registry=http://url.to.private.npm.repository/

package.json

{
"dependencies": {
"@ceneau/backoffice": "^1",
...
},
...
}

You should be good to go !

Bonus: recover from wrong npm config

If you accidentally changed your global npm registry, your next npm/yarn commands are likely to fail (right now with a 500 error coming from I have no idea). My NPM config was showing some concerning bits:

> npm config list
; userconfig C:\Users\L.npmrc
registry = "http://npm-repo.x.ceneau.com/"

Quick! Hurry and set back the default registry up !

> npm set registry https://registry.npmjs.org/