Category: DevOps

Posted on

Procédure de patch sur une version en prod

Cette procédure est conçue pour le cas suivant :

  • une version x.y.z est déployée en production
  • cette version contient un bug
  • des commits de développement ont été pushés sur le repository entre la version en prod et la date actuelle (si ce n’est pas le cas, il suffit de commiter le fix et de faire une nouvelle version patch standard)

La version en prod x.y.z correspond nécessairement à un tag du source-control (sinon, y’a eu un raté!).

git checkout app-name_x.y.z

A partir de là, en dev, on est iso-prod: le bug doit pouvoir être reproduit, et le fix nécessaire développé.
Créer une nouvelle branche “x.y” (sans le z, qui justement correspond aux versions patch). 

git branch x.y
git checkout x.y

Développer le fix et commiter/pusher (ca le fera sur la branche).

Tagger une nouvelle version avec un numéro de patch supérieur: le pipeline CI devrait construire et déployer l’image corrigée.

Après le déploiement du fix, il convient de copier le commit de fix sur la branche ‘main’ afin que celle-ci profite également de la correction.

Un moyen simple est le cherry-picking. Noter le hash du commit (visible via git log, ou sur Gitlab), puis, en repassant sur la main:

git checkout main
git cherry-pick <commit-hash>
git push


Posted on

Swarm + Traefik: dreadful pitfalls

Swarm is an exquisitely simple container orchestrator, and allied with Traefik for the reverse proxy + load-balancing + certificate generation, you’ve got yourself a powerful solution in a matter of hours or even minutes.

Yet, the structure of the docker-compose file used by Swarm, and the need for each web-exposed service (using a domain name) to join a Traefik network (in the Swarm definition) has often kept me perplexed, even suspicious. And one should be, as potentially dreadful pitfalls await!

Non isolation

If you want your services to be reachable via domain names, they will have to join the Swarm public network you created for Traefik, so that Traefik can proxy the requests to it when the time comes.

In other words, they will have to share the same network. This means that all exposed services can access one another with their service name (the one defined in the docker compose file), while one could have expected for services in a given stack to be isolated from other services from other stacks, if not specified differently.

That could be bearable, but then what happens if two stacks each have their own service, with the same name?

Using same service names in different stacks

It is totally allowed to use the same service name in different stacks, such as a ‘db’ service in a PHP+MySQL+phpMyAdmin stack named ‘api-server’, and another ‘db’ service in a NestJS+MariaDB stack named ‘ws-calendar’. Nobody yields any warning during the stacks deployment, and the command docker service ls will even show different full-named services, here: api-server_db and ws-calendar_db. Great!

Yet, it’s not. Indeed, if they share the same network (as it’s the case if these two services are exposed to the web via domain names and Traefik), Swarm will consider they’re the same, replicated services, basing itself only on the short name of each service, and not on its full name. As a result, it will start load-balancing between them, which might cause disastrous results, or at the very least not the ones you were expecting.

One should therefore be very cautious about the naming of the stacks’ services, which should be distinct all over your solution.

It especially applies if one would have hoped (naively) to manage services of different environments (production, staging, ci) with the same Traefik, relying only on different stacks with different service configurations. If the service names are not cautiously distinct, staging processes might very well end up modifying production databases just like that. It would be more prudent to create different Traefik networks with different Traefik instances (necessarily on different servers to avoid ports conflicts), or better, to manage totally different Swarm clusters per environment.

Posted on

Git: avoid commits of fdescribe() and fit() (jasmine/jest tests)

While writing tests, some frameworks offers the possibility to “focus” on a particular test or particular suite by running only them and not the all of them.

While this is useful when writing the tests of some component, it can be a dangerous tool when the focus is baked within the tests themselves, such as Jasmine‘s special instructions fdescribe and fit. Indeed, if these instructions make it through the source-control, it will render the whole Continuous Testing inefficient (as part of the Continuous Integration), as some errors could pass undetected by the filtered test files.

To avoid to break Continuous Testing, we’ll add a pre-flight check before any git push.

To set-up the check, we’ll create some directories at the root of our git folder, and a pre-commit file (the check file will itself be pushed in the source control):

misc/git-hooks/pre-commit

#!/bin/sh

# Hooks to do particular checks before allowing a commit.
# Configure Git to use this file for pre-commit checks:
#   git config core.hooksPath $GIT_DIR/../misc/git-hooks/

STATUS=0

# Checking that 'fdescribe(' or 'fit(' (focus test in Angular/NestJS) are not committed by mistake.
MATCHES=$(git --no-pager diff --staged -G'[fit|fdescribe]\(' -U0 --word-diff | grep -P '\-\]\{\+(fdescribe|fit)' | wc -l)
if [ $MATCHES -gt 0 ]
then
    echo "You forgot to remove all 'fit(' or 'fdescribe(' from your test files."
    STATUS=1
fi

exit $STATUS

Check this file in the source control, and have all developper pull it in time.

Each developper must then configurer its git setting by entering the following command (the exact path might have to be adapted, depending on configurations):

git config core.hooksPath $GIT_DIR/../misc/git-hooks/

CI is now protected!

Posted on

Off to Continuous Integration !

“When is a good time to adopt continuous delivery? It was yesterday.”

Indeed it’s about time !

Continuous integration, yet, is only productive with a wide-enough coverage, which is not yet the case on all aspects of the product. In a first phase, only tests on the front will be added to the pipe, followed during the year by tests on the scripts – once those have been refactored in a testable manner – and the API server.

In the meantime, this site provides some interesting reading on the subject, which you’re invited to read if you don’t master the basic principles of Continuous Integration, Delivery and Deployment: https://www.atlassian.com/continuous-delivery and all articles on the left navigation of https://www.atlassian.com/continuous-delivery/ci-vs-ci-vs-cd.

A friend of mine just whispered I should get a look at Sonar (www.sonarsource.com). Time to see if it has something to add with the Angular tool set !