Craft a complete GitLab pipeline for Angular. Part 1

This article is part of a series of two. In this first part, you'll learn GitLab pipelines basics and craft an Angular pipeline including build, tests, coverage and lint in a docker environment. The second article focuses on deployment: publish docker image and deploy to GitLab Pages.

Don't worry if you don't know GitLab, this series is a step by step guide. Along with comments, you'll find many links to the well-written documentation. If you use other CI/CD tools, it can still be interesting because the concepts and commands to execute are similar.

Getting started

Let's begin from a basic Angular app generated with ng-cli.

$ npm install --global @angular/cli
$ ng new my-app

You don't have much code to update to make it work with the pipeline. The essential part is to define each step of the pipeline in the gitlab-ci.yml file.

Take a look at how the pipeline looks like in action:

This is the GitLab pipeline you'll get by the end of the series. For now, we'll focus on the essential: the three first steps. First, you need to understand basic concepts related to pipelines. Again, very similar to what other CI/CD tools such as Jenkins, CircleCI and TeamCity have.

GitLab CI basics

Having a CI tool helps to build and test your code in a neutral environment to ensure it works on any computer and server. Sometimes it only works on your machine because of some special setup you have.

The idea is to run tasks in the CI environment. As many projects would need to run tasks, many machines are available to run the tasks. In GitLab world, we don't talk about tasks running on machines but Jobs executed by Runners.

By default, jobs run one after the other. Yet, it's possible to run several jobs in parallel by arranging them in stages.

As explained before, jobs run on runners. Yet, it's not runner responsibility to run the script on its own. Runners delegate the work to executors which comes in different flavors such as Docker, shell and ssh.

Let's practice with a dummy job running the node --version command. To make sure nodejs is available, we need the docker executor. The executor can run the command inside a docker container, built from node:12-alpine image.

job_1:
  stage: stage_a
  image: node:12-alpine
  tags:
    - docker
  script:
    - node --version

The image keyword defines the docker image to use. Yet, you need to make sure the runner picking your job implements the docker executor. It's possible to select runners for a job using the tags keyword.

A pipeline is the accumulation of all jobs you defined. Make sure to remind what jobs and runners are, so it'll be easier for you to understand the rest of the article. You can read the Getting started with GitLab CI/CD documentation if you need more details.

Install dependencies job

Before building and testing your app it's necessary to install all dependencies. You can do it by running npm install. Making it a separate job is important because it's required for the following jobs. You don't want to install dependencies and loose time afterwards.

Each job runs on GitLab runners. If you have many runners, you'll need to share the job result with other runners (the node_modules directory in our situation). You can do it with either a cache or an artifact, but in our situation, the cache is a better option.

stages:
  - install

install_dependencies:
  stage: install
  image: node:12-alpine
  tags:
    - docker
  script:
    - yarn install
    - yarn ngcc --properties es2015 --create-ivy-entry-points
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
  only:
    refs:
      - merge_requests
      - master
    changes:
      - yarn.lock

There is an extra step after dependencies install. If your project is an Ivy app, you need to run the compatibility compilation for libraries using Angular. This step is usually done when running ng build and ng test. In the pipeline, it's done beforehand to get the final working node_modules at once. Note running ngcc isn't mandatory if it's already setup in postinstall script.

Sharing the resulting node_modules with the following jobs works with the cache keyword. Two small optimizations to note here:

• the cache is invalidated only when yarn.lock file changes
• other jobs will use the pull policy to avoid uploading the cache

cache:
  key:
    files:
      - yarn.lock
  paths:
    - node_modules
  policy: pull

Other jobs will pull node_modules from the cache. It'll be available for any other pipeline and job. Make sure all runners can access the cache location. If you have a company license, your project may have both corporate runners and GitLab shared runners. Those two kinds of runners must access the cache and your company registry if applicable.

Using the only:changes keyword on this job makes sure it doesn't run if the yarn.lock has no changes. It must be combined with only:refs to make it work properly in merge requests.

The cache key is also based on this file, it means when the job runs there is no matching cache to pull, so it begins with from clean node_modules.  In this situation, if the install is too slow you can take advantage of the fallback cache key. With the fallback cache containing most of the dependencies, downloading some new ones would be quick. No demonstration for this optimisation. To be honest, I'm not sure it worths it.

Make sure this job is included in the pipeline the first time it runs. Following jobs build and test needs the cache to be set. Two suggestions to make it happen:

• Commit a first time without only:changes on this job
• Commit yarn.lock change and the job together. It's the case if you follow this article to the end (you'll add dependencies for tests report later).

If you prefer to use npm for this job, the documentation provides a short example using npm ci command. Don't forget to replace yarn.lock with package-lock.json in the aforementioned samples.

Build application job

Beside code validation, the pipeline should build the project and produce a prod-ready artifact. You can build Angular app with ng build --prod command.

The project configuration changes to output the built app to artifacts/app. This isn't mandatory but having a dedicated folder can help to gather jobs artifacts if many of them are producing some.

{  
  "projects": {
    "angular-app-example": {
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            "outputPath": "artifacts/app"
          }
        }
      }
    }
  }
}

GitLab provides many environment variables about the project and the context of the pipeline. They're always available and can be used along with the variables keyword. For instance, you can define the path to the artifact.

variables:
  PROJECT_PATH: "$CI_PROJECT_DIR"
  APP_OUTPUT_PATH: "$CI_PROJECT_DIR/artifacts/app"

build_app:
  stage: build_and_test
  image: node:12-alpine
  tags:
    - docker
  script:
    - yarn ng build --prod
  after_script:
    - cp $PROJECT_PATH/Dockerfile $APP_OUTPUT_PATH
  artifacts:
    name: "angular-app-pipeline"
    paths:
      - $APP_OUTPUT_PATH
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
    policy: pull

Note the classic ng build command comes with yarn before. It ensures, you use ng-cli from the current project and not a global installed version. In my experience, having commands as scripts in package.json is good way to keep short commands and rely on the project ng-cli.

{
  "scripts": {
    "ng": "ng",
    "build": "ng build --prod"
  }
}

$ yarn build

Extra files must be included in the artifact. Indeed, in the next article, you'll build the project docker image which needs a Dockerfile. The after_script keyword defines the commands to run after the job script.

A job can only produce a single artifact but the support for many artifact may come in the future. Yet, an artifact can include several directories if necessary. Artifacts produced during a pipeline are available for other jobs.  You can download them from many places in the UI as a zip file.

Note you may need artifacts:expire_in keyword to set an expiration date for your artifact. If your artifacts are big, you don't want to fill the runners' disk. The default expiration is 30 days, meaning all pipelines artifact are available for a month.

Test application job

During the test step, both unit tests and lint runs. Having the lint in the same job is debatable, this is discussed at the end of this section.

Besides the job result, we want unit tests and code coverage reports. The good news is these reports appears in GitLab merge requests. But first, here is a very basic job that we'll iterate on.

variables:
  OUTPUT_PATH: "$CI_PROJECT_DIR/artifacts"

test_app:
  stage: build_and_test
  image: node:12-alpine
  tags:
    - docker
  before_script:
    - apk add chromium
    - export CHROME_BIN=/usr/bin/chromium-browser
  script:
    - yarn ng lint
    - yarn ng test --watch=false

Note the disabled watch (enabled by default). You don't want the runner to be stuck forever waiting for file changes. Karma runs Angular unit tests runs on Chrome browser. It means the docker container needs to have Chromium installed. This is the before_script job but note it's called each time the job runs. You'll learn in the next article how to optimize this installation which takes up to 30 seconds.

With the default karma configuration, unit tests won't work on GitLab for two reasons:

• Runners don't have any monitor, so tests must run on Headless Chrome
• Sandbox mode must be disabled when running Chrome in a docker container

By default Angular tests run in Chrome browser, but you can change this with the browsers option.

Try to run your unit tests with Headless Chrome:
ng test --browsers=ChromeHeadless

Let's create a custom Karma launcher to have Chrome in headless mode but with sandbox mode disabled.

module.exports = function (config) {
  config.set({
    customLaunchers: {
      GitlabHeadlessChrome: {
        base: 'ChromeHeadless',
        flags: ['--no-sandbox'],
      },
    },
  });
}

Once this new custom launcher defined, you can use it through the browsers option: ng test --browsers=GitlabChromeHeadless.

Unit tests report

When running tests, you get tests results in the console. It's possible to generate a complete report that CI tools understand. This report is important to check no tests fails after a merge but also in merge requests.

Default reporters enabled in Karma aren't compatible with GitLab. Only the classic JUnit report works. Let's add this new reporter to the project.

$ npm install --save-dev karma-junit-reporter

module.exports = function (config) {
  config.set({
    plugins: [
      require('karma-junit-reporter')
    ],
    junitReporter: {
      outputDir: 'artifacts/tests',
      outputFile: 'junit-test-results.xml',
      useBrowserName: false,
    },
    reporters: ['progress', 'kjhtml', 'junit'],
  });
}

Running the tests now generates a JUnit report placed in artifacts/tests/junit-test-results.xml. The last step is to let the job know about this location so GitLab can find and analyze the report.

variables:
  OUTPUT_PATH: "$CI_PROJECT_DIR/artifacts"

test_app:
  artifacts:
    name: "tests-and-coverage"
    reports:
      junit:
        - $OUTPUT_PATH/tests/junit-test-results.xml

Code coverage report

Did you know coverage report can be generated while running tests? Use --code-coverage option while running unit tests, it's working out of the box. Angular relies on Istanbul which is able to provide several types of reports.

The truth is you won't get a detailed report integrated in GitLab. Yet, it's possible to have the project and merge requests coverage.

Only cobertura is compatible with GitLab and Istanbul. Let's modify the karma configuration to generate reports.

module.exports = function (config) {
  config.set({
    coverageIstanbulReporter: {
      dir: path.join(__dirname, './artifacts/coverage'),
      reports: ['html', 'lcovonly', 'text-summary', 'cobertura'],
      fixWebpackSourcePaths: true,
      'report-config': {
        'text-summary': {
          file: 'text-summary.txt'
        }
      },
    },
  });
}

Istanbul should already be setup in Karma. Make sure to enable both cobertura and text-summary reporters. The first one is for coverage in merge requests while the second exposes metrics for the whole project.

If you run tests with coverage enabled, you should get the reports in artifacts/coverage directory as defined in karma configuration. Besides cobertura and text-summary reports, you'll also find the html report from the image before.

variables:
  OUTPUT_PATH: "$CI_PROJECT_DIR/artifacts"

test_app:
  coverage: '/Statements\s+:\s\d+.\d+%/'
  artifacts:
    name: "tests-and-coverage"
    reports:
      cobertura:
        - $OUTPUT_PATH/coverage/cobertura-coverage.xml

It works the same as for JUnit report before, the coverage report for merge requests is placed in an artifact. In merge requests you have red and green borders besides the new code to indicate coverage status.

When running the tests with text-summary reporter, the project metrics coverage appears in the console. GitLab looks up in the console and use coverage keyword regex to match the coverage output.

In case the project metrics don't appear, there are saved in  artifacts/coverage/text-summary.txt. You can display them manually by running the cat command in the job script.

If you need more than one coverage metric, you can use coverage-average package which makes an average. For detailed information, you can provide a metrics report exposed in merge requests (premium feature).

Merge request aren't the only place where the project coverage appears. For thoses into project badges, there is a dedicated badge for coverage.

Last words on test job

Did you notice the test job also runs lint? This isn't a separate job for two main reasons:

• A second runner needs to run this new job. Depending on the number and availability of runners it can be important.
• With test and lint jobs in parallel, the pipeline will wait for two jobs to complete even if one failed. It means the longer test job will run for nothing.

This example uses a single job with lint first and then unit test. Yet, this solution has some drawbacks: the test job is ~8 sec longer and it can fail because of the lint without running tests.

variables:
  OUTPUT_PATH: "$CI_PROJECT_DIR/artifacts"

test_app:
  stage: build_and_test
  image: node:12-alpine
  tags:
    - docker
  before_script:
    - apk add chromium
    - export CHROME_BIN=/usr/bin/chromium-browser
  script:
    - yarn ng lint
    - yarn ng test --code-coverage --watch=false --browsers=GitlabHeadlessChrome
  coverage: '/Statements\s+:\s\d+.\d+%/'
  artifacts:
    name: "tests-and-coverage"
    reports:
      junit:
        - $OUTPUT_PATH/tests/junit-test-results.xml
      cobertura:
        - $OUTPUT_PATH/coverage/cobertura-coverage.xml
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
    policy: pull

In the final pipeline, test and build jobs run in parallel. The reason is they only need node_modules to run and don't depend on each other. Also, these two steps take about the same time to complete.

If you don't want to run build and tests on the same stage, make sure the jobs don't download each other artifacts. Use dependencies keyword and set its value to an empty array.

Wrapping up

You now have solid knowledge about GitLab pipelines. Jobs and runners are something you know to work with. At the end of this first article, our Angular app pipeline includes install dependencies, build and tests jobs.

Tests and coverage reports appear in merge requests and you can find artifacts generated by your jobs. The jobs for an Angular library are the same except you may need to specify the library name when using ng command.

For a complete and live example check my angular-app-pipeline sample project on GitLab.

In case you need to validate the format of your pipeline file, check out CI Lint. Continue your reading with the second article to implement the deployment jobs for Angular apps and libraries. Looking forward for your comments.

Thanks for reading!