General Discussions

Testing your code is crucial. Having it tested automatically and reported in a concise and understandable way is a must. There are multiple aspects of testing:

- Unit tests + code coverage

- Integration tests

- Static analysis

- Dynamic analysis

In this post, we'll show you how to setup testing and analysing your code with SonarQube Cloud and Unity. It will cover:

• Unit tests setup with Unity

• Code coverage report gcovr

• SonarQube Cloud report

• CMake integration

The Part 2 will cover the integration into a GitLab CI pipeline to run the analysis and the tests on each push.

Project architecture

The project files are available as a zip:

. They are organized as follows:

.
├── CMakeLists.txt
├── main.c
├── src
│   ├── app.c
│   ├── app.h
│   ├── CMakeLists.txt
│   └── my_math
│       ├── CMakeLists.txt
│       ├── my_math.c
│       └── my_math.h
├── test
│   ├── CMakeLists.txt
│   ├── my_math_test.c
│   └── test_main.c

• The sources are located in src/ folder

• The tests are in the test/ folder

• The project is configured and built using CMake. The top CMakeLists.txt is the main list file.

Building the project

cmake -B build # without tests
cmake -B build --DBUILD_TYPE=testing # with tests 
cmake --build build 

This will build the sources and our test. The -DBUILD_TYPE=testing flag is here to add the coverage option to GCC as well as adding the test/ folder to the configuration/build.

Unit tests using Unity

As stated before, we use Unity framework to write our tests. Its advantages are:

- Easy C/C++ integration

- Simple .c and .h library

- Tests are easy to write

- Concise reports

- Can be integrated with Ceedling

Running the tests

Once the sources and tests are compiled, one can run the tests using ctest:

ctest --test-dir build
Internal ctest changing into directory: /home/dtruan/posts/app/build
Test project /home/dtruan/posts/app/build
    Start 1: test_math
1/2 Test #1: test_math ........................   Passed    0.00 sec
    Start 2: test_main
2/2 Test #2: test_main ........................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 2

Total Test time (real) =   0.01 sec

The tests run and you get a report on stdout. We'll later use a specific format to be compatible with junit report, used by GitLab CI.

Running the code coverage

The code coverage will be reported using gcovr:

gcovr -e main.c -e build 
------------------------------------------------------------------------------
                          GCC Code Coverage Report
Directory: .
------------------------------------------------------------------------------
File                                       Lines    Exec  Cover   Missing
------------------------------------------------------------------------------
src/app.c                                      6       6   100%   
src/my_math/my_math.c                          4       4   100%   
test/test_main.c                              11      11   100%   
test/test_my_math.c                           20      20   100%   
------------------------------------------------------------------------------
TOTAL                                         41      41   100%
------------------------------------------------------------------------------

This will run the code coverage to check if our tests cover enough lines of code. We can tell it to generate the report in sonarqube format and to save it in coverage.xml:

gcovr -e main.c -e build --sonarqube > coverage.xml

SonarQube Cloud

SonarQube is a powerful static analysis and code report tool. It covers a variety of languages and offers multiple pricing plans in order to just test the tool or to support a full scale enterprise setup.

In this article, we are using the free plan and their public Cloud server. It is assumed that you already have a SonarQube account and an organization already created. You must also have your SONAR_TOKEN saved, as it will be used when running the analysis

How SonarQube works

SonarQube works in two steps:

- First it collects the build environment using a build wrapper. It generates some output files which will be used by the next step.

- Then it uses the sonar-scanner executable to analyse the code. It uses a configuration files to define the sources to analyse, the account/server information, ... It generates a SonarQube compatible report after a successful run.

The configuration file for this project is like this:

sonar.projectKey=<your_project_key>
sonar.organization=<your_project_org>
sonar.sources=src
sonar.cfamily.compile-commands=bw-output/compile_commands.json
sonar.host.url=https://sonarcloud.io
sonar.sourceEncoding=UTF-8
sonar.coverageReportPaths=coverage.xml

It specifies:

• The project key (from SonarQube Cloud)

• The organization this project is in

• Where our sources are located

• Where to look at the build command, which is the build-wrapper output

• The sonar server URL, here we use the public server

• The sources encoding

• The coverage report file path (optional)

Run an analysis

Here are the steps, from a fresh state, without any build done before:

cmake -B build -DBUILD_TYPE=testing
build-wrapper-linux-x86-64 --out-dir bw-output cmake --build build
ctest --test-dir build
gcovr -e main.c --sonarqube > coverage.xml 
SONAR_TOKEN=<YOUR_TOKEN> sonar-scanner

It will build the project, run the tests and code coverage, feed them to the analysis and send the report to the SonarQube Cloud server, which can then be consulted using the Web UI:

We can see the report on our main branch. Here everything passes:

• No Security issue was found

• Our tests cover enough of the code
  

Next time, we'll see how to integrate all of this into the GitLab CI to automatically run the analysis!