How to set up SonarQube and integrate it with Codemagic

This post is written by Kalgi Shah

What is SonarQube?

SonarQube of sonar source is the leading tool for continuously inspecting the code quality and security of your code base and leading development teams during code reviews. It is an open source tool which has support for 29 programming languages at the time of writing this article, and the number is growing.

The main features of SonarQube include:

  • Code quality checks: SonarQube checks the overall health of your code and, more importantly, highlights code-related issues. This makes it a great tool for checking code quality.
  • Intelligent error detection: SonarQube provides code analyzers and uses powerful path-sensitive dataflow engines that can show errors such as null dereferences, logic errors, and resource leaks.
  • Multilingual support: SonarQube has more than 29 code analyzers for different languages/platforms, such as C/C++, JavaScript, C#, Java, COBOL, PL/SQL, PHP, ABAP, VB.NET, Python, RPG, Flex, Objective-C, Swift , website, and more.
  • DevOps integration: It can be easily integrated with CI/CD tools using webhooks and REST APIs.

SonarQube can run on your local machine or as a Docker container. You can also host it on an on-premise or cloud-based server. SonarQube comes in Community, Developer, and Enterprise editions. The Community Edition is free and open source. On the other hand, although it is not free, the Developer Edition comes with C, C++, Objective-C, Swift, ABAP, T-SQL, and PL/SQL support, branch analysisand pull request decoration.

In this article, we’ll walk you through hosting SonarQube locally and on an AWS EC2 instance, as well as implementing it into a CI/CD pipeline in Codemagic.

Using SonarQube on your local system

You can either download the SonarQube Community/Developer Edition ZIP file here or use theirs Docker image. In this article, we will use SonarQube Developer Edition.

You can check the instructions here to learn how to install the local instance using the ZIP file.

To use their Docker image for testing, start the Docker server using:

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Now, log in to http://localhost:9000 with system administrator credentials (login=admin, password=admin).

Click on the Create a Project a button When asked how you want to create your project, choose by hand.

Enter a Project display name and a Project key and click set up.

Under Provide a tokenenter a symbol name and click Generate. Copy the token and click continue. You will need this when running the analyze CLI command.

Choose your main language of project and follow the instructions.

SonarQube has a dedicated Gradle plugin called SonarScanner for Gradlewhich you can use to generate the SonarQube analysis for your Android project.

SonarQube provides Quick support in the Developer Edition. Unfortunately, Swift is not supported in the Community edition. However, you can use theirs SonarScanner as a CLI tool to generate the SonarQube analysis for your iOS project.

You always can too request a free trial of Developer Edition and try it yourself.

SonarQube Android integration

Integrating SonarQube with an Android project is quite simple. Follow the steps below:

  1. Navigate to your app/build.gradle and open it.

  2. Add the SonarQube Gradle plugin:

plugins {
    id "org.sonarqube" version "3.0"
}
  1. Run project sync if you use Android Studio or just run ./gradlew --refresh-dependencies in the terminal.

  2. Then run the following command from the terminal:

./gradlew sonarqube \
  -Dsonar.projectKey=<project_key> \
  -Dsonar.host.url=http://localhost:9000 \
  -Dsonar.login=<login_token>

iOS integration

Download SonarScanner from here, and add the bin directory to the PATH environment variable. To achieve that, get started Terminal and run the following command:

These commands will open your bash_profile in vi editor Then add the following lines at the end:

export PATH=$PATH:/Applications/SonarScanner/bin
export PATH=$PATH:/Applications/SonarQube/bin

press ESC key and colon will appear at the lower left corner in vi editor. Get in wq save & quit

Use the following command to upload the analysis results:

sonar-scanner \
  -Dsonar.projectKey=<project_key> \
  -Dsonar.sources=. \
  -Dsonar.host.url=http://localhost:9000 \
  -Dsonar.login=<login_token>

You will see the code analysis status displayed on the SonarQube dashboard. To connect Code magic to your local SonarQube host, you will need to make it accessible to the internet. In this case, you can use ngrok. Download the tool and follow the instructions on their website.

Connecting to SonarQube with Codemagic using an AWS Linux EC2 instance

When you need SonarQube to be available to the entire team and plan to integrate it into the CI/CD pipeline, hosting it on the server is the best option. Here, we’ll explore how to deploy SonarQube on an AWS EC2 instance and integrate it with Codemagic to generate code analysis of your Android and iOS projects.

We need to get a Linux EC2 server operating with south privileges before installing sonar server. You can use a machine type of t2 medium or bigger as we need at least 3 GB of RAM run SonarQube effectively. Also add custom TCP security rule for the EC2 instance to allow incoming traffic to the selected SonarQube port (default: 9000).

Step 1: Configure the AWS Linux EC2 instance

Connect to the EC2 instance using a secure shell:

ssh -i <<path to your .pem file>> ec2-user@<<ip address of your EC2>>

Update the system packages in Amazon Linux 2:

Install vim, wgetand curl on Amazon Linux 2:

sudo yum install vim wget curl -y

Increase the vm.max_map_count core, file descriptorand ulimit for the current session at runtime:

sudo sysctl -w vm.max_map_count=262144
sudo sysctl -w fs.file-max=65536
ulimit -n 65536
ulimit -u 4096

If you want to increase these permanently, open the limits.conf configuration file and enter the appropriate values ​​as shown below:

 sudo nano /etc/security/limits.conf
 sonarqube   -   nofile   65536
 sonarqube   -   nproc    4096

Install Java. We need JDK 11 or higher to run SonarQube 7.9, which we use in this blog post.

sudo amazon-linux-extras install java-openjdk11
java -version  // To check java version

You can also install OpenJDK 11 using curl.

Step 2: Install and configure PostgreSQL 13 on Amazon Linux 2 for SonarQube

We need to set up a database for SonarQube to store the report analysis. This also helps to keep the report versions. We will use PostgreSQL as our database, which can be configured on EC2:

First, enable the EPEL repository on Amazon Linux 2 using the command below:

sudo amazon-linux-extras install epel

Add the PostgreSQL 13 repository on Amazon Linux 2:

sudo tee /etc/yum.repos.d/pgdg.repo<<EOF
[pgdg13]
name=PostgreSQL 13 for RHEL/CentOS 7 - x86_64
baseurl=https://download.postgresql.org/pub/repos/yum/13/redhat/rhel-7-x86_64
enabled=1
gpgcheck=0
EOF

Now let’s install and initialize PostgreSQL 13 on Amazon Linux using the command below:

sudo yum install postgresql13 postgresql13-server
sudo /usr/pgsql-13/bin/postgresql-13-setup initdb
sudo systemctl enable --now postgresql-13
sudo systemctl status postgresql-13  //To check PostgreSQL service

Change the password for the default PostgreSQL user:

sudo passwd postgres
su - postgres  //Switch to postgres user

Create a new user by typing:

Switch to the PostgreSQL shell:

Create a user and database for sonar:

ALTER USER sonar WITH ENCRYPTED password 'sonar_password';
CREATE DATABASE sonarqube OWNER sonar;
GRANT ALL PRIVILEGES ON DATABASE sonarqube to sonar;

Exit the PostgreSQL shell:

\q

Switch back to the sudo user by running the exit command:

exit

Step 3: Install SonarQube on Amazon Linux 2

Now that we have all the prerequisites, we will download the binaries for SonarQube and use them to install it.

Note: Please be careful when choosing the edition to install. As mentioned before, we will be using the Development Edition in this article. More details can be found here. If you want to install another edition, right click on the respective download button, and copy the link address.

sudo wget https://binaries.sonarsource.com/CommercialDistribution/sonarqube-developer/sonarqube-developer-9.3.0.51899.zip

Open the SonarQube configuration file and move it to the /opt directory:

sudo unzip sonarqube-developer-9.3.0.51899.zip
sudo mv -v sonarqube-9.3.0.51899 /opt/sonarqube

Step 4: Set up SonarQube on Amazon Linux 2

Running the SonarQube instance as root user causes it to stop working. You can create a new group and user to overcome this problem.

Create the group first. We’ll call it sonar:

Now, add the user with directory access:

sudo useradd -c "user to run SonarQube" -d /opt/sonarqube -g sonar sonar 
sudo chown -R sonar:sonar /opt/sonarqube

Open the SonarQube configuration file with your favorite text editor. We will use nano in this example:

sudo nano /opt/sonarqube/conf/sonar.properties

Find the following lines:

#sonar.jdbc.username=
#sonar.jdbc.password=

Uncomment and type the PostgreSQL database username and password we created in the steps above. Add the PostgreSQL connection string.

sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:postgresql://localhost:5432/sonarqube

In the sonar script file, uncomment RUN_AS_USER and change it to RUN_AS_USER=sonar.

sudo nano /opt/sonarqube/bin/linux-x86-64/sonar.sh

type CTRL+X to save and close the file.

Step 5: Start SonarQube

Now, it’s time to start SonarQube.

Switch to the sonar user:

Move to the script directory:

cd /opt/sonarqube/bin/linux-x86-64/

Run the script to start SonarQube:

Check the running status of SonarQube:

To check the SonarQube logsnavigate to the /opt/sonarqube/logs/sonar.log directory.

Step 6: Set the systemd service for SonarQube

First, stop the SonarQube service, as we started it manually using the steps above. Go to the SonarQube installation path:

cd /opt/sonarqube/bin/linux-x86-64/
./sonar.sh stop

Create a systemd service file for SonarQube to run at system startup:

sudo nano /etc/systemd/system/sonar.service

Add the following lines:

[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=forking

ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start
ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop

User=sonar
Group=sonar
Restart=always

LimitNOFILE=65536
LimitNPROC=4096

[Install]
WantedBy=multi-user.target

Save and close the file.

Now, let’s stop the SonarQube script we started running:

Start the SonarQube daemon by running:

sudo systemctl start sonar

Enable the SonarQube service to start automatically at system startup:

sudo systemctl enable sonar
sudo systemctl status sonar

Step 7: Access the SonarQube UI

If you are already in your instance, you can get the public IP of your Linux EC2 instance using the command below:

curl -s v4.ident.me

Your SonarQube should be active now. You can access the SonarQube UI at http://<<EC2 instance public ip>>:9000/sonarqube. By default, the credentials remain login=admin and password=admin.

The next step is to add SonarQube to our CI/CD environment so that code quality checks can be automatically triggered by certain events.

Step 8: Using SonarQube with Codemagic

We can integrate easily SonarQube with Codemagic using the codemagic.yaml file. Codemagic recently collaborated with Christophe Havard (Product Manager at SonarSource) to add Codemagic to the list of supported CIs for branch and pull request detection. You can check the SonarQube release notes here.

To integrate SonarQube with Codemagic, we will need to configure the SONAR_TOKEN, SONARQUBE_URLand SONAR_PROJECT_KEY environment variables in the Codemagic UI, as shown below. Mark the environment variables as safeand add the respective a group (sonarqube) to the codemagic.yaml file.

Also navigate to your app/build.gradle and add the SonarQube Gradle plugin:

plugins {
    id "org.sonarqube" version "3.0"
}

Let’s define the build pipeline script in the codemagic.yaml file for both the Android and iOS projects. In these examples we use Codemagic Mac Pro’s premium build engine. We want our code quality checks to be triggered on both commits and pull requests, so we specify that in the triggering section You can also reference the sample Android and iOS YAML file settings.

Android project with SonarQube integration

workflows:
  android-workflow:
    name: Android Workflow
    instance_type: mac_pro
    cache:
      cache_paths:
        - ~/.sonar
    environment:
      groups:
        - sonarqube # includes SONAR_TOKEN, SONARQUBE_URL, SONAR_PROJECT_KEY
    triggering:
      events:
        - push
        - pull_request
      branch_patterns:
        - pattern: '*'
          include: true
          source: true
    scripts:     
      - name: Build Android app
        script: ./gradlew assembleDebug
      - name: Generate and upload code analysis report
        script: |
           ./gradlew sonarqube \
           -Dsonar.projectKey=$SONAR_PROJECT_KEY \
           -Dsonar.host.url=$SONARQUBE_URL \
           -Dsonar.login=$SONAR_TOKEN           

Once the build is successful, you can check your code analysis on the SonarQube UI.

iOS project with SonarQube integration

For the iOS build analysis, we first need to download and add the SonarScanner to the road

SonarScanners running in Codemagic can automatically detect branches and merge or pull requests in certain jobs.

workflows:
  ios-workflow:
    name: ios_workflow
    instance_type: mac_pro
    cache:
      cache_paths:
        - ~/.sonar
    environment:
      groups:
        - sonar
      vars:
        XCODE_WORKSPACE: "Sonar.xcodeproj"  # PUT YOUR WORKSPACE NAME HERE
        XCODE_SCHEME: "Sonar" # PUT THE NAME OF YOUR SCHEME HERE
      xcode: latest
      cocoapods: default
    triggering:
      events:
        - push
        - pull_request
      branch_patterns:
        - pattern: '*'
          include: true
          source: true
    scripts:
      - name: Run tests
        script: |
          xcodebuild \
          -project "$XCODE_WORKSPACE" \
          -scheme "$XCODE_SCHEME" \
          -sdk iphonesimulator \
          -destination 'platform=iOS Simulator,name=iPhone 12,OS=15.4' \
          clean build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO          
      - name: Build debug app
        script: |
          xcodebuild build -project "$XCODE_WORKSPACE" \
          -scheme "$XCODE_SCHEME" \
          CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO          
      - name: Sonar
        script: |
            # download and install the SonarScanner
            wget -O $FCI_BUILD_DIR/sonar-scanner.zip https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.4.0.2170-macosx.zip
            unzip $FCI_BUILD_DIR/sonar-scanner.zip
            mv sonar-scanner-* sonar-scanner            
      - name: Coverage tests
        script: |
            xcodebuild \
            -project "$XCODE_WORKSPACE" \
            -scheme "$XCODE_SCHEME" \
            -sdk iphonesimulator \
            -destination 'platform=iOS Simulator,name=iPhone 11 Pro,OS=15.4' \
            -derivedDataPath Build/ \
            -enableCodeCoverage YES \
            clean build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO            
      - name: convert coverage report to sonarqube format
        script: bash xccov-to-sonarqube-generic.sh Build/Logs/Test/*.xcresult/ > sonarqube-generic-coverage.xml
      - name: Generate and upload code analysis report
        script: |
            export PATH=$PATH:$FCI_BUILD_DIR/sonar-scanner/bin
            sonar-scanner \
            -Dsonar.projectKey=$SONAR_PROJECT_KEY \
            -Dsonar.host.url=$SONARQUBE_URL \
            -Dsonar.c.file.suffixes=- \
            -Dsonar.cpp.file.suffixes=- \
            -Dsonar.coverageReportPaths=sonarqube-generic-coverage.xml \            
    artifacts:
      - $HOME/Library/Developer/Xcode/DerivedData/**/Build/**/*.app
      - /tmp/xcodebuild_logs/*.log
      - $HOME/Library/Developer/Xcode/DerivedData/**/Build/**/*.dSYM
    publishing:
      email:
        recipients:
            - kalgi@nevercode.io

Coverage reports cannot be uploaded in this format, so you will need to use the following script to convert it from an .xcresult to an .xml file:

File name: xccov-to-sonarqube-generic.sh

#!/usr/bin/env bash
set -euo pipefail

function convert_file {
  local xccovarchive_file="$1"
  local file_name="$2"
  echo "  <file path=\"$file_name\">"
  local line_coverage_cmd="xcrun xccov view"
  if [[ $@ == *".xcresult"* ]]; then
    line_coverage_cmd="$line_coverage_cmd --archive"
  fi
  line_coverage_cmd="$line_coverage_cmd --file \"$file_name\" \"$xccovarchive_file\""
  eval $line_coverage_cmd | \
    sed -n '
    s/^ *\([0-9][0-9]*\): 0.*$/    <lineToCover lineNumber="" covered="false"\/>/p;
    s/^ *\([0-9][0-9]*\): [1-9].*$/    <lineToCover lineNumber="" covered="true"\/>/p
    '
  echo '  </file>'
}

function xccov_to_generic {
  echo '<coverage version="1">'
  for xccovarchive_file in "$@"; do
    local file_list_cmd="xcrun xccov view"
    if [[ $@ == *".xcresult"* ]]; then
      file_list_cmd="$file_list_cmd --archive"
    fi
    file_list_cmd="$file_list_cmd --file-list \"$xccovarchive_file\""
    eval $file_list_cmd | while read -r file_name; do
      convert_file "$xccovarchive_file" "$file_name"
    done
  done
  echo '</coverage>'
}

xccov_to_generic "$@"

Run this script using:

bash xccov-to-sonarqube-generic.sh Build/Logs/Test/*.xcresult/ > sonarqube-generic-coverage.xml

Pass the result to SonarQube specifying the following properties:

-Dsonar.cfamily.build-wrapper-output.bypass=true \
-Dsonar.coverageReportPaths=sonarqube-generic-coverage.xml \
-Dsonar.c.file.suffixes=- \
-Dsonar.cpp.file.suffixes=- \
-Dsonar.objc.file.suffixes=-

And that’s it! We have successfully integrated SonarQube with Codemagic to work for Android and iOS projects. Before we wrap up, let’s discuss two small details you may have noticed in the YAML settings above that have to do with autotriggering and caching.

Automatically detecting pull requests

In order for SonarQube to automatically detect pull requests when you use Codemagic, you need to add an event in the trigger section of your codemagic.yaml file, as shown in the following snippet:

    triggering:
      events:
        - pull_request

You may have noticed this in the YAML scripts above. But that’s not all: For triggering to work, you must also set a web hook between Codemagic and your repository (eg, Bitbucket, GitHub).

Cache of the .sonar folder

Cache the .sonar directory will save build time in later analyses. To do this, add the following snippet to yours codemagic.yaml file:

    cache:
      cache_paths:
        - ~/.sonar

Conclusion

Integrating SonarQube with Codemagic is really simple when you use the codemagic.yaml file. In this post, we covered the basic configuration required to generate the code analysis report, but there are various other properties that you can specify using SonarQube, especially with SonarQube Developer Edition. However, even the free tier can be enough to significantly improve the code quality of your project by automating reporting with SonarQube and Codemagic.

Try it! And if you have any questions or suggestions, we’re always happy to hear them our Slack or on Twitter (just tag @codemagio).

Learn more

Leave a Comment