Integrations Monitoring

To centralize all security issues that may arise in the different phases of the pipeline, we will integrate with our monitoring tools.

DefectDojo

To integrate DefectDojo,the first thing we have to do is to create a project. Once we have created the project, we will need two components in our pipeline. The first will consist of creating a stage that precedes all the sections and that will create an engagement for that execution. Then, we will develop a reusable function to import the different analyses.

In both cases, we will use the DefectDojo API, which requires authentication. This information is located at the top right of the DefectDojo interface.

Once there, we will be able to see the API Key, which must be included in our requests in the header under the heading Authorization: Token XXXXXXXXXXXXXXXXXXXXXX

The base url of the v2 API simply consists of the base URL of your DefectDojo instance + '/api/v2' (e.g. https://defectdojo.example.com/api/v2 ).

Your product ID can be obtained from the URL when you access the product in the user interface.

Engagement

Now that we have the data, we will create our engagement. To do this, we will create a stage called '.pre', which will return an ID that we will store in a new environment variable. Then, we will pass this ID to the following DefectDojo related tasks using the artifact 'artifacts:reports:dotenv'.

The first thing to do is to create the IC variables, which are as follows:

• DD_SERVER: The address of the DefectDojo server.
• DD_API_KEY: The API key to authenticate to DefectDojo. Be sure to check the 'mask' attribute so that it is not displayed in the logs.
• DD_PRODUCTID: The ID of the product on which we are going to create the engagement.

We are going to create a file called defectdojo-engagement.sh in the folder .gitlab-ci, where we will include the call curl that will create the engagement for this CI run.

.gitlab-ci/defectdojo-engagement.sh

#!/bin/bash

TODAY=$(date +%Y-%m-%d)
ENDDAY=$(date -d "+${DD_ENGAGEMENT_PERIOD} days" +%Y-%m-%d)

ENGAGEMENTID=$(curl --fail --location --request POST "${DD_SERVER}/api/v2/engagements/" \
  -H "Authorization: Token ${DD_API_KEY}" \
  -H 'Content-Type: application/json' \
  --data-raw "{
    \"tags\": [\"GITLAB-CI\"],
    \"name\": \"#${CI_PIPELINE_ID}\",
    \"description\": \"${CI_COMMIT_DESCRIPTION}\",
    \"version\": \"${CI_COMMIT_REF_NAME}\",
    \"first_contacted\": \"${TODAY}\",
    \"target_start\": \"${TODAY}\",
    \"target_end\": \"${ENDDAY}\",
    \"reason\": \"string\",
    \"tracker\": \"${CI_PROJECT_URL}/-/issues\",
    \"threat_model\": \"${DD_ENGAGEMENT_THREAT_MODEL}\",
    \"api_test\": \"${DD_ENGAGEMENT_API_TEST}\",
    \"pen_test\": \"${DD_ENGAGEMENT_PEN_TEST}\",
    \"check_list\": \"${DD_ENGAGEMENT_CHECK_LIST}\",
    \"status\": \"${DD_ENGAGEMENT_STATUS}\",
    \"engagement_type\": \"CI/CD\",
    \"build_id\": \"${CI_PIPELINE_ID}\",
    \"commit_hash\": \"${CI_COMMIT_SHORT_SHA}\",
    \"branch_tag\": \"${CI_COMMIT_REF_NAME}\",
    \"deduplication_on_engagement\": \"${DD_ENGAGEMENT_DEDUPLICATION_ON_ENGAGEMENT}\",
    \"product\": \"${DD_PRODUCTID}\",
    \"source_code_management_uri\": \"${CI_PROJECT_URL}\",
    \"build_server\": ${DD_ENGAGEMENT_BUILD_SERVER},
    \"source_code_management_server\": ${DD_ENGAGEMENT_SOURCE_CODE_MANAGEMENT_SERVER},
    \"orchestration_engine\": ${DD_ENGAGEMENT_ORCHESTRATION_ENGINE}
  }" | jq -r '.id')

echo "DD_ENGAGEMENTID=${ENGAGEMENTID}" >> defectdojo.env

From our .pre sgate we are going to launch the creation of the engagement calling the previous script.

.gitlab-ci.yml

defectdojo_create_engagement:
  stage: .pre
  image: alpine
  variables:
    DD_NOT_ON_MASTER: "false"
    DD_ENGAGEMENT_PERIOD: 7
    DD_ENGAGEMENT_STATUS: "Not Started"
    DD_ENGAGEMENT_BUILD_SERVER: "null"
    DD_ENGAGEMENT_SOURCE_CODE_MANAGEMENT_SERVER: "null"
    DD_ENGAGEMENT_ORCHESTRATION_ENGINE: "null"
    DD_ENGAGEMENT_DEDUPLICATION_ON_ENGAGEMENT: "false"
    DD_ENGAGEMENT_THREAT_MODEL: "true"
    DD_ENGAGEMENT_API_TEST: "true"
    DD_ENGAGEMENT_PEN_TEST: "true"
    DD_ENGAGEMENT_CHECK_LIST: "true"
  rules:
    - if: '$DD_NOT_ON_MASTER == "true" && $CI_COMMIT_BRANCH == "master"'
      when: never
    - when: always
  before_script:
    - apk add curl jq coreutils
  script:
    - sh .gitlab-ci/defectdojo-engagement.sh 
  artifacts:
    reports:
      dotenv: defectdojo.env

This stage in the GitLab CI/CD configuration file is called "defectdojo_create_engagement".

• stage: .pre: This stage is placed in the stage .pre of the pipeline, which means that it will be executed before other stages in the pipeline.
• image: alpine: Use the Alpine Linux image to run this stage.
• variables: Defines a set of environment variables that will be used in this stage to configure the engagement in DefectDojo. For example, the duration of the engagement (DD_ENGAGEMENT_PERIOD) and various indicators of which parts of the engagement will be created (DD_ENGAGEMENT_THREAT_MODEL, DD_ENGAGEMENT_API_TEST, DD_ENGAGEMENT_PEN_TEST, DD_ENGAGEMENT_CHECK_LIST) are set.
• rules: Establishes rules that determine when this stage will be executed. This rule says that if $DD_NOT_ON_MASTER is equal to "true" and $CI_COMMIT_BRANCH is equal to "master", then the stage will never be executed (when: never). Otherwise, it will always be executed (when: always).
• before_script: Here you define the commands that will be executed before the main script. In this case, the tools curl, jq, and coreutils are installed using apk add.
• script: The script to be executed at this stage is defectdojo-engagement.sh, which is located in the folder .gitlab-ci. This script uses the variables defined above to create an engagement in DefectDojo using curl.
• artifacts: Here it is specified that this stage will generate a report in the form of a file .env called defectdojo.env, which will be used in other stages of the pipeline.

With these steps, we have created our engagement for this run, which will be used in the following stages.

Findings

To send the results of the different tools, we are going to use another file in our folder .gitlab-ci called defectdojo-finding.sh, which will send the results to our DefectDojo and the corresponding engagement.

.gitlab-ci/defectdojo-finding.sh

#!/bin/bash

TODAY=$(date +%Y-%m-%d)

curl --location --request POST "${DD_SERVER}/api/v2/import-scan/" \
    -H 'accept: application/json' \
    -H "Authorization: Token ${DD_API_KEY}" \
    -F "scan_date=${TODAY}" \
    -F "minimum_severity=${DD_SCAN_MINIMUM_SEVERITY}" \
    -F "active=${DD_SCAN_ACTIVE}" \
    -F "verified=${DD_SCAN_VERIFIED}" \
    -F "scan_type=${DD_SCAN_TYPE}" \
    -F "engagement=${DD_ENGAGEMENTID}" \
    -F "file=@gitleaks-report.json;type=application/json" \
    -F "close_old_findings=${DD_SCAN_CLOSE_OLD_FINDINGS}" \
    -F "environment=${DD_SCAN_ENVIRONMENT}" \
    -F "product_name=${DD_PRODUCT_NAME}"  \
    -F 'create_finding_groups_for_all_findings=true' \
    -F 'group_by=component_name+component_version'

Dependency-track

Para integrar Dependency-track, el primer paso es crear un proyecto.

To perform this integration, we will use the Dependency-Track API, which requires authentication. The necessary information can be found at Administration > Acess Management > Teams. Here we can use the equipment from Automation or create our own. By clicking on it, we will get an API KEY and the minimum permissions required for integration.

We also need the project ID, which we get from the URL when accessing the project we want to integrate.

Now that we have the data, the first thing to do is to create the IC variables, which are as follows:

• DT_SERVER: The address of the Dependency-Track server.
• DT_API_KEY: The API key to authenticate to Dependency-Track. Be sure to check the 'mask' attribute so that it is not displayed in the logs.
• DT_PROJECT_ID: The ID of the project we are going to integrate.

To send the results to Dependency-Track, we will use another file in our .gitlab-ci folder called dependency-track.sh, which will send the results to our Dependency-Track and the corresponding project.

.gitlab-ci/dependency-track.sh

#!/bin/bash

curl --location --request "POST" "${DT_SERVER}/api/v1/bom" \
        -H "X-Api-Key: $DT_API_KEY" \
        -F "project=$DT_PROJECT_ID" \
        -F "projectVersion=$CI_COMMIT_BRANCH" \
        -F "bom=@sbom.json" -f