Container Scanning Nuances

In the world of DevOps and agile pushing fast deliveries, atomic containers which are services and packaged code/software that runs in orchestrated environments are highly adopted. As containers have a web of open source libraries, third-party software and packages, finding third-party vulnerabilities and fixing them becomes our responsibility in publishing consumable software and artifacts to consumers. Container scanning is all about how well we detect third party software ( library, packages, OS) vulnerabilities.

Types of scanning

1. Container scanning in ci cd

2. registry direct scanning

3. dynamic scanning via agent provided by CSPMs

evaluating container scanning tools

This applies to all types of scanning.

open source players

1. Trivy

2. Grype

3. Clair

4. Dockerscout

5. DepScan

6. ScanCode

For the scope of comparison, I have taken the two famous tools, Trivy and Grype

Trivy is one of the early container scanning tools that was open sourced in 2019. Whenever a container scanning tool comes to mind, it’s Trivy. Let’s analyse all Trivy and Grype tools

Sources

If you think of sources in anatomy, it’s the blood, without it, no organ is going to function. it covers advisories, managing the vulnerabilities from the advisories and the inclusion of custom sources ( in scaling criteria).

Trivy’s sources

1. PHP Security Advisories

2. Github Advisory

3. Open Source vulnerabilities (PyPI)

4. Ruby Advisory Database

5. Ecosystem Security Working Group

6. Gitlab Advisory

For Grype, it covers

1. Alpine advisory

2. Amazon advisory

3. Chainguard advisory

4. Debian advisory

5. Github advisory

6. Mariner advisory

7. Nvd advisory

8. Oracle advisory

9. RHEL advisory

10. SLES advisory

11. Ubuntu advisory

12. Wolfi advisory

Grype has vunnel and Grype db for managing the sources. Trivy has Trivy db for managing the Trivy db. Trivy db is not mature like Grype db. The main use of Trivy db is to build trivy sources by yourself for an air gap environment. Grype DB provides tools to add, manage and build sources explicitly,

Coverage

Trivy

1. Ruby ( Gems)

2. python(egg, wheel, conda)

3. PHP ( composer )

4. Nodejs ( package.json )

5. .NET ( packages config

6. java ( Jar/war/par/ear)

7. GO ( binaries)

8. Rust ( cargo )

Grype

1. Ruby (Gems)

2. Java (JAR, WAR, EAR, JPI, HPI)

3. JavaScript (NPM, Yarn)

4. Python (Egg, Wheel, Poetry, requirements.txt/setup.py files)

5. Dotnet (deps.json)

6. Golang (go.mod)

7. PHP (Composer)

8. Rust (Cargo)

Vulnerability Matching techniques

Vulnerability Matching depends on sources and advisories. NVD advisory will have CPE that will have the package name and version to match against. Github advisory will match based on the ecosystem. Advisories are moving towards purl based matching cause CPE matching leads to false positives and NVD software naming issues.

Trivy will use certain advisories for certain programming languages which are listed in the below image ( taken from Trivy Site)

Grype does not have individual advisories matching for language ecosystems, but it does for vendor based matching, for example, if the library is from Redhat, it will take from Redhat rather than GIthub advisory. It decides on the priority based on advisory source quality (Grype prefers Github advisory over NVD) and vendor advisories for correct severity.

SBOM generation

Trivy and Syft ( from Grype ) generate the same SBOM results. The current SBOM generators were not able to include SBOMs of binaries inside the container. There are packaged containers where you will be running a service build from go/java. Instead of scanning the container and binary separately, Grype and Trivy support binary detection and scanning for third party vulnerabilities which can ease us in simplifying our scans.

But Trivy does not report go, java standard library issues where Grype finds it.

We should not consider that these tools depend on the SBOM and find vulnerabilities. SBOM generators are different from vulnerability scanning tools. For example, Syft SBOM generators have support for 24 ecosystems whereas Grype does not have support for C package managers. SBOM based scanning supported by tools is different from regular scanning the tool does.

SBOM VEX support

Grype has both SBOM and VEX support. Trivy has SBOM only. VEX is in the early adoption stages. VEX has its values when all customers can accept the VEX but the problem is customers still ask to fix all vulnerabilities no matter whether it’s relevant to the source code which makes VEX hard to adopt.

CPE matching

Grype uses Github Advisories in most of the cases where there is no CPE based matching and it reduces false positives since the matching is based on the ecosystem. CPE matching does not have an ecosystem where two libraries with the same name in different ecosystems could be matched which leads to false positives.
OWASP has a dedicated page covering NVD recommendations where they discuss CPE Gaps and purl usage.

Scanning image with binaries inside

Currently, One of the SBOM generators uses Go reverse engineering techniques to find deps. A similar technique is used in Trivy and Grype in scanning those binaries. However, we are not sure it would find 100% of the dependencies. Where you have the source code, it’s always better to scan the binary to its source code separately. 

I am not concluding that Trivy is better than Grype or vice versa. This is just an example of a comparison I gave. Both tools are great and it varies depending on your environment.