Getting started with ROS security scanning

It’s a new year, and an especially great time to reflect on the security of your robots. After all, those interested in breaching it are probably doing the same. In previous articles, we discussed ROS security by detailing practical steps for securing your robots on Ubuntu and the challenges the community faces. In this blog post, I’ll give you some strategies, tips and open-source tools you can integrate into your development process today to ramp up the security of your project.

Increase ROS security with SAST scans

Static Application Security Testing (SAST), or static analysis, is a testing method that analyses your source code to find, track and fix security issues that make your application vulnerable before they become a real problem. It is a low cost way to dramatically increase the quality and security of your application, without needing to compile or run it. 

The main advantage of SAST is that it examines all possible execution paths and variable values, not just those invoked during normal execution. This way, it can reveal errors that may not manifest themselves for weeks, months or even years after the release of your application. In other words, you’ll catch these bugs long before someone else does.

You may wonder though, what are the best free SAST tools for your ROS project? And how can you make them an integral part of your development pipeline? We’ll take a deeper look at these questions next.​

Select the best tools for your application

As you might know, there are many tools out there, tailored to different languages and frameworks. Because each one has unique features, strengths and weaknesses, you want to combine different tools.

To get started picking the most effective one(s) for your ROS application, consider these important criteria:

• As a prerequisite: they must support your programming language. Consider open-source tools that support at least Python and C++ in your ROS project.
• Their effectiveness to detect vulnerabilities. For example, based on the Open Web Application Security Project (OWASP) Top Ten, or the Open Source Security Testing Methodology Manual (OSSTMM).
• Their accuracy. Consider the False Positive/False Negative rates they have. False Positives are results that are erroneously identified in the code, while False Negatives are real issues that the tool does not pick up on. The OWASP Benchmark score measures how good these tools are at discovering and properly diagnosing security problems in applications. If the False Positive rate is too high, your team will spend too much time triaging results, while a high False Negative rate might leave you with more potentially critical undetected issues.

• License cost. Some licenses are proprietary and require paying a fee. Some are fully open source, while others are free for open-source projects.
• Integration with CI/CD tools. You want to be able to integrate scans into your development environment in an automated way, to keep track of any issues seamlessly. 
• Interoperability of the output format they support. How will you integrate the output of different tools for analysis? Consider those that support SARIF (Static Analysis Results Interchange Format) output natively, or for which conversion tools are available. This standard makes it straightforward to combine the output of different tools, which usually complement each other.

Which brings us to…

Consider output formats of your ROS security scans

Since you will be using multiple tools concurrently, you might want to think about how you will integrate their results to run a comprehensive analysis. Every tool offers one or more output formats, commonly JSON and XML, and their file structures can vary widely from one tool to the next. This can be an issue when the time comes to remove duplicates, triage and analyze unique results, or generate metrics (for example, how many bugs exist in each program component).

What is the alternative?

The Static Analysis Results Interchange Format (SARIF) format was created to solve this problem. This JSON-based standard makes sure all relevant details for each result are presented in a predictable format. You can upload SARIF logs to DevOps tools, such as Github, to show issues directly in their web UI and convert the tool outputs into issues for tracking. Alternatively, you can visualise them locally in IDEs like Visual Studio Code using a specialised plugin. 

The good news is, many SAST tools now support SARIF, and there are toolkits available to transform results from other formats into SARIF. For example, Coverity Scan (v.2021.03 or later), a well-known tool used with C/C++, supports SARIF format natively. Others like Cppcheck support XML output, and provide a number of templates to reformat it for environments like Visual Studio – but not SARIF. In this case however, you can use the SARIF Multitool SDK to convert Cppcheck output into SARIF format​​.

Other functionality in this SDK is worth exploring too. For example, you can merge multiple SARIF files into one, rewrite from an older SARIF version to the most recent one, extract or suppress particular results, or compare new results to a previous baseline to detect new findings. These will be handy at different stages of your security scans, and can be easily integrated into your CI/CD pipeline. Next, we’ll look at how specific tools can be integrated.

Some tools we have tried for ROS security scanning

Here are some examples of open source tools we have worked with. You can integrate these easily into your CI/CD pipeline, to automatically keep your code secure.

Semgrep

In order to run many checks at once, we are using Semgrep. It is an open-source static analysis engine that can be customised to apply groups of rules, called ‘rulesets’. For example, the `p/gitlab-bandit` ruleset, maintained by GitLab, has very similar coverage to Bandit. Bandit is a widely used open source tool that finds security bugs in Python code:

# scan with hand-picked rulesets and output sarif format natively
semgrep --config p/gitlab-bandit --config p/security-audit --config p/secrets --config p/xss --metrics=off --output semgrep_output.sarif --sarif [<path>] || true

Cppcheck, and integrations for ROS

Cppcheck is an open source analyser for C/C++. These commands run Cppcheck on your codebase, then convert the XML output into SARIF format using the SDK:  

cppcheck [<path>] --xml --xml-version=2 2> cppcheck_output.xml
# convert output to SARIF format using SDK
npx @microsoft/sarif-multitool-linux convert cppcheck_output.xml -t CppCheck -o cppcheck_output.sarif

The Cppcheck CMake integration `ament_cmake_cppcheck` is also available for your ROS 2 project (on Ubuntu 22.04 and later). It will check files with the extensions .c, .cc, .cpp, .cxx, .h, .hh, .hpp, and .hxx. You can run the check from within a CMake ament package as part of the tests, or on the command line, with any CLI options available with Cppcheck:

ament_cppcheck [<path>] [options]

IKOS, and integrations for ROS

IKOS is an analyser for C/C++ based on LLVM and developed at NASA, originally designed to target embedded systems written in C. It allows for a lot of customisation, as you can see in the documentation. Also check out ament_ikos for your ROS 2 projects. This is a Python-based wrapper tool that is used to integrate IKOS into the ament-based build and test system.

These commands build and analyse a whole project, then generate a SARIF report from a package’s output database:

ikos-scan colcon build
ikos-report --format=sarif --report-file=results.sarif <output-file-name>.db

colcon mixins

Another tool for ROS developers to be aware of are colcon mixins. These are command line options that are tedious to write and/or difficult to remember. To make common ones easier to invoke, this repository makes these “shortcuts” available for the command line tool colcon. You can also create your own mixins and contribute them upstream. To apply mixins, pass the option –mixin to the colcon verb, followed by the names of the mixins you want to use. In the example below, we’re invoking AddressSanitizer conveniently via the asan mixin to analyse C code:

colcon build --mixin=asan

As mentioned, SARIF is growing in support, and GCC 13, soon to be released, will also include support for native SARIF output. You will then be able to use a flag like:

-DCMAKE_CXX_FLAGS=-fsanitize=address -fdiagnostics-format=sarif-file

Merge results

Now that you have the output of all your tools in the same format, use the SARIF SDK to integrate all *.sarif output into a handy single file:

npx @microsoft/sarif-multitool-linux merge $CI_PROJECT_DIR/*.sarif --recurse --output-file=integrated_scan_results.sarif

You’re all ready to review your results! 

In a future post, we will look at the process of analysing output by setting up a dashboard, triaging, and identifying the most critical issues in your codebase.

We’d like to hear from you about your ROS security efforts

As ROS Melodic and ROS 2 Foxy approach EOL in 2023, keeping your applications secure is more important than ever. We’d love to hear about how you’re securing your ROS-based software. And if you’re curious about our ROS Extended Security Maintenance offering for ROS 1 and ROS 2 distros, also get in touch with our Robotics team.