How To Automate Security Analysis with the RIPS API23 Aug 2017 by Hendrik Buchwald
RIPS exposes a powerful REST-API, an interface specifically designed for developers and their applications. It is used to provide the web interface with analysis results, to start scans through plugins, to manage users, and much more. In short, the API enables easy automation of all RIPS features through other programs.
In this blog post the architecture of the RIPS API is explained and the advantages of a RESTful API are shown. To demonstrate the simplistic nature of our interface, a small CI integration example is given that will reject code commits with security issues and hence protects the production server from new vulnerabilities.
This post uses the RIPS API version 2.7.1. Newer versions may contain more functionality.
For a convenient API usage it is important to understand what input is expected and what output is returned. There are multiple API specification standards available that allow developers to explicitly define an API in a machine-readable format. This definition can be used to generate documentations, libraries, tests, and other useful components.
RIPS uses OpenAPI 2.0 to define its API 1, a common standard that is maintained by the Open API Initiative 2. The specification is written in YAML, so it is human-readable as well. Instead of looking at the raw content though it is advised to use a client like Swagger to view the specification. Figure 1 shows the specification for our API call to initiate a new scan. The values can be modified on-the-fly in our Swagger interface on https://api.ripstech.com/ for testing. Please note that the specification defines all input and output but not all conditions, for example it does not consider user permissions. It is up to the user to look this information up.
RIPS has a RESTful API, but what does that mean? REST is the acronym for Representational State Transfer, an architecture designed for interoperability between computer systems on the Internet 3. The API is based on the Hypertext Transfer Protocol (HTTP) and provides stateless operations to get and manipulate data in a machine-readable format from RIPS.
There are many reasons why REST is a good choice 4:
- Easy to integrate in any system since HTTP libraries and tools are available everywhere
- Easy to scale and route through load balancers, reverse proxies, caches, …
- Relatively easy to maintain in the long term since
- changes are backwards compatible
- it has a clean and easy to understand interface
- Small attack surface
- Well suited for web applications and existing web technologies
In this API, the following HTTP methods are used to specify the type of action that should be performed.
|GET||Get one or more objects||JSON-encoded object(s)|
|DELETE||Delete one or more objects||empty|
|POST||Create (usually) one object||JSON-encoded object(s)|
|PATCH||Update one object||JSON-encoded object(s)|
Uniform Resource Identifiers (URI) are used to specify resources. Each resource has a tree-like structure, e.g., an application resource has scans, scans have issues, issues have comments, etc. An URI can refer to a single object or to a collection of objects. With the help of the previously introduced methods a get, delete, create, or update of resources can be initiated. The following table provides some URI examples for the RIPS API.
|All scans of single application||/applications/1/scans|
|Single scan of a single application||/applications/1/scans/7|
|All issues of a single scan||/applications/1/scans/7/issues|
|Single issue of a single scan||/applications/1/scans/7/issues/19|
HTTP status codes are used to communicate the success or failure of a request. In general all codes that are defined by the HTTP/1.1 standard can occur but there are 5 codes that occur predominantly.
|200||Everything worked as expected|
|400||Invalid user input|
|401||Invalid or missing credentials|
|404||Item not found|
Additionally to the HTTP code a JSON object is generated and returned. Next to the status code it contains a more detailed human-readable error message for easy debugging. If the status code is not
200, the request should be considered failed and throwing an exception is recommended.
The authentication system of the RIPS API 2.7 expects a username and a password in the custom HTTP headers
X-API-Password for every request. If no credentials are given or if the given credentials are not correct a
401 error is thrown.
If there are no errors the requested resource is returned in JSON format. For a GET request this can either be an object or a collection of objects. Both POST and PATCH requests return a single object that was created or modified by the request, representing its state after the action.
For convenience and to save API requests most objects contain sub objects. Sub objects are included or excluded based on their depth. For example, if you directly read out an application it also contains the user object of its creator. If you directly read out a scan it contains the application object but this application object does not contain the user object anymore.
Input always consists of a JSON object containing one or more additional objects. For a list of all possible parameters please refer to our detailed API specification.
For example, to create or update an application the following JSON message can be used:
To create a new scan the following JSON message can be used. Besides the main object
scan it also contains the object
php that defines additional settings for the scan, in this case the PHP environment that should be simulated by the RIPS engine.
Most endpoints that work with collections allow to filter the results based on certain properties. This is a powerful tool to save bandwidth and time by avoiding manual filter processes on the full result set.
Different filters can be combined but each specific filter has to be unique. All filters have to be fulfilled, so technically speaking the conditions are AND-connected.
We now have a good understanding of the basic concepts of the rich RIPS API. With a proper HTTP library it is a matter of minutes to talk to the API and to start controlling RIPS through own programs. In the following practical example we will write a simple CI plugin in Python that can be used to automatically check a code base for security issues with RIPS.
We want to implement the following basic functionality:
- Create a zip archive that contains the source code
- Upload the archive to RIPS (SaaS or On-Premises)
- Start a new scan
- Wait for the scan to finish
- Search for unreviewed issues
Unless specified otherwise, RIPS will automatically compare the analysis results of a new scan to its previous scan. Review flags as well as comments are automatically inherited. Hence if we review all reported security issues in our first analysis, perform a rescan of our code and there are unreviewed issues reported, we know that there are new issues we have not reviewed before.
We start by creating a class
API for our API communication. This class is merely a wrapper around an HTTP library. As recommended previously, our class raises an exception if a request is not successful. Also, the JSON format is used for everything but the file upload. The following class is sufficient to perform
post actions of the RIPS API with credentials and to
upload file archives.
Now we can utilize the methods of our API class to fulfill the tasks. For this example we are using the SaaS version of RIPS but by modifying the base URL in the constructor (line 17) the code can also be used in the On-Premises version.
Our script expects the four parameters
path. The credentials are used for the RIPS API connection. The
app_id is the ID of your existing RIPS application. The script will then pack the code in your local file
path, upload it to RIPS and initiate the rescan. It then checks the scan status every 5 seconds and waits for completion. Finally, it checks if unreviewed security issues were found and warns the user.
Running the script may look like this.
In this example, the second directory
dvwa_x contains new security issues. Since these do not have any review flags, our script exits with status code 1 and informs the user. At this point the developer can either review the new issues with a flag in the RIPS interface or fix the code according to RIPS patch instructions.
In this blog post we provided a simplistic example script that demonstrates the usage of the RIPS API. Although we provide ready-to-use plugins for many popular systems, our RESTful API is a powerful tool for developers. It enables to fully automate nearly any task and thus to seamlessly integrate the process of automated code analysis in highly specialized development environments. The code sample given in this blog post can be used as an initial prototype and can be extended to perform more complex tasks.