Skip to main content

Analysis Engine

The Analysis Engine is a central component of the application.
It coordinates and controls analyses and also contains the implementations of the individual analysis tools.
All associated components of this important functional unit are located in the jobs folder and are organized in subfolders according to their respective functions.

Definitions​

Job:
A job defines a series of analyses to be performed on multiple firmware images as part of a single execution request.

Task:
A task represents the analysis of one individual firmware image using a series of analysis tools and belongs to a job.

Analysis tools:
Analysis tools are software utilities used to analyze firmware, for example to extract file systems or identify security vulnerabilities.

Overview​

The following illustration shows the architecture of the Analysis Engine. Each gray box represents a separate Docker container.
This allows the individual analysis tools and their dependencies to be isolated from each other and enables more efficient parallel processing of the analyses.

Scraping Components

The individual components are discussed in more detail below.

The Executor​

  • Accepts analysis jobs via a Flask API and divides them into individual tasks for each firmware image and the analysis tools to be executed.
  • Controls and coordinates the individual tasks by running several worker processes in parallel.
  • Automatically resolves dependencies between different analysis tools and arranges them in the correct order.
  • Monitors the progress of all tasks during execution and handles any timeouts or errors that may occur.

Redis DB​

  • Communication between the executor and analysis tools takes place via Redis queues; each analysis tool has its own Redis queue for receiving tasks.
  • Notifications about completed analyses are sent back from the individual analysis tools to a global return queue for further processing by the executor.
  • This solution ensures low communication overhead while providing loose coupling of the components.
  • Additionally, the executor uses Redis to track the status and timeouts of tasks.

Analysis Tools​

  • All analysis tools run in isolated Docker containers. An overview of the tools currently implemented is listed in the project documentation.
  • New tools can be easily integrated using predefined templates, which already include Redis communication and multithreading support.
  • In cases where there are dependencies between analysis tools, the executor determines the correct execution sequence and controls it accordingly.