Load Generation

When a given computer system is idling, i.e., doing nothing, it can typically react to external events very quickly. But, on many systems, this behavior can drastically change when a load is applied. For this reason, measuring the latencies of a real-time platform without any load usually does not give representative results. Having the actual application software running would ideally allow for measuring the real latencies a particular system experiences. However, this software might not always be available, or running it would not be practical for some reason. Also, if the application is doing some specific operation very rarely, catching the longest possible latencies could take a significant amount of time. For these situations, it is usually best to run some artificial loads during the latency measurements.

There are many tools available for this purpose, but stress-ng (version 0.13.0) was selected for this test, as it is one of the most flexible utilities and is already widely used in the industry. It has been specially designed to test various operating system interfaces and to exercise physical subsystems of computer platforms. The original purpose of stress-ng was to find hardware issues such as thermal overruns. Today, the tool has a very wide range of stressors capable of, for example, discovering kernel bugs and executing benchmarking. But most importantly, it can be also used to reveal unexpected latencies from real-time systems.

In total, the stress-ng implements over 220 different stressors which makes it possible to design a comprehensive set of tests. For this study, every category was selected so that they loaded distinct parts of the system but also mimicked realistic use cases best as possible. To get broad information about the system behavior, a total of five different stress categories were selected for further inspection. Once each of the categories was executed, always a total of four processes were started, one for each core. This made sure that all cores had an equal amount of load applied to them.

• idle: system without any background workers performing computation. The system will be able to respond as quickly as possible (stress-ng not invoked).
• cpu: create workers that perform various floating-point arithmetic operations. This will add CPU and cache stress (stress-ng --matrix 4).
• os: create workers that exercise various set*() system calls. This will stress internal kernel operations and data structures (stress-ng --set 4).
• memory: create workers that continuously call mmap() operations and write to the allocated memory. This will stress memory handling (stress-ng --vm 4).
• storage: create workers that repeatedly create and remove directories. This will stress the filesystem and Input/Output (I/O) infrastructure (stress-ng --dir 4).

A simple process was followed to select the most appropriate stressors for each category. First, all available stressors were briefly tested, and the measured latencies were recorded. Then, the results were sorted according to the maximum observed latencies, and from this list, it was possible to pick the most appropriate stressors. The final selection was done so that each one of the categories would represent a typical workload in an embedded real-time system, but in a way that would reveal some interesting results. Therefore, it would have been possible to find stressors that generate even longer latencies with some specially crafted arguments, but as they would not represent realistic scenarios, these options were disregarded.