Monitoring a Process Vitals on Linux (CPU/IO/RAM)

When developing software for installations that need to remain up for very long time without being restarted, it is pretty relevant to monitor it to verify it is working properly. I typically use tools like gdb and Valgrind, but in some cases I found these insufficient. Valgrind, for example, proved impossible to use in some cases, as it slows down the application to the point I couldn’t even start it. With video processing applications or apps on cheap embedded hardware, I found it difficult to use it sometimes. In those cases, I found it useful to monitor the process in a less invasive way.

Available Applications

This question lists many applications available. Some techniques are pretty basic, some others use more involved applications with more features. This answer shows many available advanced options.

Procweb

I used psrecord sometimes, and I find it excellent. The chart is very useful. But there are some limitations. I therefore wrote an alternative for myself using technologies that I wanted to practice anyway: here is the public repo.

I wanted the app to be a single independent binary, available for, at least, Linux x64, arm64 and armhf, providing an embedded web interface, without any need for space to write data persistently. In addition, I wanted the binary to be built by a CI system based on GitLab.

Solution

At the beginning I wrote the app in Qt, to experiment with the new HTTP server module. The result worked well:

https://github.com/carlonluca/procweb/tree/master/procweb-qt

Then I reimplemented it in Rust, which I rarely use:

https://github.com/carlonluca/procweb/tree/master/procweb-rust

The web interface is implemented in Angular, and embedded into the binaries:

https://github.com/carlonluca/procweb/tree/master/procweb-webui

Packages

For the requirement “single independent binary” I experimented a bit with the AppImage format. This is the solution I implemented for the Qt binary. You can find the configuration files in the repo.

For Rust, instead, I preferred another option: a completely static binary based on musl. This seems to work as well so far. Runs everywhere, without dependencies.

Building

The application can be built directly by first building the Angular frontend:

cd procweb-webui
./build.sh

then the Qt binary:

cd procweb-qt
mkdir build
cd build
cmake ..
make

you’ll need to have the dependencies in place for it to build. Best way is to use the docker image I uploaded here.

For the Rust binary instead, use Cargo:

cargo build --release --target=x86_64-unknown-linux-musl

By default, the server listens on port 3000.

CI

The other requirement was that the package should be built by a CI system based on GitLab. This is possible for both the AppImage and the static Rust binary.

For the Qt version, I used my Docker images here. With those Docker images, I can build my AppImage with Qt >= 6.4 for both x64 and arm64. In this case, I failed the armhf requirement, as I do not maintain a armhf image at the moment. I added support for appimage-builder to those images.

For the Rust version, I created another specific Docker image:

https://github.com/carlonluca/procweb/blob/master/.gitlab-ci.yml

Installation

I have not uploaded an AppImage for the Qt version yet. You should build it yourself first. Then you can use the AppImage without any specific installation procedure.

Procweb-rust is on crate.io, so you can simply:

cargo install procweb-rust

run it:

procweb-rust 1234

then go with a browser to:

http://<ip-addr>:3000

In alternative, I’ll upload some statically built binaries for x64, arm64 and armv7 here: https://github.com/carlonluca/procweb/tags.

Have fun! 😉