Formatting Generic Text in the Terminal

When developing products and managing servers I frequently notice that properly formatting and colorizing text, logs and generic output in the terminal is very useful. Errors, warnings and messages are much clearer and don’t get unnoticed.

grc

Some time ago I started to use this excellent tool: https://github.com/garabik/grc. It uses regexes to capture and format strings. It is very useful when formatting logs or command output. I frequently use it to colorize logs printed by my applications during investigations. In some cases though, logs may be very frequent, like when processing signals or video streams. In those cases this tool was someone slow.

cgrc

I therefore took a few hours to re-write it in C++, but still keeping the concepts of grc unaltered. Qt includes a great regex engine, so I decided to use the Qt core module. I implemented a subset of the features of grc, but I also tried to keep it somehow compatible.

I typically use logcat-colorize to format the Android’s logcat output, but in some cases I used grc with a specific conf file. I then included it into cgrc as well, and this is the result of formatting a 25MB dump of logcat using grc:

cat logcat.data 0.01s user 0.21s system 0% cpu 43.652 total
grcat conf.logcat 38.42s user 4.99s system 99% cpu 43.791 total

This is instead the time taken by cgrc:

cat logcat.data 0.00s user 0.14s system 0% cpu 15.411 total
cgrc logcat 10.41s user 4.91s system 99% cpu 15.458 total

These tests were run on a aarch64 Raspberry Pi system using the snap package.

This is still slower than the specialized logcat-colorize binary, but sufficient.

Adding Configuration Files to cgrc

Instead of having configuration files installed in the system, I included a set of configuration files into the binary itself, by using the Qt resource system. For custom files, I added one user-specific location and one system-wide location, exactly like grc. I tend to forget where to place custom files, so I added a couple of ways to ask cgrc where to place it, instead of looking it up in the readme everytime:

$ cgrc --list-locations
Locations on your system used by cgrc:
       System location: /etc/cgrc
       User location  : /home/luca/.config/cgrc

This is particularly useful when using it through snap packages:

$ cgrc --list-locations
Locations on your system used by cgrc:
       System location: /var/snap/cgrc/84
       User location  : /home/luca/snap/cgrc/84

This is for Mac OS instead:

$ cgrc --list-locations
Locations on your system used by cgrc:
System location: /etc/cgrc
User location : /Users/luca/Library/Preferences/cgrc

I also added a way to list the configurations available:

cgrc --list-configurations
Embedded configurations:
        :/conf/dockerps -> Formats the output of docker ps.
        :/conf/logcat -> Parser the Android logcat output
        :/conf/nginx -> Formats the default nginx log output
        :/conf/ping -> Formats the output of the ping Linux command
        :/conf/prio -> Formats the output of logs containing typical words associated to priorities

User configurations:
        /home/luca/snap/cgrc/84/unittest -> Configuration to format C++ unit tests

System configurations:

Colorizing Prioritized Logging

If you are like me, you add priorities to your logs. I also worked on software from companies doing exactly the same. When this is implemented, logs are frequently not colored, but simply include the text “Debug”, “Info”, “Verbose”, “Warning”, “Error” or something similar. So I wrote my configuration that typically adapts to these cases well:

desc=Formats the output of logs containing typical words associated to priorities
# Error
regexp=.*(\s+|^)(?i)(Error|Critical|Erro|Err|Fatal)(\s+|$).*
colours=red
count=stop
# Warning
regexp=.*(\s+|^)(?i)(Alarm|Warn|Warning)(\s+|$).*
colours=yellow
count=stop
# Message
regexp=.*(\s+|^)(?i)(Info|Message)(\s+|$).*
colours=green
count=stop
# Debug
regexp=.*(\s+|^)(?i)(Debug|Dbg)(\s+|$).*
colours=blue
count=stop
# Verbose
regexp=.*(\s+|^)(?i)(Trace|Verb|Verbose)(\s+|$).*
colours=white
count=stop

With cgrc (or grc) you can use this to colorize output from your process:

./app | cgrc prio

or from log files:

cat logfile.txt | cgrc prio

or even (only if you are not using the snap version, which has classic confinement):

cgrc prio --input-path logfile.txt

Installing

The only options I implemented for installation are the snap package on Linux (this one uses Qt5):

sudo snap install cgrc --edge

or AUR pkgbuild for Arch/Manjaro:

sudo pamac install cgrc

For Mac OS, you can use Macports:

sudo port install cgrc

Building

You can quickly build cgrc by linking to Qt5 or Qt6 using cmake.

The tool includes only what I needed to work, but if you wanted to add something else, then you are free to report bugs or file contributions on the GitHub repo.

Have fun! Bye 😉