UserAgents html-report different from show-useragents with JA3 active

[0nnyx]

Running dockerized 4.8.0 version
Before enabling ja3 zkg two months ago, outputs of "rita show-useragents" and "useragents.html" from html-report matched.
Since ja3 zkg installation, "rita show-useragents" and "useragents.html" don't match at all.
useragents.html is filled with 1000 ja3 hashes all used 1 time while show-useragents only displays 819 occurences of 1 time use.

rita show-useragents lastweek | grep ,1$ | wc -l
819

Example:
rita show-user-agents lastweek output snip

User Agent,Times Used
11e1137464a4343105031631d470cd92,12662
3e5e8d5979858e1f495ff02782601670,7710
28a2c9bd18a11de089ef85a160da29e4,6479
3fed133de60c35724739b913924b6c24,3978
Debian APT-HTTP/1.3 (1.4.11),3854
Windows-Update-Agent/10.0.10011.16384 Client-Protocol/2.33,3783

HTML report output snip

User Agent	Times Used
48b822f5ebf7646c7886969c18de908f	1
fc9ef63605fe74399ea14d0556bf2c35	1
b044fef1febe704bcb7b9a3b5c1cf675	1
2f07c6d21d5bf537d866371a99f054c3	1
e5359b0f68ca972adf7e192fd8c01ebe	1
983203c4a1ac38a4c4c1e14403d02ecd	1
92ca88530c12182264332eae85c180ba	1
7a34b08d43190057863fd8438fca4cf2	1
5464aa06da909e8e9f32bb4ae046d327	1
1a2dd688b1f2551493b7d540f8d4bdb2	1
f61eff6f78df0ec9336e97214005bf31	1

Expectation is to have html-reports match exactly the show-X commands

[Zalgo2462]

Hello, sorry for the confusion. The default sorting for the two displays is different. We prefer to display the agents in the order of least used to most used since anomalous useragents tend to be more interesting. However, the show-useragents command was developed before this preference was decided upon. In order to maintain compatibility with existing scripts, the default sorting for the show-useragents command was not updated to match the html-report.

The html-report outputs 1000 useragents in order of least used to most used. The show-useragents command will, by default, output 1000 useragents in order of most used to least used. I suspect the different sortings coupled with the limit of 1000 useragents is resulting in the differing numbers you are seeing.

In order to make the console output from show-useragents match the html report, please call it like so (assuming the dataset is named lastweek).

rita show-useragents --limit 1000 --least-used lastweek

As a side note, you may pass --no-limit to obtain the full result set, however this will likely take additional computation time.
Each of these flags are documented and may be found by calling rita show-useragents --help.

Please let us know if the command above resolves the issue you are seeing. Thank you for writing in.

[0nnyx]

Thanks for the clarifications which make complete sense.
I wished there would be a way to get an automatic ja3 lookup since the output turns out useless and I'm getting this result only running rita on a small home network weekly. However, I don't think there is any publicly available ja3 DB.

I believe there might be ways to improve the usefulness of rita (html-report for) user agents. Maybe diff'ing datasets would highlight "more unique" ja3 or limiting the results to known malicious ja3 hashes only.

Given the verbosity introduced by ja3 hashes, I don't see a point in displaying them by count only. I know I'll get 1000 unique ja3 hashes which won't tell me anything relevant.
Using SIEM and doing statistical analysis on ja3 & ja3s can turn useful to highlight the bottom 100 combination along with associated domains.
So, unless you/anyone has better suggestions on how to use the ja3 hashes effectively in the scope of rita html-report, I'd rather stick to http user agents only.