the "full" profile doesn't contain a full file listing ( bodyfile )

[59e5aaf4]

Describe the bug
When running uac -p full, one might assume a full file listing is collected, it's not.

To Reproduce
Run uac -p full

Expected behavior
I expected a full file listing of the local disks to be present. Timelining an entire hard drive is extremely useful for forensics operations, and I (wrongly) assumed full will contain that ( vs ir_triage ).

Please add bodyfile to the list of things done by "full" as your users might assume "full" means "grab everything possible". Thanks !

UAC Information

• Version: 3.2.0

Target System Information:

• OS: FreeBSD
• Version: 11.4-NETSCALER-13.1
• Arch: amd64

[59e5aaf4]

Nevermiiiind... bodyfile is indeed in there, but FreeBSD doesn't have the required tools.

INF Parsing bodyfile/bodyfile.yaml
ERR _find_based_collector: Cannot run stat collector. Target system has neither 'stat', 'statx' nor 'perl' tool available

But find is there

INF find operators support: true
INF find -atime support: true
INF find -ctime support: true
INF find -maxdepth support: true
INF find -mtime support: true
INF find -nogroup support: true
INF find -nouser support: true
INF find -path support: true
INF find -perm support: true
INF find -print0 support: true
INF find -prune support: true
INF find -size support: true
INF find -type support: true

So maybe you could have some sort of find . -type f -printf "%AY-%Am-%Ad-%AH-%AM-%AS %CY-%Cm-%Cd-%CH-%CM-%CS %TY-%Tm-%Td-%TH-%TM-%TS %y %s %G %U %m %p\n" equivalent ?

[tclahr]

Yes. I think in situations like that where neither stat nor perl is available.
I will work on it and let you know when it is ready for testing. Thanks

[59e5aaf4]

FYI, in our scope were several FreeBSD with or without "stat" ( despite having the same OS version & patch .. ) , we ended up using a script like the following to timeline our several hosts as Python3 was available everywhere.

• Format is our own custom TSV because I don't like the "bodyfile" de-facto standard for timelines.
• Thanks for your work, it's really useful ! ( And really less complex to instadeploy than Velociraptor :D )
• Warning that mode_to_string function is AI-generated, I found it to work correctly but didn't run extensive tests

from datetime import datetime
from pathlib import Path
import os
import stat


def mode_to_string(mode: int) -> str:
    """Return ls -l style mode string with suid/sgid/sticky bits."""
    # file type
    if   stat.S_ISDIR(mode):  ftype = "d"
    elif stat.S_ISLNK(mode):  ftype = "l"
    elif stat.S_ISCHR(mode):  ftype = "c"
    elif stat.S_ISBLK(mode):  ftype = "b"
    elif stat.S_ISSOCK(mode): ftype = "s"
    elif stat.S_ISFIFO(mode): ftype = "p"
    else:                     ftype = "-"

    # user perms
    ur = "r" if (mode & stat.S_IRUSR) else "-"
    uw = "w" if (mode & stat.S_IWUSR) else "-"
    ux = "x" if (mode & stat.S_IXUSR) else "-"
    if mode & stat.S_ISUID:
        ux = "s" if (mode & stat.S_IXUSR) else "S"

    # group perms
    gr = "r" if (mode & stat.S_IRGRP) else "-"
    gw = "w" if (mode & stat.S_IWGRP) else "-"
    gx = "x" if (mode & stat.S_IXGRP) else "-"
    if mode & stat.S_ISGID:
        gx = "s" if (mode & stat.S_IXGRP) else "S"

    # other perms
    or_ = "r" if (mode & stat.S_IROTH) else "-"
    ow  = "w" if (mode & stat.S_IWOTH) else "-"
    ox  = "x" if (mode & stat.S_IXOTH) else "-"
    if mode & stat.S_ISVTX:
        ox = "t" if (mode & stat.S_IXOTH) else "T"

    return ftype + ur + uw + ux + gr + gw + gx + or_ + ow + ox

def generate_bodyfile(root_dir, output_file):
    with open(output_file, "wb") as out:
        for dirpath, _, filenames in os.walk(root_dir):
            for fname in filenames:
                fpath = os.path.join(dirpath, fname)
                full_path = Path(fpath).absolute()
                try:
                    st = full_path.stat()
                except:
                    line = f'?\t?\t?\t?\t?\t?\t?\t?\t?\t?\t{full_path}\n'
                    out.write(bytes(line,'utf-8','surrogatepass'))
                    continue
                inode = st.st_ino
                mode_i = st.st_mode
                mode_s = mode_to_string(st.st_mode)
                uid = st.st_uid
                gid = st.st_gid
                size = st.st_size
                atime = datetime.fromtimestamp(st.st_atime)
                mtime = datetime.fromtimestamp(st.st_mtime)
                ctime = datetime.fromtimestamp(st.st_ctime)
                # Use full path as "name"
                line = f'{atime}\t{ctime}\t{mtime}\t{mode_s}\t{mode_i}\t{mode_i:o}\t{uid}\t{gid}\t{inode}\t{size}\t{full_path}\n'
                out.write(bytes(line,'utf-8','surrogatepass'))

if __name__ == "__main__":
    import argparse
    hostname = os.uname().nodename
    now = datetime.now().timestamp()
    parser = argparse.ArgumentParser(description="Collects all timestamps recursively in a directory. Outputs in full_listing.hostname.txt.")
    parser.add_argument("-r","--root", help="base folder",default="/")
    parser.add_argument('-o','--output', default=f'full_listing.{hostname}.{now:.0f}.txt')
    args = parser.parse_args()
    print(f"Will recursively list '{args.root}' and store results in {args.output}")
    generate_bodyfile(args.root, args.output)
    chrono = datetime.now().timestamp() - now
    print(f"Done in {chrono:5.2f}s. Send {args.output} back to whomever asked you to run that)")

[tclahr]

The BSD-based systems such as macOS, FreeBSD, NetBSD, OpenBSD and NetScaler do not support find -printf. So using it to create the bodyfile would not be a solution.

$ find /etc -printf "0|%p|%i|%M|%U|%G|%s|%A@|%T@|%C@|%b\n"
find: -printf: unknown option

I would not use python either as is not installed by default on some of those systems and uses a lot of resources to run.

[tclahr]

I wrote a very small stat c program (based on statx) to run on FreeBSD-based systems (i386 and amd64). It is a statically binary file that outputs file stats in the same bodyfile format. I have pushed the code to develop branch for testing.

PR #404

[59e5aaf4]

Agreed, here we ended up using Python as the handful of hosts without find with -printf had Python installed and it worked quickly, but a good old C program makes sense here. Not sure what the cross-OS/FS permission bits will look like. Thanks again for the good work and have a nice day.

[tclahr]

I tested several combinations using special bits and it seems to be working fine. Tested on both amd64 and i386 archs.

root@uac-freebsd-8-amd64-tools:/home/tclahr/tools/make_perms_test/perms_special_test # ls -la
total 54
drwxr-xr-x  5 root    tclahr   512 Nov 11 01:40 .
drwxr-xr-x  3 tclahr  tclahr   512 Nov 11 01:40 ..
-rwsr-xr-x  1 root    tclahr    20 Nov 11 01:40 demo_script.sh
drwxr-xr-x  2 root    tclahr   512 Nov 11 01:40 dir_normal
drwxrwsr-x  2 root    tclahr   512 Nov 11 01:40 dir_setgid_2775
drwxrwxrwt  2 root    tclahr   512 Nov 11 01:40 dir_sticky_1777
-rwxr-xr-x  1 root    tclahr     0 Nov 11 01:40 f_special_0
-rwxr-xr-t  1 root    tclahr     0 Nov 11 01:40 f_special_1000
-rwxr-sr-x  1 root    tclahr     0 Nov 11 01:40 f_special_2000
-rwxr-sr-t  1 root    tclahr     0 Nov 11 01:40 f_special_3000
-rwsr-xr-x  1 root    tclahr     0 Nov 11 01:40 f_special_4000
-rwsr-xr-t  1 root    tclahr     0 Nov 11 01:40 f_special_5000
-rwsr-sr-x  1 root    tclahr     0 Nov 11 01:40 f_special_6000
-rwsr-sr-t  1 root    tclahr     0 Nov 11 01:40 f_special_7000
-rwxr-xr-x  1 root    tclahr  8339 Nov 11 01:40 hello_printuids
-rw-r--r--  1 root    tclahr   607 Nov 11 01:40 hello_printuids.c
-rwxr-sr-x  1 root    wheel   8339 Nov 11 01:40 hello_sgid
-rwsr-sr-x  1 root    wheel   8339 Nov 11 01:40 hello_su_sg
-rwsr-xr-x  1 root    tclahr  8339 Nov 11 01:40 hello_suid
root@uac-freebsd-8-amd64-tools:/home/tclahr/tools/make_perms_test/perms_special_test # statf *
0|demo_script.sh|3721676|-rwsr-xr-x|0|1001|20|1762832442|1762832442|1762832442|1762832442
0|dir_normal|3721665|drwxr-xr-x|0|1001|512|1762832442|1762832442|1762832442|1762832442
0|dir_setgid_2775|3721669|drwxrwsr-x|0|1001|512|1762832442|1762832442|1762832442|1762832442
0|dir_sticky_1777|3721666|drwxrwxrwt|0|1001|512|1762832442|1762832442|1762832442|1762832442
0|f_special_0|3721657|-rwxr-xr-x|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_1000|3721658|-rwxr-xr-t|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_2000|3721659|-rwxr-sr-x|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_3000|3721660|-rwxr-sr-t|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_4000|3721661|-rwsr-xr-x|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_5000|3721662|-rwsr-xr-t|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_6000|3721663|-rwsr-sr-x|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|f_special_7000|3721664|-rwsr-sr-t|0|1001|0|1762832442|1762832442|1762832442|1762832442
0|hello_printuids|3721672|-rwxr-xr-x|0|1001|8339|1762832442|1762832442|1762832442|1762832442
0|hello_printuids.c|3721671|-rw-r--r--|0|1001|607|1762832442|1762832442|1762832442|1762832442
0|hello_sgid|3721674|-rwxr-sr-x|0|0|8339|1762832442|1762832442|1762832442|1762832442
0|hello_su_sg|3721675|-rwsr-sr-x|0|0|8339|1762832442|1762832442|1762832442|1762832442
0|hello_suid|3721673|-rwsr-xr-x|0|1001|8339|1762832442|1762832442|1762832442|1762832442