Linux Server Inventarisierung erstellen

Vorbereitung für die Inventarisierung

Als erstes werden die benötigten Pakete installiert und eine Passwort-Datei für die automatisch Verschlüsselung der gesammelten Daten angelegt.

# Grundpakete installieren apt install acl dstat dmidecode dnsutils gnupg2 lynis nmap osquery pigz psmisc rng-tools ufw # Osquery installieren export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $OSQUERY_KEY add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main' aptitude update && aptitude install osquery # Lynis installieren apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C80E383C3DE9F082E01391A0366C67DE91CA5D5F echo "deb https://packages.cisofy.com/community/lynis/deb/ stable main" > /etc/apt/sources.list.d/cisofy-lynis.list aptitude update && aptitude install lynis # GPG-Key Passwortdatei für das verschlüsselte Inventory-Backup erstellen echo $PASSWORD > /root/.inventory

Das Inventarisierungs-Skript

Das Skript legt mit den Variablen INVENT_DIR und BACKUP_DIR die Ordnerstruktur an, setzt für diese stabile Berechtigungen fest und fängt dann mit der sch(w)eißtreibenden Inventarisierung (Baselining) an. Alle Infos zu dem System werden in dafür vorgegebenen Dateien gepumpt. Am Ende wird werden die gesammelten Dateien gepackt, verschlüsselt, das unverschlüsselte Archive entfernt und als letztes werden alte Inventarisierungen entfernt. Somit bleiben zum inspizieren die gesammelten Dateien und das verschlüsseltes Archiv übrig, welches man z.B per SCP auf einen Remote-Server pumpen sollte.

Aufbau der Baselinig-Dateien:

# sys=Systeminfos, net=Network, sec=Security, perf=Performance, osq=Osquery System-Infos -> sys-memory.txt Netzwerk-Infos -> net-ip-addr.txt Security-Infos -> sec-ufw-firewall.txt Performance-Infos -> perf-dd-disk-speed.txt Osquery-Abfragen -> osq-sys-osversion.txt

inventory.sh

#!/usr/bin/env bash #========================================================== # Autor: Hackspoiler # URL: https://hackspoiler.de # Version: 1.9 # Shellcheck: True # Date: 02.05.2020 # Modification: 21.04.2023 # Scriptname: sec-os-inventory.sh # Scriptpfad: /root/server-scripts/ # Packages: acl dstat dmidecode dnsutils gnupg2 lnav lynis nmap osquery pigz psmisc rng-tools ufw # Glossar: net=Network, osq=Osquery, perf=Performance, sec=Security, sys=Systeminfos # Description: Erstellt eine Inventarisierung (Baselining) eines Linux Servers #========================================================== # Check for root permissions if [[ $(id -u) -ne 0 ]]; then echo "Requires root permissions" > /dev/stderr exit 1 fi # Set Bash-Defaults set -o nounset # Beenden, falls ein ungesetzter Variablenname verwendet wird set -o pipefail # Beenden, falls eine Pipeline fehlschlägt #BACKIFS="${IFS}" #IFS=$'\n\t' #========================================================== # Grundeinstellungen #========================================================== # Variablen TIMESTAMP=$(date +'%F-%H-%M-%S') LOGFILE="/var/log/sec-os-inventory.log" INVENT_DIR="/root/inventory/${TIMESTAMP}" BACKUP_DIR="/var/backups/inventory/${TIMESTAMP}" #BACKUP_SRV="root@$IP_ADDRESS" # Programm-Optionen je nach Ressourcen anpassen XARGS=('xargs' '--max-procs=2') PIGZ="pigz --fast --processes 2 --keep" ## Logging starten exec > >(tee --ignore-interrupts "${LOGFILE}") exec 2>&1 # Ordnerstruktur anlegen mkdir --parents "${INVENT_DIR}" "${BACKUP_DIR}" # Berechtigung brav setzen find "${INVENT_DIR}" "${BACKUP_DIR}" -type d | "${XARGS[@]}" chmod --verbose 700 #========================================================== # Datei-Indexierung auf dem System #========================================================== file_index_check () { echo "# Inventory - Datei-Indexierung auf dem System startet - $(date +'%H-%M-%S') #" ls -l --all --human-readable /* > "${INVENT_DIR}"/sys-list-root-files.txt find / -xdev ! -path "/mnt/*" ! -path "/proc/*" -print0 2> /dev/null | "${XARGS[@]}" -0 ls -l > "${INVENT_DIR}"/sys-list-all-files.txt } #========================================================== # System-Infos #========================================================== system_check () { echo "# Inventory - Systeminfo-Collection startet - $(date +'%H-%M-%S') #" lastlog > "${INVENT_DIR}"/sys-lastlog.txt free --mega > "${INVENT_DIR}"/sys-memory.txt ps -afjx > "${INVENT_DIR}"/sys-processlist.txt hostnamectl > "${INVENT_DIR}"/sys-hostinfo.txt cat /etc/group > "${INVENT_DIR}"/sys-group.txt cat /etc/fstab > "${INVENT_DIR}"/sys-fstab.txt cat /etc/passwd > "${INVENT_DIR}"/sys-user.txt dpkg --list > "${INVENT_DIR}"/sys-deb-packages.txt who --all > "${INVENT_DIR}"/sys-logged-in-users.txt uname --all > "${INVENT_DIR}"/sys-kernelversion.txt dmidecode --type 4 > "${INVENT_DIR}"/sys-cpu-info.txt pstree --arguments > "${INVENT_DIR}"/sys-process-pstree.txt cat /root/.bash_history > "${INVENT_DIR}"/sys-root-history.txt sysctl --all > "${INVENT_DIR}"/sys-sysctl-settings.txt df --human-readable | sort --key 2 --reverse > "${INVENT_DIR}"/sys-disk-space.txt systemctl --type=service --state=active > "${INVENT_DIR}"/sys-systemctl-active-services.txt systemctl list-dependencies graphical.target > "${INVENT_DIR}"/sys-systemctl-services-tree.txt systemctl list-units --all --type=service --no-pager > "${INVENT_DIR}"/sys-systemctl-services.txt grep --extended-regexp --invert-match "^\s*(#|$)" /var/spool/cron/crontabs/* > "${INVENT_DIR}"/sys-crontabs.txt lsblk --output NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT,UUID,OWNER,GROUP,MODE,RO,MODEL,STATE > "${INVENT_DIR}"/sys-disks.txt du --human-readable --summarize --one-file-system --exclude=/{proc,sys,dev,run} /* > "${INVENT_DIR}"/sys-directory-space.txt find / -xdev ! -path "/mnt/*" ! -path "/proc/*" -type f -size +500M | "${XARGS[@]}" --delimiter '\n' ls -l --all --human-readable > "${INVENT_DIR}"/sys-big-files.txt } 2> /dev/null #========================================================== # Netzwerk-Infos #========================================================== network_check () { echo "# Inventory - Netzwerkinfo-Collection startet - $(date +'%H-%M-%S') #" ip -brief -color route show > "${INVENT_DIR}"/net-route.txt ip -brief -color addr show > "${INVENT_DIR}"/net-ip-addr.txt ip -color -details addr show >> "${INVENT_DIR}"/net-ip-addr.txt lsof -i -i4 -i6 > "${INVENT_DIR}"/net-lsof-stat-networkresources.txt ss --listening --processes --tcp --udp > "${INVENT_DIR}"/net-ss-stat.txt } 2> /dev/null #========================================================== # Security-Infos #========================================================== security_check () { echo "# Inventory - Securityinfo-Collection startet - $(date +'%H-%M-%S') #" umask > "${INVENT_DIR}"/sec-umask.txt cat /etc/sudoers > "${INVENT_DIR}"/sec-sudoers.txt ufw status verbose > "${INVENT_DIR}"/sec-ufw-firewall.txt nmap -p 1-65535 -T4 -sS localhost --max-retries 1 > "${INVENT_DIR}"/sec-nmap-portscan.txt # Mehr Performance = Rustscan lynis audit system --auditor "The Boss" --quick --quiet --logfile "${INVENT_DIR}"/sec-lynis.txt cat /root/.ssh/{known_hosts,authorized_keys} > "${INVENT_DIR}"/sec-root-ssh-keys.txt find /etc -type f -print0 | "${XARGS[@]}" --null sha256sum > "${INVENT_DIR}"/sec-sha256sum-etc.txt find /usr -type f -print0 | "${XARGS[@]}" --null sha256sum > "${INVENT_DIR}"/sec-sha256sum-usr.txt find / -xdev ! -path "/mnt/*" ! -path "/proc/*" -size +0 -print0 | "${XARGS[@]}" --null -I {} getfacl --absolute-names {} > "${INVENT_DIR}"/sec-getfacl-permissions.acl find / -xdev ! -path "/mnt/*" ! -path "/proc/*" -nouser -o -nogroup -print0 | "${XARGS[@]}" --null ls -l --all --human-readable > "${INVENT_DIR}"/sec-nouser-nogroup.txt find / -xdev ! -path "/mnt/*" ! -path "/proc/*" ! -type l ! -type c ! -type b -perm -2 -print0 | "${XARGS[@]}" --null ls -l --all --human-readable > "${INVENT_DIR}"/sec-world-writable.txt } 2> /dev/null #========================================================== # Performance-Infos #========================================================== performance_check () { echo "# Inventory - Performance-Collection startet - $(date +'%H-%M-%S') #" dstat --top-cpu-adv --top-latency --top-io --top-mem --mem --load --time 1 20 > "${INVENT_DIR}"/perf-dstat.txt dd if=/dev/zero of="${BACKUP_DIR}"/testfile bs=512 count=1000 oflag=dsync 2> "${INVENT_DIR}"/perf-dd-disk-speed.txt && rm --preserve-root "${BACKUP_DIR}"/testfile dd if=/dev/zero of="${BACKUP_DIR}"/testfile bs=1MB count=1 oflag=dsync 2>> "${INVENT_DIR}"/perf-dd-disk-speed.txt && rm --preserve-root "${BACKUP_DIR}"/testfile dd if=/dev/zero of="${BACKUP_DIR}"/testfile bs=100MB count=1 oflag=dsync 2>> "${INVENT_DIR}"/perf-dd-disk-speed.txt && rm --preserve-root "${BACKUP_DIR}"/testfile dd if=/dev/zero of="${BACKUP_DIR}"/testfile bs=500MB count=1 oflag=dsync 2>> "${INVENT_DIR}"/perf-dd-disk-speed.txt && rm --preserve-root "${BACKUP_DIR}"/testfile #fio --name inventory-"${TIMESTAMP}" --eta-newline=5s --filename="${BACKUP_DIR}"/fio-testfile.dat --rw=randread --size=1000m --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=100 --direct=1 --numjobs=1 --runtime=60 --group_reporting && rm --preserve-root "${BACKUP_DIR}"/fio-testfile.dat } 2> /dev/null #========================================================== # Osquery-Abfragen #========================================================== OSQUERY="$(which osqueryi)" osquery_check () { echo "# Inventory - Osquery-Collection startet - $(date +'%H-%M-%S') #" # Osquery - Config check osqueryi "select config_hash, config_valid from osquery_info;" > "${INVENT_DIR}"/osq-configcheck.txt # Osquery - Systeminfos osqueryi "SELECT * FROM memory_info;" > "${INVENT_DIR}"/osq-sys-memory.txt osqueryi "SELECT * FROM os_version;" > "${INVENT_DIR}"/osq-sys-osversion.txt osqueryi "SELECT count(*) FROM crontab;" > "${INVENT_DIR}"/osq-sys-crontabs.txt osqueryi "SELECT command, path FROM crontab;" >> "${INVENT_DIR}"/osq-sys-crontabs.txt osqueryi "SELECT days, datetime(time.local_time - uptime.total_seconds, 'unixepoch') AS last_rebooted FROM time, uptime;" > "${INVENT_DIR}"/osq-sys-uptime.txt osqueryi "SELECT hostname, cpu_brand, cpu_physical_cores, cpu_logical_cores, computer_name, physical_memory FROM system_info;" > "${INVENT_DIR}"/osq-sys-hardware.txt osqueryi "SELECT pid, name, ROUND((total_size * '10e-7'), 2) AS used FROM processes ORDER BY total_size DESC LIMIT 10;" > "${INVENT_DIR}"/osq-sys-memory-intensive-processes.txt osqueryi "SELECT pid, name, path, cmdline, state, cwd, root, uid, gid, suid, sgid, on_disk, total_size, user_time, system_time, start_time, parent, pgroup, threads, nice FROM processes LIMIT 20;" >> "${INVENT_DIR}"/osq-sys-memory-intensive-processes.txt osqueryi "SELECT device, path, type, blocks_size, blocks, blocks_free, blocks_available, inodes, inodes_free, flags FROM mounts where type!='cgroup' and type!='tmpfs' and inodes!='0';" > "${INVENT_DIR}"/osq-sys-devices.txt osqueryi "SELECT device, device_alias, path, blocks, blocks_free, inodes, inodes_free, flags, encrypted, encryption_status FROM mounts m, disk_encryption d WHERE m.device_alias = d.name;" > "${INVENT_DIR}"/osq-sys-devices-encrypted.txt osqueryi "SELECT pid, uid, name, ROUND(( (user_time + system_time) / (cpu_time.tsb - cpu_time.itsb)) * 100, 2) AS percentage FROM processes, (SELECT (SUM(user) + SUM(nice) + SUM(system) + SUM(idle) * 1.0) AS tsb, SUM(COALESCE(idle, 0)) + SUM(COALESCE(iowait, 0)) AS itsb FROM cpu_time) AS cpu_time ORDER BY user_time+system_time DESC LIMIT 20;" > "${INVENT_DIR}"/osq-sys-cpu-intensive-processes.txt # Osquery - Paketmanagement osqueryi "SELECT count(*) FROM deb_packages;" > "${INVENT_DIR}"/osq-sys-deb-packages.txt osqueryi "SELECT name, version, source, size, arch, revision, status FROM deb_packages;" >> "${INVENT_DIR}"/osq-sys-deb-packages.txt osqueryi "SELECT name, base_uri, release, maintainer, components FROM apt_sources ORDER BY name;" > "${INVENT_DIR}"/osq-sys-deb-sources.txt # Osquery - Usermanagement osqueryi "SELECT * FROM last;" > "${INVENT_DIR}"/osq-sys-login-users.txt osqueryi "SELECT * FROM logged_in_users;" > "${INVENT_DIR}"/osq-sys-loggedin-users.txt osqueryi "SELECT * FROM users WHERE gid < 65534 AND uid >= 1000;" > "${INVENT_DIR}"/osq-sys-non-system-users.txt osqueryi "SELECT count(*) FROM users" > "${INVENT_DIR}"/osq-sys-all-users.txt osqueryi "SELECT uid, gid, uid_signed, gid_signed, username, description, directory, shell FROM users;" >> "${INVENT_DIR}"/osq-sys-all-users.txt # Osquery - Aktive Kernelmodule osqueryi "SELECT count(*) FROM kernel_modules WHERE status='Live';" > "${INVENT_DIR}"/osq-sys-kernelmodule.txt osqueryi "SELECT name, used_by, status FROM kernel_modules WHERE status='Live';" >> "${INVENT_DIR}"/osq-sys-kernelmodule.txt # Osquery - Securitymanagement osqueryi "SELECT count(*) FROM suid_bin;" > "${INVENT_DIR}"/osq-sec-setuid.txt osqueryi "SELECT * FROM suid_bin;" >> "${INVENT_DIR}"/osq-sec-setuid.txt osqueryi "SELECT count(*) FROM sudoers;" > "${INVENT_DIR}"/osq-sec-sudoers.txt osqueryi "SELECT * FROM sudoers;" >> "${INVENT_DIR}"/osq-sec-sudoers.txt osqueryi "SELECT count(*) FROM iptables;" > "${INVENT_DIR}"/osq-sec-firewall-all.txt osqueryi "SELECT * FROM iptables;" >> "${INVENT_DIR}"/osq-sec-firewall-all.txt osqueryi "SELECT chain, policy, src_ip, dst_ip FROM iptables;" > "${INVENT_DIR}"/osq-sec-firewall-short.txt osqueryi "SELECT suid_bin.path, username, groupname, permissions, hash.sha256 FROM hash JOIN suid_bin USING (path);" >> "${INVENT_DIR}"/osq-sec-setuid.txt # Osquery - Network osqueryi "SELECT count(*) FROM routes;" > "${INVENT_DIR}"/osq-net-routes.txt osqueryi "SELECT * FROM routes;" >> "${INVENT_DIR}"/osq-net-routes.txt osqueryi "SELECT * FROM etc_hosts;" > "${INVENT_DIR}"/osq-net-hosts.txt osqueryi "SELECT count(*) FROM listening_ports;" > "${INVENT_DIR}"/osq-net-listening-ports.txt osqueryi "SELECT DISTINCT processes.name, listening_ports.path, processes.pid, listening_ports.port, listening_ports.protocol, listening_ports.family, listening_ports.address, listening_ports.socket FROM listening_ports JOIN processes using (pid);" >> "${INVENT_DIR}"/osq-net-listening-ports.txt osqueryi "SELECT p.cmdline, pos.local_address, pos.remote_address, local_port, pos.remote_port from process_open_sockets as pos JOIN processes as p ON pos.pid=p.pid WHERE pos.state='ESTABLISHED';" > "${INVENT_DIR}"/osq-net-established-connections.txt # Osquery - Docker osqueryi "SELECT id, key, value FROM docker_image_labels;" > "${INVENT_DIR}"/osq-docker-images.txt osqueryi "SELECT name, id, os, cpus, memory, containers, containers_running, containers_paused, containers_stopped, images, storage_driver, server_version, root_dir FROM docker_info;" > "${INVENT_DIR}"/osq-docker-infos.txt } 2> /dev/null #========================================================== # Abschluss #========================================================== abschluss () { # Checksum aus allen Dateien bilden echo "# Inventory - Checksum startet - $(date +'%H-%M-%S') #" find "${INVENT_DIR}"/ -type f -print0 | "${XARGS[@]}" --null sha256sum > "${INVENT_DIR}"/sec-sha256sum-inventory.txt # Inventory-Archiv inklusive wichtiger Systemdaten (Beispiel etc, usr) erstellen echo "# Inventory - Inventory-Archivierung startet - $(date +'%H-%M-%S') #" tar --use-compress-program="${PIGZ}" --create --file "${BACKUP_DIR}"/sec-os-inventory-$(hostname --fqdn)-"${TIMESTAMP}".tgz \ /etc \ /usr \ "${INVENT_DIR}"/* 2> /dev/null # Inventory-Archiv verschlüsseln echo "# Inventory - Inventory-Archiv-Encryption startet - $(date +'%H-%M-%S') #" cat /root/.inventory | /usr/bin/gpg --batch --yes --passphrase-fd 0 --symmetric "${BACKUP_DIR}"/sec-os-inventory-$(hostname --fqdn)-"${TIMESTAMP}".tgz # Unverschlüsseltes Inventory-Archiv entfernen echo "# Inventory - Entfernung des einfachen Inventory-Archiv startet #" rm --force --preserve-root "${BACKUP_DIR}"/sec-os-inventory-$(hostname --fqdn)-"${TIMESTAMP}".tgz # Inventory-Archive älter als einen Tag rasieren echo "# Inventory - Entfernung alter Inventory-Archive startet - $(date +'%H-%M-%S') #" find /root/inventory/ /var/backups/inventory/ -mindepth 1 -type d -mtime +1 -print0 | "${XARGS[@]}" --null -I {} rm --force --preserve-root --recursive {} # Archiv auf Backup-Server pumpen #rsync --progress --verbose "${BACKUP_DIR}"/inventory-*.tgz "${BACKUP_SRV}":/"${BACKUP_DIR}"/ # Haben fertig echo "# Inventory - Inventory wurde um $(date +'%H-%M-%S') in ${BACKUP_DIR} erstellt - Zeit für ein hochverdientes Bio-Bierchen #" } #========================================================== # Funktionsaufrufe #========================================================== file_index_check system_check network_check security_check performance_check [[ -n "${OSQUERY}" ]] && osquery_check abschluss #========================================================== # Decrypt + Dekomprimierung #========================================================== # gpg --batch --passphrase $PASSWD --decrypt "${BACKUP_DIR}"/sec-os-inventory-$(hostname --fqdn)-"${TIMESTAMP}".tgz.gpg | tar --extract --gunzip # cat /root/.inventory | /usr/bin/gpg --batch --yes --passphrase-fd 0 -d "${BACKUP_DIR}"/sec-os-inventory-$(hostname --fqdn)-"${TIMESTAMP}".tgz.gpg | tar --extract --gunzip

Ist das Skript durchgerödelt hat man es sich auf jeden Fall wieder verdient, dem Gott des Hochprozentigen zu frönen und zur Feier des Tages grüßt täglich das Murmeltier 😅🤣😅

PS: Aus Zeitmangel konnte ich das Skript nicht so hart ran nehmen und durchtesten. Falls etwas nicht passen sollten oder es leckere Verbesserungsvorschläge gibt dann bitte in die Kommentare ballern.