Skip to content

Codebase Deep Dive

This document explains what exists folder-by-folder and module-by-module, with practical guidance for extending behavior safely.

Repository layout

  • cli/: CLI package (Typer app, interactive UX, subcommands) -- replaces former monolithic cli.py
  • main.py: FastAPI app, static mounts, websocket endpoints
  • api/: REST handlers for scan/tools/report/health operations
  • core/: install logic, wrappers, parsers, persistence, reporting, logging/comms, shared utilities
  • core/parsers/: per-tool output parsers -- extracted from former reporter.py
  • frontend/: multi-page SPA browser dashboard
  • templates/: report rendering templates
  • scripts/: install/release/VM orchestration scripts
  • tests/: unit and integration tests
  • docs/: maintained documentation set
  • pyproject.toml: project metadata and dependencies -- replaces deleted setup.py

Deleted files

Former file Replacement
cli.py (monolithic, 973 lines) cli/ package
core/report_gen.py Merged into core/reporter.py
setup.py pyproject.toml

cli/ (package)

The former monolithic cli.py has been split into focused submodules:

Module Responsibilities
cli/__init__.py Typer app construction, top-level command wiring, version callback (via importlib.metadata)
cli/styles.py Rich console themes, shared style constants
cli/menu.py Interactive menu and UX loops
cli/server.py Server start/stop/status/logs wrappers
cli/scan.py Scan orchestration (run_quick_scan), flags: --json, --exit-code, --quiet, --verbose, --only-failures
cli/tools_cmd.py tools status / tools install subcommands
cli/history.py History browsing and report export
cli/elevation.py Privilege elevation gating (ensure_root)

Key commands:

  • scan
  • history
  • tools status / tools install
  • server run|start|status|logs|stop
  • doctor (new -- environment health check)
  • findings <scan_id> (new -- show parsed findings for a scan)

Notes:

  • Uses command whitelist for non-root commands.
  • Background server process writes timestamped logs under <runtime_home>/logs/web_server_<YYYYmmdd_HHMMSS>.log.

main.py

Responsibilities:

  1. FastAPI construction + lifespan checks.
  2. Static routes for UI assets.
  3. WebSocket handlers (/ws, /ws/, legacy compatibility routes).
  4. Router include wiring (api/scan.py, api/report.py, api/tools.py).

Operational details:

  • WebSocket limits patched via core/ws_patch.py to avoid header-size handshake failures.
  • CORS enabled permissively for local dashboard and testing scenarios.

api/

scan.py

  • Start background scans.
  • Return status/history with pagination support.
  • Generate consolidated report for scan IDs.
  • GET /scan/{id}/findings -- parsed findings for a scan.
  • GET /scan/{id}/timeline -- scan event timeline.
  • GET /scan/{id}/logs -- per-tool log retrieval.
  • DELETE /scan/{id} -- delete a scan and its artifacts.

tools.py

  • Tool compatibility/status inspection.
  • Tool install triggering.

report.py

  • PDF endpoint and report-file download helpers.

Health

  • GET /api/health -- application health check endpoint.

core/

New shared modules

Module Purpose
core/deps.py Singleton ToolManager instance (avoids re-creating across modules)
core/constants.py Shared regex patterns, magic values, and reusable constants
core/os_detect.py Unified OS/distro/version detection (replaces scattered detection in tool_manager.py and wrappers)
core/ws_patch.py WebSocket header-size and handshake patch applied at startup

tool_manager.py

Core installer/orchestration logic:

  • Package manager dispatch (apt, dnf/yum, apk, pacman, zypper)
  • Distro-version package mapping resolution (OS detection now delegates to core/os_detect.py)
  • Unsupported distro/version skip guards
  • Tool verify hooks
  • Windows Java bootstrap for CIS-CAT

tools_config.py

Static metadata:

  • Tool definitions per OS
  • Package mapping table (PACKAGE_MAPPINGS)
  • Per-tool notes and compatibility restrictions

wrappers/linux.py

Scanner execution:

  • OpenSCAP content autodiscovery and profile selection
  • Lynis invocation + output paths
  • USG invocation (when available)
  • CIS-CAT logic extracted to wrappers/ciscat_linux.py

wrappers/ciscat_linux.py (new)

  • CIS-CAT benchmark selection and execution on Linux
  • Extracted from linux.py for separation of concerns

wrappers/windows.py

Scanner execution:

  • HardeningKitty invocation (PowerShell)
  • CIS-CAT runner invocation
  • SCT (LGPO) integration path

core/parsers/ (new package)

Per-tool output parsers, extracted from the former monolithic reporter.py:

Module Parses
core/parsers/oscap.py OpenSCAP XML ARF output
core/parsers/lynis.py Lynis .dat log files
core/parsers/hardeningkitty.py HardeningKitty CSV output
core/parsers/ciscat.py CIS-CAT results

reporter.py (slim orchestrator)

  • Aggregates parsed findings from core/parsers/*
  • Generates consolidated PDF/HTML reports
  • report_gen.py has been merged into this module and deleted
  • Handles partial/missing artifacts with explicit failure entries

database.py

  • SQLite with WAL mode
  • Lazy initialization -- no module-level init_db() call; DB is created on first access
  • completed_at column for scan completion tracking
  • delete_scan() and count_scans() helpers
  • Pagination support for scan queries

scan_logger.py

  • Per-tool log files (e.g., <scan_id>/lynis.ndjson, <scan_id>/openscap.ndjson)
  • Lazy directory initialization (dirs created on first write)

logger.py

  • Standard logging configuration
  • JSON structured logging via LOG_FORMAT=json environment variable

utils.py

  • OS helper utilities
  • get_clean_env() for subprocess safety
  • command stream helpers

comms.py

  • WebSocket connection management + broadcasts

scripts/

  • Artifact download and VM test orchestration
  • In-VM Linux and Windows test runners
  • Install scripts (install.sh, install.ps1)
  • Inno Setup file for Windows installer

frontend/ and templates/

The frontend is a multi-page SPA with hash-based routing:

Page Description
Dashboard KPI cards, compliance overview
Scans Scan list with pagination, per-scan inline logs, modal log viewer
Tools Tool status and install controls
Logs Aggregated log browser

Additional features: - Dark/light theme toggle - Toast notifications - CSV/JSON export

Report HTML templates for PDF generation are in templates/. Both frontend/ and templates/ must be bundled in packaged binaries.

tests/

  • API, DB, wrappers, websocket, tool-manager unit coverage
  • integration and docker-assisted test options

Version management

The package version is read from importlib.metadata at runtime. It is defined solely in pyproject.toml (not hardcoded anywhere in source).

Extension guidelines

  1. Add or change tool definitions in core/tools_config.py first.
  2. Keep installer behavior in core/tool_manager.py generic and data-driven.
  3. If scanner invocation changes, update wrapper + parser (in core/parsers/) + docs together.
  4. New parsers go in core/parsers/ and are wired into core/reporter.py.
  5. New CLI subcommands go in cli/ as separate modules and are registered in cli/__init__.py.
  6. For any packaging-affecting path change, update .github/workflows/release.yml.
  7. For any release behavior change, update VM runner scripts and docs in one PR.