feat: add --siem flag to security-audit report for NDJSON export#1876
feat: add --siem flag to security-audit report for NDJSON export#1876jlima8900 wants to merge 4 commits intoKeeper-Security:releasefrom
Conversation
0460eea to
465e68a
Compare
465e68a to
d46b168
Compare
| report_title = f'Security Audit Report{" (BreachWatch)" if show_breachwatch else ""}' | ||
|
|
||
| # SIEM-ready NDJSON output | ||
| if kwargs.get('siem'): |
There was a problem hiding this comment.
This gets silently ignored when --record-details is set, because the method returns from the record-detail branch before it ever reaches the SIEM branch. If we want to support that combination, it needs handling there; otherwise we should reject the flag combo explicitly instead of accepting it and returning the normal record-detail report.
| report = '\n'.join(ndjson_lines) + '\n' | ||
|
|
||
| if filename: | ||
| if not os.path.splitext(filename)[1]: |
There was a problem hiding this comment.
This silently bypasses the shared --format / report-output path. Once --siem is set, fmt no longer governs behavior, and --output is honored even though the shared parser help says output is ignored for table mode. The main issue here is the lack of validation or explicit contract for how --siem interacts with --format and the normal report-output path. I would either validate the combination and fail closed, or make the override explicit in the CLI contract.
| } | ||
| ndjson_lines.append(json.dumps(event, default=str)) | ||
|
|
||
| report = '\n'.join(ndjson_lines) + '\n' |
There was a problem hiding this comment.
For the empty case this produces a single blank line, not zero NDJSON records, because "\n".join([]) + "\n" is just "\n". That is not actually "one JSON object per line" output, and some ingest pipelines will treat the blank line as a parse error. We should special-case the empty report to emit an empty string/file instead.
1be7358 to
978cb83
Compare
Adds NDJSON output for SIEM ingestion (Splunk, Elastic, Datadog). - --siem flag produces one JSON object per line - Rejects --siem + --record-details combination - Warns when --siem overrides explicit --format - Empty reports produce empty output - security_score field only emitted in non-BreachWatch mode - Output file written with 0o600 permissions
978cb83 to
5f59cb2
Compare
Address PR Keeper-Security#1876 review from @aaunario-keeper: - --siem + --format now raises CommandError instead of silently ignoring the format flag, making the override explicit in the CLI contract - Add clarifying parentheses on empty-report ternary to make precedence obvious (behavior unchanged — empty list already produced '')
|
Good catches across the board — addressing each:
Empty report blank line: Good edge case — Clear-text storage (CodeQL): Will review the flagged line and ensure sensitive fields are masked or excluded from the NDJSON output. |
Address review feedback: - Reject --siem + --record-details combination with clear error - Reject --siem + --format combination (SIEM is NDJSON-only) - Fix empty report producing blank line instead of empty output - Mask sensitive fields in SIEM NDJSON output (CodeQL finding)
- Add os.fchmod(fd, 0o600) to ensure existing files get restrictive permissions (os.open mode only applies on creation) - Add 25 tests covering NDJSON format, PII masking, risk factors, empty report handling, file permissions, flag validation, and BreachWatch mode
Summary
Adds SIEM-ready output to
security-audit reportusing NDJSON format (one JSON event per line). The existingaudit-logcommand has SIEM export (Splunk, syslog, Sumo, Azure) for event streaming, butsecurity-audit report(posture snapshots) has no SIEM output.Usage
Output format (NDJSON — one line per user)
{"event_type":"keeper.security_audit","timestamp":"2026-03-14T22:30:00+00:00","source":"keepersecurity.eu","user":{"email":"user@company.com","securityScore":52,"weak":8,"reused":6,"twoFactorChannel":"none"},"security_score":52,"risk_factors":["weak_passwords","reused_passwords","no_2fa"]}Design decisions
jq -c, Filebeat, Splunk HEC, Datadog logs.--siemflag not--format siem— avoids modifying the sharedreport_output_parserused by all commands. Clean addition, no side effects.risk_factorsandsecurity_score. SIEM correlation rules should define what's critical for each organization, not Commander..ndjsonextension — standard for newline-delimited JSON files.No breaking changes
--siemis additive — only affects output when explicitly usedaudit-logSIEM targets unaffectedTest plan
--siemproduces valid NDJSON (one JSON object per line)jq--breachwatch --siemincludes at_risk/passed/ignored--outputwrites to file with.ndjsonextension--output, prints to stdout--format jsonoutput unchanged