Proposed changes

This PR implements the complete Template Profile Improvements spec from #5567 via a backward-compatible preprocessing layer that runs before goflags.MergeConfigFile, requiring zero modifications to upstream goflags or any other dependency.

Architecture

Rather than forking or patching goflags to understand new YAML keys (an approach that broke CI/CD in several competing PRs), this implementation introduces a single preprocessing stage in pkg/types/profile.go that intercepts the raw profile YAML, extracts non-standard constructs, and emits a cleaned config that goflags can parse natively. This keeps the change surface minimal and entirely self-contained within Nuclei.

Feature Breakdown

1. Metadata Field Stripping Extra fields (id, name, purpose, description) are silently removed from the config map before serialization. Users can now annotate profiles with human-readable metadata without triggering unknown-flag parse errors.

2. Inline Target Lists When the list key contains a YAML block scalar (multiline string) instead of a file path, the preprocessor materializes the targets into a secure temp file (0600 via os.CreateTemp) and replaces the list value with the temp path. Single-line file paths pass through untouched. Whitespace-only blocks are gracefully discarded.

3. Inline Secrets A secrets: block with static and/or dynamic sub-keys is extracted, serialized into an authx-compatible YAML temp file, and injected into the cleaned config via the secret-file flag. The secrets key is unconditionally removed so goflags never encounters it.

A strict isNonEmptySlice validation guard ensures that nil, empty, or non-list values under static/dynamic are filtered out before file generation. This prevents producing a vacuous authx file that pkg/authprovider/file.go (NewFileAuthProvider) would reject with ErrNoSecrets, which was a startup crash vector that other implementations did not account for.

4. Bulletproof Lifecycle & Cleanup All temporary artifacts are tracked in ProfilePreprocessResult.TempFiles and cleaned up via a sync.Once-guarded Cleanup() method that is both idempotent and concurrency-safe.

Critical cleanup paths fixed in this PR:

• defer profileCleanup() in main() — covers normal return and panic-recovery paths.
• fatalWithCleanup() wrapper — every options.Logger.Fatal() call reachable after temp file creation has been replaced with this helper, which invokes cleanup before os.Exit(1). This is necessary because Go’s os.Exit (and by extension gologger.Fatal) skips all deferred functions.
• Signal handler (SIGINT) — profileCleanup() is explicitly called before os.Exit(1) in both the DAST-server and graceful-shutdown branches, ensuring Ctrl+C never leaves artifacts on disk.
• readFlagsConfig return-not-defer — the preprocessing result from NUCLEI_CONFIG_DIR configs is returned to the caller and accumulated in preprocessResults, not deferred locally (which would delete temp files before the scan reads them).

5. NUCLEI_CONFIG_DIR Parity Custom config directories specified via the NUCLEI_CONFIG_DIR environment variable now go through the identical PreprocessProfileFile pipeline, ensuring inline targets and secrets work regardless of how the config file is loaded.

6. Security Posture All temp files are created with 0600 permissions via os.CreateTemp. No secrets data is written to world-readable paths. This fully satisfies the hardening notes raised by the Neo Security Bot review.

Files Changed

Proof

• CI/CD: All platform checks pass — Ubuntu, macOS, Windows.
• CodeRabbit AI: All actionable findings resolved across 4 review rounds. Zero outstanding issues.
• Neo Security Bot: No security issues found. All hardening notes addressed (secure temp file permissions, sync.Once thread safety, comprehensive cleanup paths).
• Unit Tests: 18 tests cover the full matrix — no-op passthrough, metadata stripping, inline targets (multiline, single-line, empty), inline secrets (static-only, dynamic-only, both, empty, non-map), append-to-existing secret-file, full end-to-end profile, cleanup verification, cleanup idempotency, hasSpecialKeys helper, non-existent file, invalid YAML, and only-extra-fields edge case.

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

/claim #5567

Summary by CodeRabbit

• New Features

  • Preprocesses profile YAMLs to strip extra metadata, inline multiline targets, and extract secrets into temporary authx-formatted files.
  • Exposes preprocessing results to callers so cleaned paths are used throughout merges.

• Bug Fixes

  • Adds a program-wide deferred cleanup flow that removes preprocessing artifacts on normal exit, interrupts, and fatal errors.

• Tests

  • Adds comprehensive tests covering preprocessing scenarios, temp-file handling, and idempotent cleanup.