Summary

Fixes #7724 – Sporadic “Permission denied (publickey,password)” errors caused by stale SSH key files on disk and stale multiplexed connections.

Root Cause

The existing validateSshKey() method only checked whether the key file existed (via a shell ls command) but never verified that its content matched the database. This caused three failure scenarios:

1. Stale key file on disk – When a user updates their SSH key in Coolify, the database is updated but the file on disk may retain old content. The next SSH command uses the wrong key and fails with “Permission denied”.
2. Multiple Horizon workers – In Coolify Cloud with multiple worker servers, each worker has its own filesystem. Worker A updates the key file successfully but Worker B still has the old key file, causing sporadic failures.
3. Changed private_key_id on a server – When a server is switched to use a different SSH key, the multiplexed connection (authenticated with the old key) was not invalidated and would continue to be reused.

Changes

• SshMultiplexingHelper::validateSshKey() – Now uses Storage::disk('ssh-keys') to compare file content against a fresh database read (bypasses Eloquent in-memory cache). On mismatch: re-writes the key file and invalidates all mux connections for servers using that key.
• SshMultiplexingHelper::ensureMultiplexedConnection() – Checks whether the SSH key fingerprint has changed since the mux was established and forces a connection refresh if it has.
• SshMultiplexingHelper::storeConnectionMetadata() – Now also stores the key fingerprint alongside the connection timestamp for future mismatch detection.
• SshMultiplexingHelper::clearConnectionMetadata() – Clears both connection time and fingerprint cache entries.
• Server model saved event – Now invalidates the mux connection when private_key_id is changed on a server.

Files Modified

• app/Helpers/SshMultiplexingHelper.php – Core fix: content validation, fingerprint tracking, mux invalidation
• app/Models/Server.php – Invalidate mux when private_key_id changes
• tests/Unit/SshKeyContentValidationTest.php – Unit tests verifying the structural changes

Security Considerations

• No key content is logged – only UUIDs and fingerprints appear in log entries
• Uses Laravel’s Storage::disk() API instead of shell commands for file operations
• Fresh database reads bypass Eloquent cache to prevent using stale key data

Test Plan

• Unit tests for all structural changes (method existence, content comparison, fingerprint tracking, mux invalidation)
• Backward compatible – connections established before this fix (without cached fingerprint) are not disrupted
• No breaking changes to public API

/claim #7724