* fix: Resolve database encryption atomicity issues and enhance debugging (#430)

* fix: Resolve database encryption atomicity issues and enhance debugging

This commit addresses critical data corruption issues caused by non-atomic
file writes during database encryption, and adds comprehensive diagnostic
logging to help debug encryption-related failures.

**Problem:**
Users reported "Unsupported state or unable to authenticate data" errors
when starting the application after system crashes or Docker container
restarts. The root cause was non-atomic writes of encrypted database files:

1. Encrypted data file written (step 1)
2. Metadata file written (step 2)
→ If process crashes between steps 1 and 2, files become inconsistent
→ New IV/tag in data file, old IV/tag in metadata
→ GCM authentication fails on next startup
→ User data permanently inaccessible

**Solution - Atomic Writes:**

1. Write-to-temp + atomic-rename pattern:
   - Write to temporary files (*.tmp-timestamp-pid)
   - Perform atomic rename operations
   - Clean up temp files on failure

2. Data integrity validation:
   - Add dataSize field to metadata
   - Verify file size before decryption
   - Early detection of corrupted writes

3. Enhanced error diagnostics:
   - Key fingerprints (SHA256 prefix) for verification
   - File modification timestamps
   - Detailed GCM auth failure messages
   - Automatic diagnostic info generation

**Changes:**

database-file-encryption.ts:
- Implement atomic write pattern in encryptDatabaseFromBuffer
- Implement atomic write pattern in encryptDatabaseFile
- Add dataSize field to EncryptedFileMetadata interface
- Validate file size before decryption in decryptDatabaseToBuffer
- Enhanced error messages for GCM auth failures
- Add getDiagnosticInfo() function for comprehensive debugging
- Add debug logging for all encryption/decryption operations

system-crypto.ts:
- Add detailed logging for DATABASE_KEY initialization
- Log key source (env var vs .env file)
- Add key fingerprints to all log messages
- Better error messages when key loading fails

db/index.ts:
- Automatically generate diagnostic info on decryption failure
- Log detailed debugging information to help users troubleshoot

**Debugging Info Added:**

- Key initialization: source, fingerprint, length, path
- Encryption: original size, encrypted size, IV/tag prefixes, temp paths
- Decryption: file timestamps, metadata content, key fingerprint matching
- Auth failures: .env file status, key availability, file consistency
- File diagnostics: existence, readability, size validation, mtime comparison

**Backward Compatibility:**
- dataSize field is optional (metadata.dataSize?: number)
- Old encrypted files without dataSize continue to work
- No migration required

**Testing:**
- Compiled successfully
- No breaking changes to existing APIs
- Graceful handling of legacy v1 encrypted files

Fixes data loss issues reported by users experiencing container restarts
and system crashes during database saves.

* fix: Cleanup PR

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: LukeGus <bugattiguy527@gmail.com>
Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: Merge metadata and DB into 1 file

* fix: Add initial command palette

* Feature/german language support (#431)

* Update translation.json

Fixed some translation issues for German, made it more user friendly and common.

* Update translation.json

added updated block for serverStats

* Update translation.json

Added translations

* Update translation.json

Removed duplicate of "free":"Free"

* feat: Finalize command palette

* fix: Several bug fixes for terminals, server stats, and general feature improvements

* feat: Enhanced security, UI improvements, and animations (#432)

* fix: Remove empty catch blocks and add error logging

* refactor: Modularize server stats widget collectors

* feat: Add i18n support for terminal customization and login stats

- Add comprehensive terminal customization translations (60+ keys) for appearance, behavior, and advanced settings across all 4 languages
- Add SSH login statistics translations
- Update HostManagerEditor to use i18n for all terminal customization UI elements
- Update LoginStatsWidget to use i18n for all UI text
- Add missing logger imports in backend files for improved debugging

* feat: Add keyboard shortcut enhancements with Kbd component

- Add shadcn kbd component for displaying keyboard shortcuts
- Enhance file manager context menu to display shortcuts with Kbd component
- Add 5 new keyboard shortcuts to file manager:
  - Ctrl+D: Download selected files
  - Ctrl+N: Create new file
  - Ctrl+Shift+N: Create new folder
  - Ctrl+U: Upload files
  - Enter: Open/run selected file
- Add keyboard shortcut hints to command palette footer
- Create helper function to parse and render keyboard shortcuts

* feat: Add i18n support for command palette

- Add commandPalette translation section with 22 keys to all 4 languages
- Update CommandPalette component to use i18n for all UI text
- Translate search placeholder, group headings, menu items, and shortcut hints
- Support multilingual command palette interface

* feat: Add smooth transitions and animations to UI

- Add fade-in/fade-out transition to command palette (200ms)
- Add scale animation to command palette on open/close
- Add smooth popup animation to context menu (150ms)
- Add visual feedback for file selection with ring effect
- Add hover scale effect to file grid items
- Add transition-all to list view items for consistent behavior
- Zero JavaScript overhead, pure CSS transitions
- All animations under 200ms for instant feel

* feat: Add button active state and dashboard card animations

- Add active:scale-95 to all buttons for tactile click feedback
- Add hover border effect to dashboard cards (150ms transition)
- Add pulse animation to dashboard loading states
- Pure CSS transitions with zero JavaScript overhead
- Improves enterprise-level feel of UI

* feat: Add smooth macOS-style page transitions

- Add fullscreen crossfade transition for login/logout (300ms fade-out + 400ms fade-in)
- Add slide-in-from-right animation for all page switches (Dashboard, Terminal, SSH Manager, Admin, Profile)
- Fix TypeScript compilation by adding esModuleInterop to tsconfig.node.json
- Pass handleLogout from DesktopApp to LeftSidebar for consistent transition behavior

All page transitions now use Tailwind animate-in utilities with 300ms duration for smooth, native-feeling UX

* fix: Add key prop to force animation re-trigger on tab switch

Each page container now has key={currentTab} to ensure React unmounts and remounts the element on every tab switch, properly triggering the slide-in animation

* revert: Remove page transition animations

Page switching animations were not noticeable enough and felt unnecessary.
Keep only the login/logout fullscreen crossfade transitions which provide clear visual feedback for authentication state changes

* feat: Add ripple effect to login/logout transitions

Add three-layer expanding ripple animation during fadeOut phase:
- Ripples expand from screen center using primary theme color
- Each layer has staggered delay (0ms, 150ms, 300ms) for wave effect
- Ripples fade out as they expand to create elegant visual feedback
- Uses pure CSS keyframe animation, no external libraries

Total animation: 800ms ripple + 300ms screen fade

* feat: Add smooth TERMIX logo animation to transitions

Changes:
- Extend transition duration from 300ms/400ms to 800ms/600ms for more elegant feel
- Reduce ripple intensity from /20,/15,/10 to /8,/5 for subtlety
- Slow down ripple animation from 0.8s to 2s with cubic-bezier easing
- Add centered TERMIX logo with monospace font and subtitle
- Logo fades in from 80% scale, holds, then fades out at 110% scale
- Total effect: 1.2s logo animation synced with 2s ripple waves

Creates a premium, branded transition experience

* feat: Enhance transition animation with premium details

Timing adjustments:
- Extend fadeOut from 800ms to 1200ms
- Extend fadeIn from 600ms to 800ms
- Slow background fade to 700ms for elegance

Visual enhancements:
- Add 4-layer ripple waves (10%, 7%, 5%, 3% opacity) with staggered delays
- Ripple animation extended to 2.5s with refined opacity curve
- Logo blur effect: starts at 8px, sharpens to 0px, exits at 4px
- Logo glow effect: triple-layer text-shadow using primary theme color
- Increase logo size from text-6xl to text-7xl
- Subtitle delayed fade-in from bottom with smooth slide animation

Creates a cinematic, polished brand experience

* feat: Redesign login page with split-screen cinematic layout

Major redesign of authentication page:

Left Side (40% width):
- Full-height gradient background using primary theme color
- Large TERMIX logo with glow effect
- Subtitle and tagline
- Infinite animated ripple waves (3 layers)
- Hidden on mobile, shows brand identity

Right Side (60% width):
- Centered glassmorphism card with backdrop blur
- Refined tab switcher with pill-style active state
- Enlarged title with gradient text effect
- Added welcome subtitles for better UX
- Card slides in from bottom on load
- All existing functionality preserved

Visual enhancements:
- Tab navigation: segmented control style in muted container
- Active tab: white background with subtle shadow
- Smooth 200ms transitions on all interactions
- Card: rounded-2xl, shadow-xl, semi-transparent border

Creates premium, modern login experience matching transition animations

* feat: Update login page theme colors and add i18n support

- Changed login page gradient from blue to match dark theme colors
- Updated ripple effects to use theme primary color
- Added i18n translation keys for login page (auth.tagline, auth.description, auth.welcomeBack, auth.createAccount, auth.continueExternal)
- Updated all language files (en, zh, de, ru, pt-BR) with new translations
- Fixed TypeScript compilation issues by clearing build cache

* refactor: Use shadcn Tabs component and fix modal styling

- Replace custom tab navigation with shadcn Tabs component
- Restore border-2 border-dark-border for modal consistency
- Remove circular icon from login success message
- Simplify authentication success display

* refactor: Remove ripple effects and gradient from login page

- Remove animated ripple background effects
- Remove gradient background, use solid color (bg-dark-bg-darker)
- Remove text-shadow glow effect from logo
- Simplify brand showcase to clean, minimal design

* feat: Add decorative slash and remove subtitle from login page

- Add decorative slash divider with gradient lines below TERMIX logo
- Remove subtitle text (welcomeBack and createAccount)
- Simplify page title to show only the main heading

* feat: Add diagonal line pattern background to login page

- Replace decorative slash with subtle diagonal line pattern background
- Use repeating-linear-gradient at 45deg angle
- Set very low opacity (0.03) for subtle effect
- Pattern uses theme primary color

* fix: Display diagonal line pattern on login background

- Combine background color and pattern in single style attribute
- Use white semi-transparent lines (rgba 0.03 opacity)
- 45deg angle, 35px spacing, 2px width
- Remove separate overlay div to ensure pattern visibility

* security: Fix user enumeration vulnerability in login

- Unify error messages for invalid username and incorrect password
- Both return 401 status with 'Invalid username or password'
- Prevent attackers from enumerating valid usernames
- Maintain detailed logging for debugging purposes
- Changed from 404 'User not found' to generic auth failure message

* security: Add login rate limiting to prevent brute force attacks

- Implement LoginRateLimiter with IP and username-based tracking
- Block after 5 failed attempts within 15 minutes
- Lock account/IP for 15 minutes after threshold
- Automatic cleanup of expired entries every 5 minutes
- Track remaining attempts in logs for monitoring
- Return 429 status with remaining time on rate limit
- Reset counters on successful login
- Dual protection: both IP-based and username-based limits

* French translation (#434)

* Adding French Language

* Enhancements

* feat: Replace the old ssh tools system with a new dedicated sidebar

* fix: Merge zac/luke

* fix: Finalize new sidebar, improve and loading animations

* Added ability to close non-primary tabs involved in a split view (#435)

* fix: General bug fixes/small feature improvements

* feat: General UI improvements and translation updates

* fix: Command history and file manager styling issues

* feat: General bug fixes, added server stat commands, improved split screen, link accounts, etc

* fix: add Accept header for OIDC callback request (#436)

* Delete DOWNLOADS.md

* fix: add Accept header for OIDC callback request

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* fix: More bug fixes and QOL fixes

* fix: Server stats not respecting interval and fixed SSH toool type issues

* fix: Remove github links

* fix: Delete account spacing

* fix: Increment version

* fix: Unable to delete hosts and add nginx for terminal

* fix: Unable to delete hosts

* fix: Unable to delete hosts

* fix: Unable to delete hosts

* fix: OIDC/local account linking breaking both logins

* chore: File cleanup

* feat: Max terminal tab size and save current file manager sorting type

* fix: Terminal display issue, migrate host editor to use combobox

* feat: Add snippet folder/customization system

* fix: Fix OIDC linking and prep release

* fix: Increment version

---------

Co-authored-by: ZacharyZcR <zacharyzcr1984@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Max <herzmaximilian@gmail.com>
Co-authored-by: SlimGary <trash.slim@gmail.com>
Co-authored-by: jarrah31 <jarrah31@gmail.com>
Co-authored-by: Kf637 <mail@kf637.tech>
This commit was merged in pull request #437.
This commit is contained in:
Luke Gustafson
2025-11-17 09:46:05 -06:00
committed by GitHub
parent 38a59f3579
commit 8366c99b0f
104 changed files with 16070 additions and 2821 deletions

View File

@@ -31,6 +31,7 @@ interface ConnectToHostData {
credentialId?: number;
userId?: string;
forceKeyboardInteractive?: boolean;
jumpHosts?: Array<{ hostId: number }>;
};
initialPath?: string;
executeCommand?: string;
@@ -57,6 +58,173 @@ const userCrypto = UserCrypto.getInstance();
const userConnections = new Map<string, Set<WebSocket>>();
async function resolveJumpHost(
hostId: number,
userId: string,
): Promise<any | null> {
try {
const hosts = await SimpleDBOps.select(
getDb()
.select()
.from(sshData)
.where(and(eq(sshData.id, hostId), eq(sshData.userId, userId))),
"ssh_data",
userId,
);
if (hosts.length === 0) {
return null;
}
const host = hosts[0];
if (host.credentialId) {
const credentials = await SimpleDBOps.select(
getDb()
.select()
.from(sshCredentials)
.where(
and(
eq(sshCredentials.id, host.credentialId as number),
eq(sshCredentials.userId, userId),
),
),
"ssh_credentials",
userId,
);
if (credentials.length > 0) {
const credential = credentials[0];
return {
...host,
password: credential.password,
key:
credential.private_key || credential.privateKey || credential.key,
keyPassword: credential.key_password || credential.keyPassword,
keyType: credential.key_type || credential.keyType,
authType: credential.auth_type || credential.authType,
};
}
}
return host;
} catch (error) {
sshLogger.error("Failed to resolve jump host", error, {
operation: "resolve_jump_host",
hostId,
userId,
});
return null;
}
}
async function createJumpHostChain(
jumpHosts: Array<{ hostId: number }>,
userId: string,
): Promise<Client | null> {
if (!jumpHosts || jumpHosts.length === 0) {
return null;
}
let currentClient: Client | null = null;
const clients: Client[] = [];
try {
for (let i = 0; i < jumpHosts.length; i++) {
const jumpHostConfig = await resolveJumpHost(jumpHosts[i].hostId, userId);
if (!jumpHostConfig) {
sshLogger.error(`Jump host ${i + 1} not found`, undefined, {
operation: "jump_host_chain",
hostId: jumpHosts[i].hostId,
});
clients.forEach((c) => c.end());
return null;
}
const jumpClient = new Client();
clients.push(jumpClient);
const connected = await new Promise<boolean>((resolve) => {
const timeout = setTimeout(() => {
resolve(false);
}, 30000);
jumpClient.on("ready", () => {
clearTimeout(timeout);
resolve(true);
});
jumpClient.on("error", (err) => {
clearTimeout(timeout);
sshLogger.error(`Jump host ${i + 1} connection failed`, err, {
operation: "jump_host_connect",
hostId: jumpHostConfig.id,
ip: jumpHostConfig.ip,
});
resolve(false);
});
const connectConfig: any = {
host: jumpHostConfig.ip,
port: jumpHostConfig.port || 22,
username: jumpHostConfig.username,
tryKeyboard: true,
readyTimeout: 30000,
};
if (jumpHostConfig.authType === "password" && jumpHostConfig.password) {
connectConfig.password = jumpHostConfig.password;
} else if (jumpHostConfig.authType === "key" && jumpHostConfig.key) {
const cleanKey = jumpHostConfig.key
.trim()
.replace(/\r\n/g, "\n")
.replace(/\r/g, "\n");
connectConfig.privateKey = Buffer.from(cleanKey, "utf8");
if (jumpHostConfig.keyPassword) {
connectConfig.passphrase = jumpHostConfig.keyPassword;
}
}
if (currentClient) {
currentClient.forwardOut(
"127.0.0.1",
0,
jumpHostConfig.ip,
jumpHostConfig.port || 22,
(err, stream) => {
if (err) {
clearTimeout(timeout);
resolve(false);
return;
}
connectConfig.sock = stream;
jumpClient.connect(connectConfig);
},
);
} else {
jumpClient.connect(connectConfig);
}
});
if (!connected) {
clients.forEach((c) => c.end());
return null;
}
currentClient = jumpClient;
}
return currentClient;
} catch (error) {
sshLogger.error("Failed to create jump host chain", error, {
operation: "jump_host_chain",
});
clients.forEach((c) => c.end());
return null;
}
}
const wss = new WebSocketServer({
port: 30002,
verifyClient: async (info) => {
@@ -79,6 +247,7 @@ const wss = new WebSocketServer({
}
const existingConnections = userConnections.get(payload.userId);
if (existingConnections && existingConnections.size >= 3) {
return false;
}
@@ -152,6 +321,10 @@ wss.on("connection", async (ws: WebSocket, req) => {
let totpPromptSent = false;
let isKeyboardInteractive = false;
let keyboardInteractiveResponded = false;
let isConnecting = false;
let isConnected = false;
let isCleaningUp = false;
let isShellInitializing = false;
ws.on("close", () => {
const userWs = userConnections.get(userId);
@@ -417,10 +590,21 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
if (isConnecting || isConnected) {
sshLogger.warn("Connection already in progress or established", {
operation: "ssh_connect",
hostId: id,
isConnecting,
isConnected,
});
return;
}
isConnecting = true;
sshConn = new Client();
const connectionTimeout = setTimeout(() => {
if (sshConn) {
if (sshConn && isConnecting && !isConnected) {
sshLogger.error("SSH connection timeout", undefined, {
operation: "ssh_connect",
hostId: id,
@@ -433,7 +617,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
);
cleanupSSH(connectionTimeout);
}
}, 60000);
}, 120000);
let resolvedCredentials = { password, key, keyPassword, keyType, authType };
let authMethodNotAvailable = false;
@@ -498,7 +682,9 @@ wss.on("connection", async (ws: WebSocket, req) => {
sshConn.on("ready", () => {
clearTimeout(connectionTimeout);
if (!sshConn) {
const conn = sshConn;
if (!conn || isCleaningUp || !sshConn) {
sshLogger.warn(
"SSH connection was cleaned up before shell could be created",
{
@@ -507,6 +693,9 @@ wss.on("connection", async (ws: WebSocket, req) => {
ip,
port,
username,
isCleaningUp,
connNull: !conn,
sshConnNull: !sshConn,
},
);
ws.send(
@@ -519,13 +708,37 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
sshConn.shell(
isShellInitializing = true;
isConnecting = false;
isConnected = true;
if (!sshConn) {
sshLogger.error(
"SSH connection became null right before shell creation",
{
operation: "ssh_shell",
hostId: id,
},
);
ws.send(
JSON.stringify({
type: "error",
message: "SSH connection lost during setup",
}),
);
isShellInitializing = false;
return;
}
conn.shell(
{
rows: data.rows,
cols: data.cols,
term: "xterm-256color",
} as PseudoTtyOptions,
(err, stream) => {
isShellInitializing = false;
if (err) {
sshLogger.error("Shell error", err, {
operation: "ssh_shell",
@@ -836,9 +1049,10 @@ wss.on("connection", async (ws: WebSocket, req) => {
tryKeyboard: true,
keepaliveInterval: 30000,
keepaliveCountMax: 3,
readyTimeout: 60000,
readyTimeout: 120000,
tcpKeepAlive: true,
tcpKeepAliveInitialDelay: 30000,
timeout: 120000,
env: {
TERM: "xterm-256color",
LANG: "en_US.UTF-8",
@@ -969,7 +1183,68 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
sshConn.connect(connectConfig);
if (
hostConfig.jumpHosts &&
hostConfig.jumpHosts.length > 0 &&
hostConfig.userId
) {
try {
const jumpClient = await createJumpHostChain(
hostConfig.jumpHosts,
hostConfig.userId,
);
if (!jumpClient) {
sshLogger.error("Failed to establish jump host chain");
ws.send(
JSON.stringify({
type: "error",
message: "Failed to connect through jump hosts",
}),
);
cleanupSSH(connectionTimeout);
return;
}
jumpClient.forwardOut("127.0.0.1", 0, ip, port, (err, stream) => {
if (err) {
sshLogger.error("Failed to forward through jump host", err, {
operation: "ssh_jump_forward",
hostId: id,
ip,
port,
});
ws.send(
JSON.stringify({
type: "error",
message: "Failed to forward through jump host: " + err.message,
}),
);
jumpClient.end();
cleanupSSH(connectionTimeout);
return;
}
connectConfig.sock = stream;
sshConn.connect(connectConfig);
});
} catch (error) {
sshLogger.error("Jump host error", error, {
operation: "ssh_jump_host",
hostId: id,
});
ws.send(
JSON.stringify({
type: "error",
message: "Failed to connect through jump hosts",
}),
);
cleanupSSH(connectionTimeout);
return;
}
} else {
sshConn.connect(connectConfig);
}
}
function handleResize(data: ResizeData) {
@@ -982,6 +1257,24 @@ wss.on("connection", async (ws: WebSocket, req) => {
}
function cleanupSSH(timeoutId?: NodeJS.Timeout) {
if (isCleaningUp) {
return;
}
if (isShellInitializing) {
sshLogger.warn(
"Cleanup attempted during shell initialization, deferring",
{
operation: "cleanup_deferred",
userId,
},
);
setTimeout(() => cleanupSSH(timeoutId), 100);
return;
}
isCleaningUp = true;
if (timeoutId) {
clearTimeout(timeoutId);
}
@@ -1019,6 +1312,12 @@ wss.on("connection", async (ws: WebSocket, req) => {
isKeyboardInteractive = false;
keyboardInteractiveResponded = false;
keyboardInteractiveFinish = null;
isConnecting = false;
isConnected = false;
setTimeout(() => {
isCleaningUp = false;
}, 100);
}
function setupPingInterval() {
@@ -1033,7 +1332,12 @@ wss.on("connection", async (ws: WebSocket, req) => {
);
cleanupSSH();
}
} else if (!sshConn || !sshStream) {
if (pingInterval) {
clearInterval(pingInterval);
pingInterval = null;
}
}
}, 60000);
}, 30000);
}
});