Files
Termix/src/backend/utils/lazy-field-encryption.ts
Luke Gustafson 8366c99b0f v1.9.0 (#437)
* fix: Resolve database encryption atomicity issues and enhance debugging (#430)

* fix: Resolve database encryption atomicity issues and enhance debugging

This commit addresses critical data corruption issues caused by non-atomic
file writes during database encryption, and adds comprehensive diagnostic
logging to help debug encryption-related failures.

**Problem:**
Users reported "Unsupported state or unable to authenticate data" errors
when starting the application after system crashes or Docker container
restarts. The root cause was non-atomic writes of encrypted database files:

1. Encrypted data file written (step 1)
2. Metadata file written (step 2)
→ If process crashes between steps 1 and 2, files become inconsistent
→ New IV/tag in data file, old IV/tag in metadata
→ GCM authentication fails on next startup
→ User data permanently inaccessible

**Solution - Atomic Writes:**

1. Write-to-temp + atomic-rename pattern:
   - Write to temporary files (*.tmp-timestamp-pid)
   - Perform atomic rename operations
   - Clean up temp files on failure

2. Data integrity validation:
   - Add dataSize field to metadata
   - Verify file size before decryption
   - Early detection of corrupted writes

3. Enhanced error diagnostics:
   - Key fingerprints (SHA256 prefix) for verification
   - File modification timestamps
   - Detailed GCM auth failure messages
   - Automatic diagnostic info generation

**Changes:**

database-file-encryption.ts:
- Implement atomic write pattern in encryptDatabaseFromBuffer
- Implement atomic write pattern in encryptDatabaseFile
- Add dataSize field to EncryptedFileMetadata interface
- Validate file size before decryption in decryptDatabaseToBuffer
- Enhanced error messages for GCM auth failures
- Add getDiagnosticInfo() function for comprehensive debugging
- Add debug logging for all encryption/decryption operations

system-crypto.ts:
- Add detailed logging for DATABASE_KEY initialization
- Log key source (env var vs .env file)
- Add key fingerprints to all log messages
- Better error messages when key loading fails

db/index.ts:
- Automatically generate diagnostic info on decryption failure
- Log detailed debugging information to help users troubleshoot

**Debugging Info Added:**

- Key initialization: source, fingerprint, length, path
- Encryption: original size, encrypted size, IV/tag prefixes, temp paths
- Decryption: file timestamps, metadata content, key fingerprint matching
- Auth failures: .env file status, key availability, file consistency
- File diagnostics: existence, readability, size validation, mtime comparison

**Backward Compatibility:**
- dataSize field is optional (metadata.dataSize?: number)
- Old encrypted files without dataSize continue to work
- No migration required

**Testing:**
- Compiled successfully
- No breaking changes to existing APIs
- Graceful handling of legacy v1 encrypted files

Fixes data loss issues reported by users experiencing container restarts
and system crashes during database saves.

* fix: Cleanup PR

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/backend/utils/database-file-encryption.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: LukeGus <bugattiguy527@gmail.com>
Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: Merge metadata and DB into 1 file

* fix: Add initial command palette

* Feature/german language support (#431)

* Update translation.json

Fixed some translation issues for German, made it more user friendly and common.

* Update translation.json

added updated block for serverStats

* Update translation.json

Added translations

* Update translation.json

Removed duplicate of "free":"Free"

* feat: Finalize command palette

* fix: Several bug fixes for terminals, server stats, and general feature improvements

* feat: Enhanced security, UI improvements, and animations (#432)

* fix: Remove empty catch blocks and add error logging

* refactor: Modularize server stats widget collectors

* feat: Add i18n support for terminal customization and login stats

- Add comprehensive terminal customization translations (60+ keys) for appearance, behavior, and advanced settings across all 4 languages
- Add SSH login statistics translations
- Update HostManagerEditor to use i18n for all terminal customization UI elements
- Update LoginStatsWidget to use i18n for all UI text
- Add missing logger imports in backend files for improved debugging

* feat: Add keyboard shortcut enhancements with Kbd component

- Add shadcn kbd component for displaying keyboard shortcuts
- Enhance file manager context menu to display shortcuts with Kbd component
- Add 5 new keyboard shortcuts to file manager:
  - Ctrl+D: Download selected files
  - Ctrl+N: Create new file
  - Ctrl+Shift+N: Create new folder
  - Ctrl+U: Upload files
  - Enter: Open/run selected file
- Add keyboard shortcut hints to command palette footer
- Create helper function to parse and render keyboard shortcuts

* feat: Add i18n support for command palette

- Add commandPalette translation section with 22 keys to all 4 languages
- Update CommandPalette component to use i18n for all UI text
- Translate search placeholder, group headings, menu items, and shortcut hints
- Support multilingual command palette interface

* feat: Add smooth transitions and animations to UI

- Add fade-in/fade-out transition to command palette (200ms)
- Add scale animation to command palette on open/close
- Add smooth popup animation to context menu (150ms)
- Add visual feedback for file selection with ring effect
- Add hover scale effect to file grid items
- Add transition-all to list view items for consistent behavior
- Zero JavaScript overhead, pure CSS transitions
- All animations under 200ms for instant feel

* feat: Add button active state and dashboard card animations

- Add active:scale-95 to all buttons for tactile click feedback
- Add hover border effect to dashboard cards (150ms transition)
- Add pulse animation to dashboard loading states
- Pure CSS transitions with zero JavaScript overhead
- Improves enterprise-level feel of UI

* feat: Add smooth macOS-style page transitions

- Add fullscreen crossfade transition for login/logout (300ms fade-out + 400ms fade-in)
- Add slide-in-from-right animation for all page switches (Dashboard, Terminal, SSH Manager, Admin, Profile)
- Fix TypeScript compilation by adding esModuleInterop to tsconfig.node.json
- Pass handleLogout from DesktopApp to LeftSidebar for consistent transition behavior

All page transitions now use Tailwind animate-in utilities with 300ms duration for smooth, native-feeling UX

* fix: Add key prop to force animation re-trigger on tab switch

Each page container now has key={currentTab} to ensure React unmounts and remounts the element on every tab switch, properly triggering the slide-in animation

* revert: Remove page transition animations

Page switching animations were not noticeable enough and felt unnecessary.
Keep only the login/logout fullscreen crossfade transitions which provide clear visual feedback for authentication state changes

* feat: Add ripple effect to login/logout transitions

Add three-layer expanding ripple animation during fadeOut phase:
- Ripples expand from screen center using primary theme color
- Each layer has staggered delay (0ms, 150ms, 300ms) for wave effect
- Ripples fade out as they expand to create elegant visual feedback
- Uses pure CSS keyframe animation, no external libraries

Total animation: 800ms ripple + 300ms screen fade

* feat: Add smooth TERMIX logo animation to transitions

Changes:
- Extend transition duration from 300ms/400ms to 800ms/600ms for more elegant feel
- Reduce ripple intensity from /20,/15,/10 to /8,/5 for subtlety
- Slow down ripple animation from 0.8s to 2s with cubic-bezier easing
- Add centered TERMIX logo with monospace font and subtitle
- Logo fades in from 80% scale, holds, then fades out at 110% scale
- Total effect: 1.2s logo animation synced with 2s ripple waves

Creates a premium, branded transition experience

* feat: Enhance transition animation with premium details

Timing adjustments:
- Extend fadeOut from 800ms to 1200ms
- Extend fadeIn from 600ms to 800ms
- Slow background fade to 700ms for elegance

Visual enhancements:
- Add 4-layer ripple waves (10%, 7%, 5%, 3% opacity) with staggered delays
- Ripple animation extended to 2.5s with refined opacity curve
- Logo blur effect: starts at 8px, sharpens to 0px, exits at 4px
- Logo glow effect: triple-layer text-shadow using primary theme color
- Increase logo size from text-6xl to text-7xl
- Subtitle delayed fade-in from bottom with smooth slide animation

Creates a cinematic, polished brand experience

* feat: Redesign login page with split-screen cinematic layout

Major redesign of authentication page:

Left Side (40% width):
- Full-height gradient background using primary theme color
- Large TERMIX logo with glow effect
- Subtitle and tagline
- Infinite animated ripple waves (3 layers)
- Hidden on mobile, shows brand identity

Right Side (60% width):
- Centered glassmorphism card with backdrop blur
- Refined tab switcher with pill-style active state
- Enlarged title with gradient text effect
- Added welcome subtitles for better UX
- Card slides in from bottom on load
- All existing functionality preserved

Visual enhancements:
- Tab navigation: segmented control style in muted container
- Active tab: white background with subtle shadow
- Smooth 200ms transitions on all interactions
- Card: rounded-2xl, shadow-xl, semi-transparent border

Creates premium, modern login experience matching transition animations

* feat: Update login page theme colors and add i18n support

- Changed login page gradient from blue to match dark theme colors
- Updated ripple effects to use theme primary color
- Added i18n translation keys for login page (auth.tagline, auth.description, auth.welcomeBack, auth.createAccount, auth.continueExternal)
- Updated all language files (en, zh, de, ru, pt-BR) with new translations
- Fixed TypeScript compilation issues by clearing build cache

* refactor: Use shadcn Tabs component and fix modal styling

- Replace custom tab navigation with shadcn Tabs component
- Restore border-2 border-dark-border for modal consistency
- Remove circular icon from login success message
- Simplify authentication success display

* refactor: Remove ripple effects and gradient from login page

- Remove animated ripple background effects
- Remove gradient background, use solid color (bg-dark-bg-darker)
- Remove text-shadow glow effect from logo
- Simplify brand showcase to clean, minimal design

* feat: Add decorative slash and remove subtitle from login page

- Add decorative slash divider with gradient lines below TERMIX logo
- Remove subtitle text (welcomeBack and createAccount)
- Simplify page title to show only the main heading

* feat: Add diagonal line pattern background to login page

- Replace decorative slash with subtle diagonal line pattern background
- Use repeating-linear-gradient at 45deg angle
- Set very low opacity (0.03) for subtle effect
- Pattern uses theme primary color

* fix: Display diagonal line pattern on login background

- Combine background color and pattern in single style attribute
- Use white semi-transparent lines (rgba 0.03 opacity)
- 45deg angle, 35px spacing, 2px width
- Remove separate overlay div to ensure pattern visibility

* security: Fix user enumeration vulnerability in login

- Unify error messages for invalid username and incorrect password
- Both return 401 status with 'Invalid username or password'
- Prevent attackers from enumerating valid usernames
- Maintain detailed logging for debugging purposes
- Changed from 404 'User not found' to generic auth failure message

* security: Add login rate limiting to prevent brute force attacks

- Implement LoginRateLimiter with IP and username-based tracking
- Block after 5 failed attempts within 15 minutes
- Lock account/IP for 15 minutes after threshold
- Automatic cleanup of expired entries every 5 minutes
- Track remaining attempts in logs for monitoring
- Return 429 status with remaining time on rate limit
- Reset counters on successful login
- Dual protection: both IP-based and username-based limits

* French translation (#434)

* Adding French Language

* Enhancements

* feat: Replace the old ssh tools system with a new dedicated sidebar

* fix: Merge zac/luke

* fix: Finalize new sidebar, improve and loading animations

* Added ability to close non-primary tabs involved in a split view (#435)

* fix: General bug fixes/small feature improvements

* feat: General UI improvements and translation updates

* fix: Command history and file manager styling issues

* feat: General bug fixes, added server stat commands, improved split screen, link accounts, etc

* fix: add Accept header for OIDC callback request (#436)

* Delete DOWNLOADS.md

* fix: add Accept header for OIDC callback request

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* fix: More bug fixes and QOL fixes

* fix: Server stats not respecting interval and fixed SSH toool type issues

* fix: Remove github links

* fix: Delete account spacing

* fix: Increment version

* fix: Unable to delete hosts and add nginx for terminal

* fix: Unable to delete hosts

* fix: Unable to delete hosts

* fix: Unable to delete hosts

* fix: OIDC/local account linking breaking both logins

* chore: File cleanup

* feat: Max terminal tab size and save current file manager sorting type

* fix: Terminal display issue, migrate host editor to use combobox

* feat: Add snippet folder/customization system

* fix: Fix OIDC linking and prep release

* fix: Increment version

---------

Co-authored-by: ZacharyZcR <zacharyzcr1984@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Max <herzmaximilian@gmail.com>
Co-authored-by: SlimGary <trash.slim@gmail.com>
Co-authored-by: jarrah31 <jarrah31@gmail.com>
Co-authored-by: Kf637 <mail@kf637.tech>
2025-11-17 09:46:05 -06:00

406 lines
10 KiB
TypeScript

import { FieldCrypto } from "./field-crypto.js";
import { databaseLogger } from "./logger.js";
interface DatabaseInstance {
prepare: (sql: string) => {
all: (param?: unknown) => unknown[];
get: (param?: unknown) => unknown;
run: (...params: unknown[]) => unknown;
};
}
export class LazyFieldEncryption {
private static readonly LEGACY_FIELD_NAME_MAP: Record<string, string> = {
key_password: "keyPassword",
private_key: "privateKey",
public_key: "publicKey",
password_hash: "passwordHash",
client_secret: "clientSecret",
totp_secret: "totpSecret",
totp_backup_codes: "totpBackupCodes",
oidc_identifier: "oidcIdentifier",
keyPassword: "key_password",
privateKey: "private_key",
publicKey: "public_key",
passwordHash: "password_hash",
clientSecret: "client_secret",
totpSecret: "totp_secret",
totpBackupCodes: "totp_backup_codes",
oidcIdentifier: "oidc_identifier",
};
static isPlaintextField(value: string): boolean {
if (!value) return false;
try {
const parsed = JSON.parse(value);
if (
parsed &&
typeof parsed === "object" &&
parsed.data &&
parsed.iv &&
parsed.tag &&
parsed.salt &&
parsed.recordId
) {
return false;
}
return true;
} catch {
return true;
}
}
static safeGetFieldValue(
fieldValue: string,
userKEK: Buffer,
recordId: string,
fieldName: string,
): string {
if (!fieldValue) return "";
if (this.isPlaintextField(fieldValue)) {
return fieldValue;
} else {
try {
const decrypted = FieldCrypto.decryptField(
fieldValue,
userKEK,
recordId,
fieldName,
);
return decrypted;
} catch (error) {
const legacyFieldName = this.LEGACY_FIELD_NAME_MAP[fieldName];
if (legacyFieldName) {
try {
const decrypted = FieldCrypto.decryptField(
fieldValue,
userKEK,
recordId,
legacyFieldName,
);
return decrypted;
} catch (error) {}
}
const sensitiveFields = [
"totp_secret",
"totp_backup_codes",
"password",
"key",
"key_password",
"private_key",
"public_key",
"client_secret",
"oidc_identifier",
];
if (sensitiveFields.includes(fieldName)) {
return "";
}
databaseLogger.error("Failed to decrypt field", error, {
operation: "lazy_encryption_decrypt_failed",
recordId,
fieldName,
error: error instanceof Error ? error.message : "Unknown error",
});
throw error;
}
}
}
static migrateFieldToEncrypted(
fieldValue: string,
userKEK: Buffer,
recordId: string,
fieldName: string,
): {
encrypted: string;
wasPlaintext: boolean;
wasLegacyEncryption: boolean;
} {
if (!fieldValue) {
return { encrypted: "", wasPlaintext: false, wasLegacyEncryption: false };
}
if (this.isPlaintextField(fieldValue)) {
try {
const encrypted = FieldCrypto.encryptField(
fieldValue,
userKEK,
recordId,
fieldName,
);
return { encrypted, wasPlaintext: true, wasLegacyEncryption: false };
} catch (error) {
databaseLogger.error("Failed to encrypt plaintext field", error, {
operation: "lazy_encryption_migrate_failed",
recordId,
fieldName,
error: error instanceof Error ? error.message : "Unknown error",
});
throw error;
}
} else {
try {
FieldCrypto.decryptField(fieldValue, userKEK, recordId, fieldName);
return {
encrypted: fieldValue,
wasPlaintext: false,
wasLegacyEncryption: false,
};
} catch {
const legacyFieldName = this.LEGACY_FIELD_NAME_MAP[fieldName];
if (legacyFieldName) {
try {
const decrypted = FieldCrypto.decryptField(
fieldValue,
userKEK,
recordId,
legacyFieldName,
);
const reencrypted = FieldCrypto.encryptField(
decrypted,
userKEK,
recordId,
fieldName,
);
return {
encrypted: reencrypted,
wasPlaintext: false,
wasLegacyEncryption: true,
};
} catch (error) {}
}
return {
encrypted: fieldValue,
wasPlaintext: false,
wasLegacyEncryption: false,
};
}
}
}
static migrateRecordSensitiveFields(
record: Record<string, unknown>,
sensitiveFields: string[],
userKEK: Buffer,
recordId: string,
): {
updatedRecord: Record<string, unknown>;
migratedFields: string[];
needsUpdate: boolean;
} {
const updatedRecord = { ...record };
const migratedFields: string[] = [];
let needsUpdate = false;
for (const fieldName of sensitiveFields) {
const fieldValue = record[fieldName];
if (fieldValue) {
try {
const { encrypted, wasPlaintext, wasLegacyEncryption } =
this.migrateFieldToEncrypted(
fieldValue as string,
userKEK,
recordId,
fieldName,
);
if (wasPlaintext || wasLegacyEncryption) {
updatedRecord[fieldName] = encrypted;
migratedFields.push(fieldName);
needsUpdate = true;
}
} catch (error) {
databaseLogger.error("Failed to migrate record field", error, {
operation: "lazy_encryption_record_field_failed",
recordId,
fieldName,
});
}
}
}
return { updatedRecord, migratedFields, needsUpdate };
}
static getSensitiveFieldsForTable(tableName: string): string[] {
const sensitiveFieldsMap: Record<string, string[]> = {
ssh_data: ["password", "key", "key_password"],
ssh_credentials: [
"password",
"key",
"key_password",
"private_key",
"public_key",
],
users: ["totp_secret", "totp_backup_codes"],
};
return sensitiveFieldsMap[tableName] || [];
}
static fieldNeedsMigration(
fieldValue: string,
userKEK: Buffer,
recordId: string,
fieldName: string,
): boolean {
if (!fieldValue) return false;
if (this.isPlaintextField(fieldValue)) {
return true;
}
try {
FieldCrypto.decryptField(fieldValue, userKEK, recordId, fieldName);
return false;
} catch {
const legacyFieldName = this.LEGACY_FIELD_NAME_MAP[fieldName];
if (legacyFieldName) {
try {
FieldCrypto.decryptField(
fieldValue,
userKEK,
recordId,
legacyFieldName,
);
return true;
} catch {
return false;
}
}
return false;
}
}
static async checkUserNeedsMigration(
userId: string,
userKEK: Buffer,
db: DatabaseInstance,
): Promise<{
needsMigration: boolean;
plaintextFields: Array<{
table: string;
recordId: string;
fields: string[];
}>;
}> {
const plaintextFields: Array<{
table: string;
recordId: string;
fields: string[];
}> = [];
let needsMigration = false;
try {
const sshHosts = db
.prepare("SELECT * FROM ssh_data WHERE user_id = ?")
.all(userId) as Array<
Record<string, unknown> & { id: string | number }
>;
for (const host of sshHosts) {
const sensitiveFields = this.getSensitiveFieldsForTable("ssh_data");
const hostPlaintextFields: string[] = [];
for (const field of sensitiveFields) {
if (
host[field] &&
this.fieldNeedsMigration(
host[field] as string,
userKEK,
host.id.toString(),
field,
)
) {
hostPlaintextFields.push(field);
needsMigration = true;
}
}
if (hostPlaintextFields.length > 0) {
plaintextFields.push({
table: "ssh_data",
recordId: host.id.toString(),
fields: hostPlaintextFields,
});
}
}
const sshCredentials = db
.prepare("SELECT * FROM ssh_credentials WHERE user_id = ?")
.all(userId) as Array<
Record<string, unknown> & { id: string | number }
>;
for (const credential of sshCredentials) {
const sensitiveFields =
this.getSensitiveFieldsForTable("ssh_credentials");
const credentialPlaintextFields: string[] = [];
for (const field of sensitiveFields) {
if (
credential[field] &&
this.fieldNeedsMigration(
credential[field] as string,
userKEK,
credential.id.toString(),
field,
)
) {
credentialPlaintextFields.push(field);
needsMigration = true;
}
}
if (credentialPlaintextFields.length > 0) {
plaintextFields.push({
table: "ssh_credentials",
recordId: credential.id.toString(),
fields: credentialPlaintextFields,
});
}
}
const user = db.prepare("SELECT * FROM users WHERE id = ?").get(userId);
if (user) {
const sensitiveFields = this.getSensitiveFieldsForTable("users");
const userPlaintextFields: string[] = [];
for (const field of sensitiveFields) {
if (
user[field] &&
this.fieldNeedsMigration(user[field], userKEK, userId, field)
) {
userPlaintextFields.push(field);
needsMigration = true;
}
}
if (userPlaintextFields.length > 0) {
plaintextFields.push({
table: "users",
recordId: userId,
fields: userPlaintextFields,
});
}
}
return { needsMigration, plaintextFields };
} catch (error) {
databaseLogger.error("Failed to check user migration needs", error, {
operation: "lazy_encryption_user_check_failed",
userId,
error: error instanceof Error ? error.message : "Unknown error",
});
return { needsMigration: false, plaintextFields: [] };
}
}
}