Fix mobile UI and SSL

This commit is contained in:
LukeGus
2025-09-25 22:56:45 -05:00
parent 158e805e04
commit 9b1dbdcc0a
12 changed files with 282 additions and 4251 deletions

View File

@@ -1,323 +0,0 @@
# Database Migration Testing Guide
## Overview
This document outlines the testing procedures for the automatic database migration system that migrates unencrypted SQLite databases to encrypted format during Docker deployment updates.
## Migration System Features
**Automatic Detection**: Detects unencrypted databases on startup
**Safe Backup**: Creates timestamped backups before migration
**Integrity Verification**: Validates migration completeness
**Non-destructive**: Original files are renamed, not deleted
**Cleanup**: Removes old backup files (keeps latest 3)
**Admin API**: Migration status and history endpoints
**Detailed Logging**: Comprehensive migration logs
## Test Scenarios
### Scenario 1: Fresh Installation (No Migration Needed)
**Setup**: Clean Docker container with no existing database files
**Expected**:
- New encrypted database created
- No migration messages in logs
- Status API shows "Fresh installation detected"
**Test Commands**:
```bash
# Clean start
docker run --rm termix:latest
# Check logs for "fresh installation"
# GET /database/migration/status should show needsMigration: false
```
### Scenario 2: Standard Migration (Unencrypted → Encrypted)
**Setup**: Existing unencrypted `db.sqlite` file with user data
**Expected**:
- Automatic migration on startup
- Backup file created (`.migration-backup-{timestamp}`)
- Original file renamed (`.migrated-{timestamp}`)
- Encrypted database created successfully
- All data preserved and accessible
**Test Commands**:
```bash
# 1. Create test data in unencrypted format
docker run -v /host/data:/app/data termix:old-version
# Add some SSH hosts and credentials via UI
# 2. Stop container and update to new version
docker stop container_id
docker run -v /host/data:/app/data termix:latest
# 3. Check migration logs
docker logs container_id | grep -i migration
# 4. Verify data integrity via API
curl -H "Authorization: Bearer $TOKEN" http://localhost:8081/database/migration/status
```
### Scenario 3: Already Encrypted (No Migration Needed)
**Setup**: Only encrypted database file exists
**Expected**:
- No migration performed
- Database loads normally
- Status API shows "Only encrypted database exists"
**Test Commands**:
```bash
# Start with existing encrypted database
docker run -v /host/encrypted-data:/app/data termix:latest
# Verify no migration messages in logs
```
### Scenario 4: Both Files Exist (Safety Mode)
**Setup**: Both encrypted and unencrypted databases present
**Expected**:
- Migration skipped for safety
- Warning logged about manual intervention
- Both files preserved
- Uses encrypted database
**Test Commands**:
```bash
# Manually create both files
touch /host/data/db.sqlite
touch /host/data/db.sqlite.encrypted
docker run -v /host/data:/app/data termix:latest
# Check for safety warning in logs
```
### Scenario 5: Migration Failure Recovery
**Setup**: Simulate migration failure (corrupted source file)
**Expected**:
- Migration fails safely
- Backup file preserved
- Original unencrypted file untouched
- Clear error message with recovery instructions
**Test Commands**:
```bash
# Create corrupted database file
echo "corrupted" > /host/data/db.sqlite
docker run -v /host/data:/app/data termix:latest
# Verify error handling and backup preservation
```
### Scenario 6: Large Database Migration
**Setup**: Large unencrypted database (>100MB with many records)
**Expected**:
- Migration completes successfully
- Performance is acceptable (under 30 seconds)
- Memory usage stays reasonable
- All data integrity checks pass
**Test Commands**:
```bash
# Create large dataset first
# Monitor migration duration and memory usage
docker stats container_id
```
## API Testing
### Migration Status Endpoint
```bash
# Admin access required
curl -H "Authorization: Bearer $ADMIN_TOKEN" \
http://localhost:8081/database/migration/status
# Expected response:
{
"migrationStatus": {
"needsMigration": false,
"hasUnencryptedDb": false,
"hasEncryptedDb": true,
"unencryptedDbSize": 0,
"reason": "Only encrypted database exists. No migration needed."
},
"files": {
"unencryptedDbSize": 0,
"encryptedDbSize": 524288,
"backupFiles": 2,
"migratedFiles": 1
},
"recommendations": [
"Database is properly encrypted",
"No action required"
]
}
```
### Migration History Endpoint
```bash
curl -H "Authorization: Bearer $ADMIN_TOKEN" \
http://localhost:8081/database/migration/history
# Expected response:
{
"files": [
{
"name": "db.sqlite.migration-backup-2024-09-24T10-30-00-000Z",
"size": 262144,
"created": "2024-09-24T10:30:00.000Z",
"modified": "2024-09-24T10:30:00.000Z",
"type": "backup"
}
],
"summary": {
"totalBackups": 1,
"totalMigrated": 1,
"oldestBackup": "2024-09-24T10:30:00.000Z",
"newestBackup": "2024-09-24T10:30:00.000Z"
}
}
```
## Log Analysis
### Successful Migration Logs
Look for these log entries:
```
[INFO] Migration status check completed - needsMigration: true
[INFO] Starting automatic database migration
[INFO] Creating migration backup
[SUCCESS] Migration backup created successfully
[INFO] Found tables to migrate - tableCount: 8
[SUCCESS] Migration integrity verification completed
[INFO] Creating encrypted database file
[SUCCESS] Database migration completed successfully
```
### Migration Skipped (Safety) Logs
```
[INFO] Migration status check completed - needsMigration: false
[INFO] Both encrypted and unencrypted databases exist. Skipping migration for safety
[WARN] Manual intervention may be required
```
### Migration Failure Logs
```
[ERROR] Database migration failed
[ERROR] Backup available at: /app/data/db.sqlite.migration-backup-{timestamp}
[ERROR] Manual intervention required to recover data
```
## Manual Recovery Procedures
### If Migration Fails:
1. **Locate backup file**: `db.sqlite.migration-backup-{timestamp}`
2. **Restore original**: `cp backup-file db.sqlite`
3. **Check logs**: Look for specific error details
4. **Fix issue**: Address the root cause (permissions, disk space, etc.)
5. **Retry**: Restart container to trigger migration again
### If Both Databases Exist:
1. **Check dates**: Determine which file is newer
2. **Backup both**: Make copies before proceeding
3. **Remove older**: Delete the outdated database file
4. **Restart**: Container will detect single database
### Emergency Data Recovery:
1. **Backup files are SQLite**: Can be opened with any SQLite client
2. **Manual export**: Use SQLite tools to export data
3. **Re-import**: Use Termix import functionality
## Performance Expectations
| Database Size | Expected Migration Time | Memory Usage |
|---------------|------------------------|--------------|
| < 10MB | < 5 seconds | < 50MB |
| 10-50MB | 5-15 seconds | < 100MB |
| 50-200MB | 15-45 seconds | < 200MB |
| 200MB+ | 45+ seconds | < 500MB |
## Validation Checklist
After migration, verify:
- [ ] All SSH hosts are accessible
- [ ] SSH credentials work correctly
- [ ] File manager recent/pinned items preserved
- [ ] User settings maintained
- [ ] OIDC configuration intact
- [ ] Admin users still have admin privileges
- [ ] Backup file exists and is valid SQLite
- [ ] Original file renamed (not deleted)
- [ ] Encrypted file is properly encrypted
- [ ] Migration APIs respond correctly
## Monitoring Commands
```bash
# Watch migration in real-time
docker logs -f container_id | grep -i migration
# Check file sizes before/after
ls -la /host/data/db.sqlite*
# Verify encrypted file
file /host/data/db.sqlite.encrypted
# Monitor system resources during migration
docker stats container_id
# Test database connectivity after migration
curl -H "Authorization: Bearer $TOKEN" \
http://localhost:8081/hosts/list
```
## Common Issues & Solutions
### Issue: "Permission denied" during backup creation
**Solution**: Check container file permissions and volume mounts
### Issue: "Insufficient disk space" during migration
**Solution**: Free up space, migration requires 2x database size temporarily
### Issue: "Database locked" error
**Solution**: Ensure no other processes are accessing the database file
### Issue: Migration hangs indefinitely
**Solution**: Check for very large BLOB data, increase timeout or migrate manually
### Issue: Encrypted file fails validation
**Solution**: Check DATABASE_KEY environment variable, ensure it's stable
## Security Considerations
- **Backup files contain unencrypted data**: Secure backup file access
- **Migration logs may contain sensitive info**: Review log retention policies
- **Temporary files during migration**: Ensure secure temp directory
- **Original files are preserved**: Plan for secure cleanup of old files
- **Admin API access**: Ensure proper authentication and authorization
## Integration with CI/CD
For automated testing in CI/CD pipelines:
```bash
#!/bin/bash
# Migration integration test
set -e
# Start with unencrypted test data
docker run -d --name test-migration \
-v ./test-data:/app/data \
termix:latest
# Wait for startup
sleep 30
# Check migration status
RESPONSE=$(curl -s -H "Authorization: Bearer $TEST_TOKEN" \
http://localhost:8081/database/migration/status)
# Validate migration success
echo "$RESPONSE" | jq '.migrationStatus.needsMigration == false'
# Cleanup
docker stop test-migration
docker rm test-migration
```
This comprehensive testing approach ensures the migration system handles all edge cases safely and provides administrators with full visibility into the migration process.

View File

@@ -28,9 +28,6 @@ JWT_SECRET=
DATABASE_KEY=
INTERNAL_AUTH_TOKEN=
# ===== DATABASE CONFIGURATION =====
DATABASE_ENCRYPTION=true
# ===== CORS CONFIGURATION =====
ALLOWED_ORIGINS=*

View File

@@ -59,7 +59,7 @@ ENV DATA_DIR=/app/data \
PORT=8080 \
NODE_ENV=production
RUN apk add --no-cache nginx gettext su-exec && \
RUN apk add --no-cache nginx gettext su-exec openssl && \
mkdir -p /app/data && \
chown -R node:node /app/data

View File

@@ -43,9 +43,6 @@ services:
- DATABASE_KEY=${DATABASE_KEY:-}
- INTERNAL_AUTH_TOKEN=${INTERNAL_AUTH_TOKEN:-}
# Database configuration
- DATABASE_ENCRYPTION=${DATABASE_ENCRYPTION:-true}
# CORS configuration
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*}

View File

@@ -138,7 +138,7 @@ ipcMain.handle("test-server-connection", async (event, serverUrl) => {
try {
let fetch;
try {
fetch = globalThis.fetch || require("node:fetch");
fetch = globalThis.fetch || require("node-fetch");
} catch (e) {
const https = require("https");
const http = require("http");

4041
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -27,14 +27,14 @@
},
"dependencies": {
"@codemirror/autocomplete": "^6.18.7",
"@codemirror/comment": "^0.19.1",
"@codemirror/commands": "^6.3.3",
"@codemirror/search": "^6.5.11",
"@codemirror/theme-one-dark": "^6.1.2",
"@codemirror/view": "^6.23.1",
"@hookform/resolvers": "^5.1.1",
"@monaco-editor/react": "^4.7.0",
"@radix-ui/react-accordion": "^1.2.11",
"@radix-ui/react-avatar": "^1.1.10",
"@radix-ui/react-checkbox": "^1.3.2",
"@radix-ui/react-collapsible": "^1.1.11",
"@radix-ui/react-dialog": "^1.1.15",
"@radix-ui/react-dropdown-menu": "^2.1.15",
"@radix-ui/react-label": "^2.1.7",
@@ -43,7 +43,6 @@
"@radix-ui/react-scroll-area": "^1.2.9",
"@radix-ui/react-select": "^2.2.5",
"@radix-ui/react-separator": "^1.1.7",
"@radix-ui/react-slider": "^1.3.5",
"@radix-ui/react-slot": "^1.2.3",
"@radix-ui/react-switch": "^1.2.5",
"@radix-ui/react-tabs": "^1.1.12",
@@ -54,31 +53,26 @@
"@types/multer": "^2.0.0",
"@types/qrcode": "^1.5.5",
"@types/speakeasy": "^2.0.10",
"@uiw/codemirror-extensions-hyper-link": "^4.24.1",
"@uiw/codemirror-extensions-langs": "^4.24.1",
"@uiw/codemirror-themes": "^4.24.1",
"@uiw/react-codemirror": "^4.24.1",
"@xterm/addon-attach": "^0.11.0",
"@xterm/addon-clipboard": "^0.1.0",
"@xterm/addon-fit": "^0.10.0",
"@xterm/addon-search": "^0.15.0",
"@xterm/addon-unicode11": "^0.8.0",
"@xterm/addon-web-links": "^0.11.0",
"@xterm/xterm": "^5.5.0",
"axios": "^1.10.0",
"bcryptjs": "^3.0.2",
"better-sqlite3": "^12.2.0",
"body-parser": "^1.20.2",
"chalk": "^4.1.2",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"dotenv": "^17.2.0",
"drizzle-orm": "^0.44.3",
"express": "^5.1.0",
"i18next": "^25.4.2",
"i18next-browser-languagedetector": "^8.2.0",
"i18next-http-backend": "^3.0.2",
"jose": "^5.2.3",
"jsonwebtoken": "^9.0.2",
"jszip": "^3.10.1",
@@ -87,7 +81,6 @@
"nanoid": "^5.1.5",
"next-themes": "^0.4.6",
"node-fetch": "^3.3.2",
"pdfjs-dist": "^5.4.149",
"qrcode": "^1.5.4",
"react": "^19.1.0",
"react-dom": "^19.1.0",
@@ -108,8 +101,6 @@
"speakeasy": "^2.0.0",
"ssh2": "^1.16.0",
"tailwind-merge": "^3.3.1",
"tailwindcss": "^4.1.11",
"validator": "^13.15.15",
"ws": "^8.18.3",
"zod": "^4.0.5"
},
@@ -125,23 +116,16 @@
"@types/ssh2": "^1.15.5",
"@types/ws": "^8.18.1",
"@vitejs/plugin-react-swc": "^3.10.2",
"autoprefixer": "^10.4.21",
"concurrently": "^9.2.1",
"cross-env": "^10.0.0",
"electron": "^38.0.0",
"electron-builder": "^26.0.12",
"electron-icon-builder": "^2.0.1",
"electron-packager": "^17.1.2",
"eslint": "^9.34.0",
"eslint-plugin-react-hooks": "^5.2.0",
"eslint-plugin-react-refresh": "^0.4.20",
"globals": "^16.3.0",
"prettier": "3.6.2",
"ts-node": "^10.9.2",
"tw-animate-css": "^1.3.5",
"typescript": "~5.9.2",
"typescript-eslint": "^8.40.0",
"vite": "^7.1.5",
"wait-on": "^8.0.4"
"vite": "^7.1.5"
}
}
}

View File

@@ -92,9 +92,6 @@ NODE_ENV=production
# CORS configuration
ALLOWED_ORIGINS=*
# Database encryption
DATABASE_ENCRYPTION=true
EOF
# Add security keys

View File

@@ -33,6 +33,9 @@ export class AutoSSLSetup {
systemLogger.info("SSL configuration already exists and is valid", {
operation: "ssl_already_configured"
});
// Log certificate information for existing certificates
await this.logCertificateInfo();
return;
}
@@ -70,12 +73,29 @@ export class AutoSSLSetup {
await fs.access(this.KEY_FILE);
// Check if certificate is still valid (at least 30 days)
const result = execSync(`openssl x509 -in "${this.CERT_FILE}" -checkend 2592000 -noout`, {
execSync(`openssl x509 -in "${this.CERT_FILE}" -checkend 2592000 -noout`, {
stdio: 'pipe'
});
systemLogger.info("SSL certificate is valid and will expire in more than 30 days", {
operation: "ssl_cert_check",
cert_path: this.CERT_FILE
});
return true;
} catch {
} catch (error) {
if (error instanceof Error && error.message.includes('checkend')) {
systemLogger.warn("SSL certificate is expired or expiring soon, will regenerate", {
operation: "ssl_cert_expired",
cert_path: this.CERT_FILE,
error: error.message
});
} else {
systemLogger.info("SSL certificate not found or invalid, will generate new one", {
operation: "ssl_cert_missing",
cert_path: this.CERT_FILE
});
}
return false;
}
}
@@ -89,6 +109,13 @@ export class AutoSSLSetup {
});
try {
// Check if OpenSSL is available
try {
execSync('openssl version', { stdio: 'pipe' });
} catch (error) {
throw new Error('OpenSSL is not installed or not available in PATH. Please install OpenSSL to enable SSL certificate generation.');
}
// Create SSL directory
await fs.mkdir(this.SSL_DIR, { recursive: true });
@@ -149,11 +176,40 @@ IP.2 = ::1
valid_days: 365
});
// Log certificate information
await this.logCertificateInfo();
} catch (error) {
throw new Error(`SSL certificate generation failed: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
}
/**
* Log certificate information including expiration date
*/
private static async logCertificateInfo(): Promise<void> {
try {
const subject = execSync(`openssl x509 -in "${this.CERT_FILE}" -noout -subject`, { stdio: 'pipe' }).toString().trim();
const issuer = execSync(`openssl x509 -in "${this.CERT_FILE}" -noout -issuer`, { stdio: 'pipe' }).toString().trim();
const notAfter = execSync(`openssl x509 -in "${this.CERT_FILE}" -noout -enddate`, { stdio: 'pipe' }).toString().trim();
const notBefore = execSync(`openssl x509 -in "${this.CERT_FILE}" -noout -startdate`, { stdio: 'pipe' }).toString().trim();
systemLogger.info("SSL Certificate Information:", {
operation: "ssl_cert_info",
subject: subject.replace('subject=', ''),
issuer: issuer.replace('issuer=', ''),
valid_from: notBefore.replace('notBefore=', ''),
valid_until: notAfter.replace('notAfter=', ''),
note: "Certificate will auto-renew 30 days before expiration"
});
} catch (error) {
systemLogger.warn("Could not retrieve certificate information", {
operation: "ssl_cert_info_error",
error: error instanceof Error ? error.message : 'Unknown error'
});
}
}
/**
* Setup environment variables for SSL configuration
*/

View File

@@ -1,5 +1,4 @@
@import "tailwindcss";
@import "tw-animate-css";
@custom-variant dark (&:is(.dark *));

View File

@@ -160,7 +160,7 @@ export function TerminalKeyboard({
more: [
"{esc} {tab} {ctrl} {alt} {end} {home} {pgUp} {pgDn}",
"1 2 3 4 5 6 7 8 9 0",
"! @ # $ % ^ & * ( ) _ +",
"! @ # $ % ^ & * ( ) _ + -",
"[ ] { } | \\ ; : ' \" , . / < >",
"F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12",
"{arrowLeft} {arrowRight} {arrowUp} {arrowDown} {backspace}",

View File

@@ -2,6 +2,7 @@ import {
Sidebar,
SidebarContent,
SidebarFooter,
SidebarGroup,
SidebarGroupLabel,
SidebarHeader,
SidebarMenu,
@@ -180,41 +181,43 @@ export function LeftSidebar({
</SidebarGroupLabel>
</SidebarHeader>
<Separator />
<SidebarContent className="px-2 py-2">
<div className="!bg-dark-bg-input rounded-lg">
<Input
value={search}
onChange={(e) => setSearch(e.target.value)}
placeholder={t("placeholders.searchHostsAny")}
className="w-full h-8 text-sm border-2 !bg-dark-bg-input border-dark-border rounded-md"
autoComplete="off"
/>
</div>
{hostsError && (
<div className="px-1">
<div className="text-xs text-red-500 bg-red-500/10 rounded-lg px-2 py-1 border w-full">
{t("leftSidebar.failedToLoadHosts")}
</div>
<SidebarContent>
<SidebarGroup className="flex flex-col gap-y-2">
<div className="!bg-dark-bg-input rounded-lg">
<Input
value={search}
onChange={(e) => setSearch(e.target.value)}
placeholder={t("placeholders.searchHostsAny")}
className="w-full h-8 text-sm border-2 !bg-dark-bg-input border-dark-border rounded-md"
autoComplete="off"
/>
</div>
)}
{hostsLoading && (
<div className="px-4 pb-2">
<div className="text-xs text-muted-foreground text-center">
{t("hosts.loadingHosts")}
{hostsError && (
<div className="px-1">
<div className="text-xs text-red-500 bg-red-500/10 rounded-lg px-2 py-1 border w-full">
{t("leftSidebar.failedToLoadHosts")}
</div>
</div>
</div>
)}
)}
{sortedFolders.map((folder) => (
<FolderCard
key={`folder-${folder}`}
folderName={folder}
hosts={getSortedHosts(hostsByFolder[folder])}
onHostConnect={onHostConnect}
/>
))}
{hostsLoading && (
<div className="px-4 pb-2">
<div className="text-xs text-muted-foreground text-center">
{t("hosts.loadingHosts")}
</div>
</div>
)}
{sortedFolders.map((folder) => (
<FolderCard
key={`folder-${folder}`}
folderName={folder}
hosts={getSortedHosts(hostsByFolder[folder])}
onHostConnect={onHostConnect}
/>
))}
</SidebarGroup>
</SidebarContent>
<Separator className="mt-1" />
<SidebarFooter>