From 550ab8486f1107e051dd48df746ba0c9f1205934 Mon Sep 17 00:00:00 2001 From: DeNNiiInc Date: Sat, 27 Dec 2025 21:59:27 +1100 Subject: [PATCH] Updated deployment template with real-world fixes and TurnKey troubleshooting --- PROXMOX_DEPLOY_TEMPLATE.md | 673 ++++++++++++++++++++++++++++++------- 1 file changed, 555 insertions(+), 118 deletions(-) diff --git a/PROXMOX_DEPLOY_TEMPLATE.md b/PROXMOX_DEPLOY_TEMPLATE.md index 5d21676..e6679b5 100644 --- a/PROXMOX_DEPLOY_TEMPLATE.md +++ b/PROXMOX_DEPLOY_TEMPLATE.md @@ -1,211 +1,648 @@ # ๐Ÿš€ Proxmox Deployment Template (TurnKey Node.js) -**Use this guide to deploy ANY Node.js application to a TurnKey Linux LXC Container.** +**Real-world tested deployment guide for ANY Node.js application to TurnKey Linux LXC Container.** + +This template is based on actual deployment experience and includes all the fixes and troubleshooting steps discovered during real deployments. --- ## ๐Ÿ“‹ Prerequisites -1. **Project**: A Node.js application (Express, Next.js, etc.) in a Git repository. -2. **Server**: A Proxmox TurnKey Node.js Container. -3. **Access**: Root SSH password for the container. -4. **Domain (Optional)**: If using Cloudflare Tunnel. +1. **Project**: A Node.js application (Express, Next.js, static site, etc.) in a Git repository +2. **Server**: A Proxmox TurnKey Node.js Container (already created) +3. **Access**: Root SSH password for the container +4. **GitHub**: Personal Access Token (PAT) with `repo` permissions +5. **Domain** (Optional): If using Cloudflare Tunnel or DNS --- -## ๐Ÿ› ๏ธ Step 1: Prepare Your Project +## ๏ฟฝ Step 0: Prepare GitHub Personal Access Token -Ensure your project is ready for production: - -1. **Port Configuration**: Ensure your app listens on a configurable port or a fixed internal port (e.g., `4001`). - ```javascript - // server.js - const PORT = process.env.PORT || 4001; - app.listen(PORT, ...); - ``` - -2. **Git Ignore**: Ensure `node_modules` and config files with secrets are ignored. - ```gitignore - node_modules/ - .env - config.json - ``` +1. Go to: https://github.com/settings/tokens +2. Click "Generate new token (classic)" +3. Set note: "Proxmox Auto-Deploy" +4. Check the `repo` scope (full control of private repositories) +5. Click "Generate token" +6. **Copy the token immediately** (you won't see it again!) --- -## ๐Ÿ–ฅ๏ธ Step 2: One-Time Server Setup +## ๐Ÿ› ๏ธ Step 1: Prepare Your Project Locally -SSH into your new container: -```bash -ssh root@ +### 1.1 Ensure Port Configuration + +Your app should listen on a configurable port: + +```javascript +// server.js +const PORT = process.env.PORT || 3000; +app.listen(PORT, () => { + console.log(`Server running on port ${PORT}`); +}); ``` -Run these commands to prepare the environment: +### 1.2 Create .gitignore -### 1. Install Essentials -```bash -apt-get update && apt-get install -y git +**CRITICAL**: Protect credentials from Git: + +```gitignore +# Credentials - NEVER commit these +deploy-config.json +credentials*.json +.env +*.pem +*.key ``` -### 2. Prepare Directory -```bash -# Standard web directory -mkdir -p /var/www/ -cd /var/www/ +### 1.3 Create deploy-config.json (Locally Only) -# Clone your repo (Use Basic Auth with Token if private) -# Format: https://:@github.com//.git -git clone . +Create this file locally but **NEVER commit it**: -# Install dependencies -npm install -``` - -### 3. Setup Permissions -```bash -# Give ownership to www-data (Nginx user) -chown -R www-data:www-data /var/www/ +```json +{ + "host": "YOUR_SERVER_IP", + "port": 22, + "username": "root", + "password": "YOUR_ROOT_PASSWORD", + "remotePath": "/var/www/your-app-name", + "appName": "your-app-name", + "github": { + "username": "YourGitHubUsername", + "token": "ghp_YourPersonalAccessToken", + "repo": "YourUsername/YourRepo" + } +} ``` --- -## โš™๏ธ Step 3: Application Configuration +## ๐Ÿ–ฅ๏ธ Step 2: Initial Server Setup (SSH) -### 1. Systemd Service -Create a service file to keep your app running. +### 2.1 SSH into your server -Create `/etc/systemd/system/.service`: -```ini +```bash +ssh root@YOUR_SERVER_IP +``` + +### 2.2 Install essentials + +```bash +apt-get update && apt-get install -y git jq +``` + +### 2.3 Verify Node.js installation + +```bash +which node +node --version +# TurnKey Node.js usually installs to: /usr/local/bin/node +``` + +--- + +## ๐Ÿ“ฆ Step 3: Deploy Your Application + +### 3.1 Create application directory + +```bash +mkdir -p /var/www/your-app-name +cd /var/www/your-app-name +``` + +### 3.2 Clone repository with authentication + +**CRITICAL**: Use HTTPS with token authentication: + +```bash +# Format: https://USERNAME:TOKEN@github.com/REPO.git +git clone 'https://YourUsername:ghp_YourToken@github.com/YourUsername/YourRepo.git' . + +# Remove credentials from Git config immediately +git remote set-url origin "https://github.com/YourUsername/YourRepo.git" +``` + +### 3.3 Configure Git credential helper (memory only) + +```bash +git config credential.helper 'cache --timeout=3600' +``` + +### 3.4 Install dependencies + +```bash +npm install --production +``` + +--- + +## โš™๏ธ Step 4: Create Systemd Service + +**Why Systemd instead of PM2?** +- Native to Linux (no extra software) +- Better logging with journalctl +- More reliable auto-restart +- Boot persistence without configuration + +### 4.1 Create service file + +```bash +cat > /etc/systemd/system/your-app-name.service << 'EOF' [Unit] -Description= Service +Description=Your App Name Service After=network.target [Service] Type=simple User=root -# OR use 'www-data' if app doesn't need root ports -# User=www-data -WorkingDirectory=/var/www/ +WorkingDirectory=/var/www/your-app-name ExecStart=/usr/local/bin/node server.js Restart=always +RestartSec=10 Environment=NODE_ENV=production -Environment=PORT=4001 +Environment=PORT=3000 + +# Logging +StandardOutput=journal +StandardError=journal +SyslogIdentifier=your-app-name [Install] WantedBy=multi-user.target +EOF ``` -Enable and start: +**IMPORTANT**: Verify the path to node (`/usr/local/bin/node`) using `which node` + +### 4.2 Enable and start service + ```bash systemctl daemon-reload -systemctl enable -systemctl start +systemctl enable your-app-name +systemctl start your-app-name + +# Check status +systemctl status your-app-name --no-pager ``` -### 2. Nginx Reverse Proxy -Configure Nginx to forward port 80 to your app (Port 4001). +**Troubleshooting**: If you get `status=203/EXEC`, verify the ExecStart path is correct. -Create `/etc/nginx/sites-available/`: -```nginx +--- + +## ๐ŸŒ Step 5: Configure Nginx (Critical for TurnKey) + +### 5.1 **CRITICAL**: Remove TurnKey Default Sites + +TurnKey Linux comes with default Nginx configurations that show the TurnKey control panel. **You MUST remove these**: + +```bash +# Remove ALL TurnKey default sites +rm -f /etc/nginx/sites-enabled/default +rm -f /etc/nginx/sites-enabled/nodejs +rm -f /etc/nginx/sites-enabled/node* +rm -f /etc/nginx/sites-enabled/tkl-webcp +``` + +### 5.2 Create your application's Nginx configuration + +**IMPORTANT**: Watch variable escaping! Use single quotes or backticks properly: + +```bash +cat > /etc/nginx/sites-available/your-app-name << 'EOF' server { - listen 80; + listen 80 default_server; server_name _; - root /var/www/; + # Serve static files directly (faster!) + root /var/www/your-app-name; index index.html; - # Serve static files (Optional) + # Try static files first, then proxy to Node.js location / { - try_files $uri $uri/ =404; + try_files $uri $uri/ /index.html; } - # Proxy API/Dynamic requests + # Proxy API requests to Node.js location /api { - proxy_pass http://localhost:4001; + proxy_pass http://localhost:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; proxy_cache_bypass $http_upgrade; } + + # Security headers + add_header X-Frame-Options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + add_header X-XSS-Protection "1; mode=block" always; + + # Cache static assets + location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2|ttf|eot)$ { + expires 1y; + add_header Cache-Control "public, immutable"; + } } +EOF ``` -Enable site: +### 5.3 Enable your site + ```bash -# Remove defaults +# Link your site configuration +ln -sf /etc/nginx/sites-available/your-app-name /etc/nginx/sites-enabled/ + +# Test configuration +nginx -t + +# Reload Nginx +systemctl reload nginx +``` + +### 5.4 Verify it's working + +```bash +# Test locally +curl -I http://localhost + +# Should show: HTTP/1.1 200 OK +# If you see "TurnKey Node.js" in the output, the old config is still active +``` + +--- + +## ๐Ÿ”„ Step 6: Setup Auto-Sync (Every 5 Minutes) + +### 6.1 Create auto-sync script + +```bash +cat > /var/www/your-app-name/auto-sync.sh << 'EOF' +#!/bin/bash + +set -e + +APP_NAME="your-app-name" +APP_DIR="/var/www/$APP_NAME" +LOG_FILE="/var/log/$APP_NAME-autosync.log" + +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" +} + +log "=========================================" +log "Starting auto-sync check..." + +cd "$APP_DIR" || exit 1 + +# Fetch latest from remote +git fetch origin main 2>&1 | tee -a "$LOG_FILE" + +# Check if local is behind remote +LOCAL=$(git rev-parse HEAD) +REMOTE=$(git rev-parse origin/main) + +if [ "$LOCAL" = "$REMOTE" ]; then + log "โœ… Already up to date. No changes detected." + exit 0 +fi + +log "๐Ÿ”„ Changes detected! Pulling updates..." + +# Pull changes +git pull origin main 2>&1 | tee -a "$LOG_FILE" + +# Install/update dependencies if package.json changed +if git diff --name-only $LOCAL $REMOTE | grep -q "package.json"; then + log "๐Ÿ“ฆ package.json changed. Running npm install..." + npm install 2>&1 | tee -a "$LOG_FILE" +fi + +# Restart the service +log "๐Ÿ”„ Restarting $APP_NAME service..." +systemctl restart "$APP_NAME" 2>&1 | tee -a "$LOG_FILE" + +# Wait and check status +sleep 2 +if systemctl is-active --quiet "$APP_NAME"; then + log "โœ… Service restarted successfully!" +else + log "โŒ WARNING: Service may have failed to start!" + systemctl status "$APP_NAME" --no-pager 2>&1 | tee -a "$LOG_FILE" +fi + +log "โœ… Auto-sync completed!" +log "=========================================" +EOF + +# Make executable +chmod +x /var/www/your-app-name/auto-sync.sh + +# Create log file +touch /var/log/your-app-name-autosync.log +chmod 644 /var/log/your-app-name-autosync.log +``` + +### 6.2 Add to crontab + +```bash +# Add cron job (every 5 minutes) +echo "*/5 * * * * cd /var/www/your-app-name && /bin/bash auto-sync.sh" | crontab - + +# Verify it was added +crontab -l +``` + +--- + +## ๐Ÿงช Step 7: Verify Deployment + +### 7.1 Check service status + +```bash +systemctl status your-app-name --no-pager +# Should show: active (running) +``` + +### 7.2 Check Nginx + +```bash +systemctl status nginx +nginx -t +``` + +### 7.3 Test the application + +```bash +# Test locally +curl http://localhost + +# Should return your HTML, NOT the TurnKey control panel +``` + +### 7.4 Test from browser + +Visit: `http://YOUR_SERVER_IP` + +**If you see the TurnKey control panel**, go back to Step 5 and verify you removed all default Nginx sites. + +--- + +## ๐Ÿšจ Common Issues & Fixes + +### Issue 1: TurnKey Control Panel Shows Instead of App + +**Symptoms**: Browser shows "TurnKey Node.js" page with Webmin link + +**Cause**: TurnKey default Nginx configuration is still active + +**Fix**: +```bash +# Remove ALL default sites rm -f /etc/nginx/sites-enabled/default rm -f /etc/nginx/sites-enabled/nodejs +rm -f /etc/nginx/sites-enabled/node* +rm -f /etc/nginx/sites-enabled/tkl-webcp -# Link new site -ln -s /etc/nginx/sites-available/ /etc/nginx/sites-enabled/ +# Verify only your site is enabled +ls -la /etc/nginx/sites-enabled/ +# Should only show: your-app-name -# Reload +# Reload Nginx nginx -t && systemctl reload nginx ``` ---- +### Issue 2: Nginx Shows "500 Internal Server Error" -## โ˜๏ธ Step 4: Cloudflare Tunnel (Secure Access) +**Cause**: Variable escaping issues in Nginx config -Expose your app securely without opening router ports. +**Fix**: Recreate the Nginx config using the exact format in Step 5.2 -### 1. Install Cloudflared ```bash -# Add Key -mkdir -p --mode=0755 /usr/share/keyrings -curl -fsSL https://pkg.cloudflare.com/cloudflare-public-v2.gpg | tee /usr/share/keyrings/cloudflare-public-v2.gpg >/dev/null +# View your current config +cat /etc/nginx/sites-enabled/your-app-name -# Add Repo -echo 'deb [signed-by=/usr/share/keyrings/cloudflare-public-v2.gpg] https://pkg.cloudflare.com/cloudflared any main' | tee /etc/apt/sources.list.d/cloudflared.list - -# Install -apt-get update && apt-get install -y cloudflared +# If you see \\$uri instead of $uri, the escaping is wrong +# Delete and recreate using Step 5.2 ``` -### 2. Create Tunnel +### Issue 3: Service Keeps Restarting (status=203/EXEC) + +**Cause**: Wrong path to node executable + +**Fix**: ```bash -cloudflared tunnel login -cloudflared tunnel create -# Follow on-screen instructions to map domain -> http://localhost:4001 +# Find correct node path +which node + +# Update service file with correct path +sudo nano /etc/systemd/system/your-app-name.service +# Change ExecStart to match the output of 'which node' + +# Reload +systemctl daemon-reload +systemctl restart your-app-name +``` + +### Issue 4: Git Clone Fails with "already exists" + +**Cause**: Directory not empty + +**Fix**: +```bash +cd /var/www +rm -rf your-app-name +mkdir -p your-app-name +cd your-app-name +# Then retry git clone +``` + +### Issue 5: Auto-Sync Not Working + +**Check**: +```bash +# Verify cron job exists +crontab -l | grep auto-sync + +# Manually run sync to see errors +cd /var/www/your-app-name +./auto-sync.sh + +# Check logs +tail -f /var/log/your-app-name-autosync.log ``` --- -## ๐Ÿ”„ Step 5: Automated Updates (PowerShell) +## ๐Ÿ“Š Useful Commands -Create a script `deploy-remote.ps1` in your project root to automate updates. +### Service Management +```bash +# Status +systemctl status your-app-name -**Pre-requisite**: Create `deploy-config.json` (Add to .gitignore!): -```json -{ - "host": "", - "username": "root", - "password": "", - "remotePath": "/var/www/" +# Restart +systemctl restart your-app-name + +# View logs +journalctl -u your-app-name -f + +# View last 50 lines +journalctl -u your-app-name -n 50 +``` + +### Nginx Commands +```bash +# Test configuration +nginx -t + +# Reload +systemctl reload nginx + +# Restart +systemctl restart nginx + +# View error logs +tail -f /var/log/nginx/error.log + +# View access logs +tail -f /var/log/nginx/access.log +``` + +### Auto-Sync Commands +```bash +# View auto-sync logs +tail -f /var/log/your-app-name-autosync.log + +# Manually trigger sync +cd /var/www/your-app-name && ./auto-sync.sh + +# Check cron jobs +crontab -l +``` + +### Git Commands +```bash +# Check current commit +git log -1 --oneline + +# Check remote +git remote -v + +# Force pull +git fetch origin main +git reset --hard origin/main +``` + +--- + +## โœ… Deployment Checklist + +Use this checklist for every deployment: + +- [ ] SSH key accepted (run `ssh root@IP` manually first) +- [ ] GitHub PAT created and copied +- [ ] `.gitignore` includes `deploy-config.json` +- [ ] Local `deploy-config.json` created with credentials +- [ ] Repository cloned with token authentication +- [ ] Dependencies installed (`npm install`) +- [ ] Systemd service created with correct node path +- [ ] Service started and active +- [ ] **ALL TurnKey Nginx defaults removed** +- [ ] Your Nginx config created and enabled +- [ ] Nginx reloaded successfully +- [ ] Browser shows YOUR app (not TurnKey page) +- [ ] Auto-sync script created and executable +- [ ] Cron job added and verified +- [ ] Test auto-sync by pushing a change + +--- + +## ๐ŸŽฏ Quick Deployment Commands (Copy-Paste) + +Replace `YOUR_APP_NAME`, `YOUR_REPO`, etc. with your values: + +```bash +# 1. Install essentials +apt-get update && apt-get install -y git jq + +# 2. Clone and setup +mkdir -p /var/www/YOUR_APP_NAME +cd /var/www/YOUR_APP_NAME +git clone 'https://USERNAME:TOKEN@github.com/USER/REPO.git' . +git remote set-url origin "https://github.com/USER/REPO.git" +npm install --production + +# 3. Create systemd service (update node path!) +cat > /etc/systemd/system/YOUR_APP_NAME.service << 'EOF' +[Unit] +Description=YOUR_APP_NAME Service +After=network.target +[Service] +Type=simple +User=root +WorkingDirectory=/var/www/YOUR_APP_NAME +ExecStart=/usr/local/bin/node server.js +Restart=always +Environment=NODE_ENV=production +Environment=PORT=3000 +StandardOutput=journal +StandardError=journal +[Install] +WantedBy=multi-user.target +EOF + +systemctl daemon-reload && systemctl enable YOUR_APP_NAME && systemctl start YOUR_APP_NAME + +# 4. Remove TurnKey defaults (CRITICAL!) +rm -f /etc/nginx/sites-enabled/default /etc/nginx/sites-enabled/nodejs /etc/nginx/sites-enabled/node* /etc/nginx/sites-enabled/tkl-webcp + +# 5. Create Nginx config +cat > /etc/nginx/sites-available/YOUR_APP_NAME << 'EOF' +server { + listen 80 default_server; + server_name _; + root /var/www/YOUR_APP_NAME; + index index.html; + location / { try_files $uri $uri/ /index.html; } + location /api { + proxy_pass http://localhost:3000; + proxy_http_version 1.1; + proxy_set_header Host $host; + } } +EOF + +ln -sf /etc/nginx/sites-available/YOUR_APP_NAME /etc/nginx/sites-enabled/ +nginx -t && systemctl reload nginx + +# 6. Test +curl -I http://localhost ``` -**Script `deploy-remote.ps1`**: -```powershell -# Reads config and updates remote server -$Config = Get-Content "deploy-config.json" | ConvertFrom-Json -$User = $Config.username; $HostName = $Config.host; $Pass = $Config.password -$RemotePath = $Config.remotePath +--- -# Commands to run remotely -$Cmds = " - cd $RemotePath - echo 'โฌ‡๏ธ Pulling code...' - git pull - echo '๐Ÿ“ฆ Installing deps...' - npm install - echo '๐Ÿš€ Restarting service...' - systemctl restart - systemctl status --no-pager -" +## ๐ŸŽ‰ Success Indicators -echo y | plink -ssh -t -pw $Pass "$User@$HostName" $Cmds -``` +You'll know deployment succeeded when: -**Usage**: Just run `./deploy-remote.ps1` to deploy! +โœ… `systemctl status your-app-name` shows **active (running)** +โœ… `curl http://localhost` returns **your HTML (not TurnKey page)** +โœ… Browser at `http://SERVER_IP` shows **your application** +โœ… Auto-sync logs show successful checks +โœ… Pushing to GitHub triggers auto-update within 5 minutes + +--- + +## ๐Ÿ“š Additional Resources + +- **Systemd Documentation**: https://www.freedesktop.org/software/systemd/man/systemd.service.html +- **Nginx Documentation**: https://nginx.org/en/docs/ +- **TurnKey Node.js**: https://www.turnkeylinux.org/nodejs + +--- + +**Last Updated**: December 2025 (Based on real deployment experience) + +**Tested On**: TurnKey Linux Node.js 18.x LXC Container on Proxmox VE 8.x