ospab.network/docs/DEPLOYMENT.md

# OSTP Deployment Best Practices

Guidelines for deploying OSTP in production environments with maximum security and reliability.

---

## Infrastructure Planning

### Server Placement Strategy

**1. Geographic Distribution**
- Deploy servers in multiple regions to minimize latency
- Consider legal jurisdictions favorable to privacy
- Use data centers with strong physical security
- Avoid countries with mandatory data retention laws

**2. IP Address Selection**
- Use clean IPs without reputation issues
- Avoid IP ranges commonly associated with VPN providers
- Consider using residential proxy IPs for maximum stealth
- Rotate server IPs periodically (every 3-6 months)

**3. Network Architecture**

```
[Internet] → [CDN/Proxy] → [OSTP Server] → [Internal Network]
                              ↓
                        [User Database]
                        [DNS Forwarder]
```

Benefits:
- CDN/proxy layer provides DDoS protection
- Hides real server IP from direct client access
- Allows for traffic distribution

---

## Security Hardening

### Server Operating System

**Minimal Installation**
```bash
# Ubuntu Server minimal install
sudo apt update
sudo apt install -y ufw fail2ban sqlite3

# Disable unnecessary services
sudo systemctl disable bluetooth
sudo systemctl disable cups
sudo systemctl disable avahi-daemon

# Enable automatic security updates
sudo apt install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
```

**Kernel Hardening** (`/etc/sysctl.conf`):
```ini
# Disable IPv6 if not needed
net.ipv6.conf.all.disable_ipv6 = 1

# SYN flood protection
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 2048

# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

# Disable source packet routing
net.ipv4.conf.all.accept_source_route = 0

# Log martian packets
net.ipv4.conf.all.log_martians = 1
```

### Firewall Configuration

**Defense in Depth:**
```bash
# Default deny policy
sudo ufw default deny incoming
sudo ufw default deny outgoing

# Allow only necessary ports
sudo ufw allow out 53/udp  # DNS
sudo ufw allow out 123/udp # NTP
sudo ufw allow in 8443/tcp # OSTP

# Allow established connections
sudo ufw allow out on tun0 from any to any

# Enable firewall
sudo ufw enable
```

### SSH Hardening

Edit `/etc/ssh/sshd_config`:
```
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 22222  # Non-standard port
AllowUsers admin
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2
```

### Fail2Ban Configuration

Create `/etc/fail2ban/jail.local`:
```ini
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3

[sshd]
enabled = true
port = 22222
```

---

## PSK Management

### Generation Strategy

```bash
# Generate PSK with high entropy
ostp-server gen-key > /secure/storage/psk-$(date +%Y%m%d).txt

# Or use system random
xxd -p -l 32 /dev/urandom | tr -d '\n'
```

### Distribution Methods

**Secure Channels:**
1. **Signal/Telegram Secret Chats** — End-to-end encrypted messaging
2. **PGP-Encrypted Email** — Encrypt PSK with user's public key
3. **Password Managers** — Share via 1Password/Bitwarden shared vaults
4. **QR Codes** — Generate QR codes for in-person distribution
5. **Hardware Tokens** — Store on encrypted USB drives

**Never:**
- Send PSKs via unencrypted email
- Post PSKs in public forums or chat groups
- Store PSKs in version control
- Reuse PSKs across multiple users

### Rotation Schedule

| User Type | Rotation Frequency | Method |
|-----------|-------------------|---------|
| Individual | 90 days | Manual update |
| Corporate | 30 days | Automated push |
| High-Risk | 7 days | Dynamic generation |

### Multi-PSK Architecture (Advanced)

For large deployments, use separate PSKs per user group:

```json
{
  "psk_groups": {
    "premium_users": "a1b2c3...",
    "trial_users": "d4e5f6...",
    "enterprise": "g7h8i9..."
  }
}
```

Requires custom `PskValidator` implementation in `ostp/src/crypto.rs`.

---

## Monitoring & Alerting

### Metrics to Track

**Connection Metrics:**
- Active connections per minute
- Failed authentication attempts
- Average session duration
- Bandwidth per user

**System Metrics:**
- CPU usage
- Memory usage
- Disk I/O
- Network throughput

**Security Metrics:**
- Failed PSK validations
- Repeated connection attempts from same IP
- Unusual traffic patterns
- Anti-debug trigger counts

### Logging Strategy

**Log Levels by Environment:**
```
Development: debug
Staging: info
Production: warn
High-Security: error only
```

**Log Aggregation:**
```bash
# Forward logs to central syslog server
sudo apt install -y rsyslog
echo "*.* @@log-server:514" >> /etc/rsyslog.conf
sudo systemctl restart rsyslog
```

**Log Retention:**
- Keep 30 days of detailed logs
- Archive 1 year of summarized logs
- Purge logs older than 1 year

### Alerting Rules

Set up alerts for:
- Server CPU >80% for 5 minutes
- Disk usage >90%
- Failed auth rate >100/minute (potential attack)
- Service downtime >1 minute
- Certificate expiration <30 days (if using real certs)

---

## Backup & Disaster Recovery

### Backup Strategy

**Daily Backups:**
```bash
#!/bin/bash
# /usr/local/bin/ostp-backup.sh

DATE=$(date +%Y%m%d-%H%M)
BACKUP_DIR=/backup/ostp

# Backup user database
sqlite3 /var/lib/ostp/users.db ".backup '$BACKUP_DIR/users-$DATE.db'"

# Backup configuration
cp /etc/ostp/server.json $BACKUP_DIR/config-$DATE.json

# Compress and encrypt
tar czf - $BACKUP_DIR/*-$DATE.* | \
  gpg --encrypt --recipient admin@ospab.host > \
  $BACKUP_DIR/ostp-backup-$DATE.tar.gz.gpg

# Upload to remote storage (S3, Backblaze, etc.)
aws s3 cp $BACKUP_DIR/ostp-backup-$DATE.tar.gz.gpg \
  s3://ostp-backups/

# Clean up old backups (keep 7 days)
find $BACKUP_DIR -mtime +7 -delete
```

**Crontab Entry:**
```bash
0 2 * * * /usr/local/bin/ostp-backup.sh
```

### Disaster Recovery Plan

**RTO (Recovery Time Objective): 1 hour**
**RPO (Recovery Point Objective): 24 hours**

1. **Server Failure:**
   - Spin up new VPS from template
   - Restore latest backup
   - Update DNS records
   - Notify users (if IP changed)

2. **Database Corruption:**
   - Stop ostp-server
   - Restore from latest backup
   - Verify data integrity
   - Restart service

3. **PSK Compromise:**
   - Generate new PSK immediately
   - Deploy to server
   - Distribute to all users via secure channels
   - Invalidate old PSK

---

## Performance Optimization

### Connection Limits

Based on server specs:

| RAM | CPU Cores | Max Connections | Notes |
|-----|-----------|----------------|-------|
| 1 GB | 1 core | 100 | Minimal |
| 2 GB | 2 cores | 500 | Recommended |
| 4 GB | 4 cores | 1000 | High capacity |
| 8 GB | 8 cores | 5000 | Enterprise |

### Load Balancing

For >1000 concurrent users:

```
          [DNS Round-Robin]
                 ↓
     +-----------+-----------+
     ↓           ↓           ↓
[OSTP Server 1] [Server 2] [Server 3]
     ↓           ↓           ↓
        [Shared Database]
```

Use database replication (PostgreSQL) instead of SQLite for multi-server deployments.

### Bandwidth Management

```bash
# Rate limit per connection (Linux tc)
tc qdisc add dev eth0 root tbf rate 10mbit burst 32kbit latency 400ms

# Or use iptables hashlimit
iptables -A INPUT -p tcp --dport 8443 \
  -m hashlimit --hashlimit-name ostp \
  --hashlimit-above 10/s --hashlimit-mode srcip \
  -j DROP
```

---

## Compliance & Legal

### Data Retention Policies

**Recommended Minimal Logging:**
- Do NOT log plaintext traffic content
- Do NOT log destination IPs
- Do log: connection timestamps, total bytes transferred, user IDs

**Compliance Requirements:**
- GDPR (EU): User data deletion requests
- CCPA (California): Data access requests
- National regulations: Vary by jurisdiction

### Privacy by Design

1. **Minimize Data Collection:** Only collect necessary metadata
2. **Encrypt at Rest:** Database encryption for user data
3. **Anonymize Logs:** Hash IPs before storing
4. **Short Retention:** Delete logs after 30 days
5. **No Third Parties:** Never share user data

### Warrant Canary (Optional)

Create a `canary.txt` on your website:

```
As of January 1, 2026, OSTP has NOT received:
- National security letters
- Gag orders
- Warrants for user data
- Requests to install monitoring software

This canary will be updated monthly.
```

If you receive a gag order, stop updating the canary (signals users).

---

## Scaling Strategy

### Phase 1: Single Server (0-500 users)
- 1 VPS with 2GB RAM
- SQLite database
- Direct client connections

### Phase 2: Vertical Scaling (500-2000 users)
- Upgrade to 4GB RAM, 4 cores
- Optimize database indexes
- Add monitoring

### Phase 3: Horizontal Scaling (2000+ users)
- Deploy multiple servers
- Shared PostgreSQL database with replication
- DNS-based load balancing
- Separate user database server

### Phase 4: Global Infrastructure (10000+ users)
- Regional server clusters
- Geo-DNS routing
- CDN integration
- Dedicated DDoS protection

---

## Cost Analysis

### Single Server Monthly Costs

| Provider | Specs | Price | Notes |
|----------|-------|-------|-------|
| Hetzner | 2GB RAM, 40GB SSD | $5 | Best value |
| DigitalOcean | 2GB RAM, 50GB SSD | $12 | Easy setup |
| Vultr | 2GB RAM, 55GB SSD | $12 | Global locations |
| AWS Lightsail | 2GB RAM, 60GB SSD | $12 | AWS ecosystem |

**Additional Costs:**
- Domain name: $10-15/year
- Backup storage: $2-5/month
- Monitoring: $10-20/month (optional)
- DDoS protection: $20-100/month (if needed)

**Revenue Model (Optional):**
- Free tier: 5GB/month
- Basic: $5/month (100GB)
- Premium: $15/month (unlimited)

---

## Update Strategy

### Binary Updates

**Server Updates:**
```bash
# Download new version
wget https://releases.ospab.host/ostp-server-v2.0.0

# Backup current binary
cp /usr/local/bin/ostp-server /usr/local/bin/ostp-server.backup

# Replace binary
mv ostp-server-v2.0.0 /usr/local/bin/ostp-server
chmod +x /usr/local/bin/ostp-server

# Restart service
systemctl restart ostp-server

# Verify version
ostp-server --version
```

**Client Updates:**
- Provide download link via secure channel
- Include SHA256 hash for verification
- Document breaking changes
- Maintain compatibility for 1 version back

### Database Migrations

For schema changes:

```sql
-- Check current version
SELECT value FROM metadata WHERE key = 'schema_version';

-- Migrate v1 → v2
ALTER TABLE users ADD COLUMN last_seen DATETIME;
UPDATE metadata SET value = '2' WHERE key = 'schema_version';
```

---

## Incident Response

### Security Incident Playbook

**1. Detection:**
- Monitor alerts for anomalies
- Investigate suspicious patterns
- Verify authenticity of alerts

**2. Containment:**
- Isolate affected servers
- Block malicious IPs
- Rotate compromised PSKs
- Snapshot system state

**3. Eradication:**
- Identify root cause
- Patch vulnerabilities
- Remove backdoors
- Reset credentials

**4. Recovery:**
- Restore from clean backups
- Verify system integrity
- Monitor for reinfection
- Document timeline

**5. Post-Mortem:**
- Write incident report
- Implement prevention measures
- Update documentation
- Notify affected users (if required)

---

## Testing & Validation

### Pre-Deployment Checklist

- [ ] Server OS fully updated
- [ ] Firewall configured and tested
- [ ] OSTP server starts without errors
- [ ] PSK generated and secured
- [ ] Client successfully connects
- [ ] Traffic encrypted (verify with Wireshark)
- [ ] TLS mimicry working (check SNI)
- [ ] Anti-debug protection active (release build)
- [ ] Logs rotating correctly
- [ ] Backups automated and tested
- [ ] Monitoring alerts configured
- [ ] Documentation complete

### Performance Testing

```bash
# Simulate 100 concurrent connections
for i in {1..100}; do
  ostp-client connect -s server:8443 -p $PSK -c US &
done

# Monitor server resources
htop
iotop
nethogs
```

### Security Testing

- **Port Scanning:** Verify only 8443 is open
- **DPI Testing:** Use GFW test tools to verify mimicry
- **Traffic Analysis:** Capture packets and analyze with Wireshark
- **Penetration Testing:** Hire professional auditors (recommended annually)

---

## Advanced Configurations

### Multi-Hop Routing

Chain OSTP servers for enhanced anonymity:

```
[Client] → [OSTP Server 1] → [OSTP Server 2] → [Internet]
```

Requires custom routing configuration.

### Split Tunneling

Route only specific traffic through VPN:

```bash
# Route only HTTPS through VPN
ip route add 0.0.0.0/0 via 10.8.0.1 table 100
ip rule add fwmark 1 table 100
iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 1
```

### Bridge Mode

Allow clients to access LAN resources:

```
[OSTP Client] ←→ [OSTP Server] ←→ [Corporate LAN]
```

Requires NAT and routing configuration on server.

---

## Support & Maintenance

### Regular Maintenance Tasks

**Daily:**
- Check service status
- Review critical alerts
- Monitor disk space

**Weekly:**
- Review security logs
- Check backup integrity
- Update block lists (if applicable)

**Monthly:**
- Apply security updates
- Review user quotas
- Analyze performance metrics
- Test disaster recovery

**Quarterly:**
- Rotate PSKs
- Audit user access
- Review documentation
- Plan capacity upgrades

---

For additional guidance or enterprise support, contact ospab@ospab.host.

*Last updated: January 2026*