updates

2025-11-26 08:15:00 -06:00
parent 3cbd4525a0
commit bb829c9b63
18 changed files with 2440 additions and 349 deletions
--- a/docs/CHANGES.md
+++ b/docs/CHANGES.md
@@ -0,0 +1,225 @@
+# Recent Changes Summary
+
+## User Configuration Changes
+
+### Non-Root User (rocky)
+- **Changed from**: `root` user for testing
+- **Changed to**: `rocky` user (standard Rocky Linux user)
+- **Reason**: Sparrowdo best practices require non-root user with sudo privileges
+
+### SSH Key Injection
+- SSH keys are now injected for both `root` and `rocky` users
+- The `rocky` user has NOPASSWD sudo access (required for Sparrowdo bootstrap)
+
+### Sparrowdo Bootstrap Step
+- **New requirement**: Must run `sparrowdo --bootstrap` before running tests
+- Bootstrap installs necessary Raku/Sparrowdo dependencies on the target VM
+- Bootstrap runs with the `rocky` user
+
+## Test Execution Flow
+
+### Old Flow (Root User)
+```bash
+1. Provision VM
+2. Wait for SSH (root user)
+3. Run sparrowdo test (root user, --no_sudo)
+```
+
+### New Flow (Rocky User with Bootstrap)
+```bash
+1. Provision VM
+2. Wait for SSH (rocky user)
+3. Run sparrowdo --bootstrap (rocky user)
+4. Run sparrowdo test (rocky user, --no_sudo, --verbose, --color)
+```
+
+## Sparrowfile Naming
+- Tests can use either `sparrowfile` or `main.raku`
+- The framework checks for `main.raku` first, then falls back to `sparrowfile`
+
+## Command Examples
+
+### Manual Bootstrap
+```bash
+sparrowdo \
+  --host 192.168.124.54 \
+  --ssh_user rocky \
+  --ssh_private_key ~/.ssh/id_rsa \
+  --bootstrap \
+  --color
+```
+
+### Manual Test Run
+```bash
+sparrowdo \
+  --host 192.168.124.54 \
+  --ssh_user rocky \
+  --ssh_private_key ~/.ssh/id_rsa \
+  --no_sudo \
+  --sparrowfile test-repo/main.raku \
+  --verbose \
+  --color
+```
+
+## Golden Image Changes
+
+### Users Created
+1. **rocky** (primary test user)
+   - Password: `rockypass`
+   - Sudo: NOPASSWD:ALL
+   - SSH key: Injected from `~/.ssh/id_rsa.pub`
+
+2. **testuser** (backward compatibility)
+   - Password: `testpass`
+   - Sudo: NOPASSWD:ALL
+   - SSH key: Not injected
+
+3. **root**
+   - Password: `rockytesting`
+   - SSH key: Injected from `~/.ssh/id_rsa.pub`
+
+## Bug Fixes
+
+### 1. virt-customize D-Bus Errors ✅
+- **Problem**: Prep scripts used `firewall-cmd`, `hostnamectl` which don't work offline
+- **Solution**: Removed D-Bus dependent commands from prep scripts
+- **Impact**: Golden image creation now works reliably
+
+### 2. VM Provisioning Script Failures ✅
+- **Problem**: `set -e` caused script to exit on arithmetic operations
+- **Solution**: Removed `set -e`, added explicit error handling
+- **Impact**: VMs provision successfully and report proper errors
+
+### 3. Virsh Connection Issues ✅
+- **Problem**: Non-root users couldn't access libvirt without explicit URI
+- **Solution**: Added `-c qemu:///system` to all virsh commands
+- **Impact**: Scripts work for jenkins user and other non-root users
+
+### 4. Image Caching ✅
+- **Problem**: Framework re-downloaded QCOW2 images on every build
+- **Solution**: Cache images by filename, copy for each build
+- **Impact**: Massive time savings (2GB download → 2 second copy)
+- **Control**: `REDOWNLOAD_IMAGE` parameter forces fresh download when needed
+
+### 5. IP Address Capture in Manual Script ✅
+- **Problem**: Script captured all output including progress messages
+- **Solution**: Extract only the last line (the IP address)
+- **Impact**: SSH connection works properly
+
+## File Changes
+
+### Modified Files
+- `Jenkinsfile` - Added bootstrap step, rocky user, main.raku support
+- `scripts/setup_base.sh` - Inject SSH keys for rocky user
+- `scripts/provision_vm.sh` - Fixed set -e issue, improved error handling
+- `scripts/cleanup_vm.sh` - Added explicit qemu:///system connection
+- `docs/default-prep.sh` - Create rocky user, remove D-Bus commands
+- `docs/manual-test-run.sh` - Add bootstrap step, use rocky user
+- `README.md` - Updated prerequisites (guestfs-tools)
+
+### New Files
+- `docs/virt-customize-guide.md` - Comprehensive guide on offline image customization
+- `docs/manual-steps.md` - Step-by-step manual testing guide
+- `docs/CHANGES.md` - This file
+
+## Testing Checklist
+
+### Before Running Tests
+- [ ] SSH keys exist (`~/.ssh/id_rsa` and `~/.ssh/id_rsa.pub`)
+- [ ] libvirt is running (`sudo systemctl status libvirtd`)
+- [ ] Default network is active (`virsh -c qemu:///system net-list`)
+- [ ] User has sudo access
+- [ ] guestfs-tools installed (`which virt-customize`)
+- [ ] Sparrowdo installed (`which sparrowdo`)
+
+### Golden Image Verification
+```bash
+# After creating golden image, verify users exist
+sudo virt-customize -a /path/to/golden.qcow2 --run-command 'id rocky'
+sudo virt-customize -a /path/to/golden.qcow2 --run-command 'cat /home/rocky/.ssh/authorized_keys'
+```
+
+### Manual VM Test
+```bash
+# Provision a test VM
+VM_IP=$(./scripts/provision_vm.sh test-vm-$(date +%s) /path/to/golden.qcow2 60)
+
+# Test SSH as rocky user
+ssh -i ~/.ssh/id_rsa -o StrictHostKeyChecking=no rocky@$VM_IP
+
+# Test sudo access
+ssh -i ~/.ssh/id_rsa rocky@$VM_IP 'sudo whoami'
+
+# Bootstrap Sparrowdo
+sparrowdo --host $VM_IP --ssh_user rocky --bootstrap --color
+
+# Clean up
+./scripts/cleanup_vm.sh test-vm-XXXXX
+```
+
+## Migration Notes
+
+### For Existing Tests
+If you have existing Sparrowdo tests that assumed root user:
+
+1. **Update TEST_MATRIX** in Jenkins to use rocky user
+2. **Ensure tests use --no_sudo** flag
+3. **Add bootstrap step** before test execution
+4. **Verify sudoers access** if tests need elevated privileges
+
+### Jenkins Pipeline Changes
+The Jenkinsfile automatically handles:
+- Creating rocky user in golden image
+- Injecting SSH keys for rocky user
+- Running bootstrap before tests
+- Using `--no_sudo` flag with rocky user
+
+No manual intervention needed for Jenkins builds.
+
+## Performance Improvements
+
+### Image Caching
+- **First build**: Downloads 2GB QCOW2 image (~5-10 minutes)
+- **Subsequent builds**: Copies from cache (~2 seconds)
+- **Disk usage**: One cached image + one per active build
+- **Cleanup**: Cached images persist, build images auto-delete
+
+### VM Provisioning Speed
+- **Linked clones**: New VM disk created in < 1 second
+- **Boot time**: ~10-20 seconds to get IP address
+- **Total provision time**: ~30 seconds from start to SSH ready
+
+## Troubleshooting
+
+### Bootstrap Fails
+```bash
+# Check if rocky user has sudo
+ssh -i ~/.ssh/id_rsa rocky@VM_IP 'sudo whoami'
+
+# Check if perl is installed
+ssh -i ~/.ssh/id_rsa rocky@VM_IP 'which perl'
+
+# Re-run bootstrap with verbose output
+sparrowdo --host VM_IP --ssh_user rocky --bootstrap --color
+```
+
+### Test Fails with Permission Denied
+```bash
+# Verify SSH key is injected
+ssh -i ~/.ssh/id_rsa rocky@VM_IP 'cat ~/.ssh/authorized_keys'
+
+# Check if using correct user
+# Should be rocky@VM_IP not root@VM_IP
+```
+
+### VM Won't Get IP
+```bash
+# Check network status
+virsh -c qemu:///system net-list --all
+
+# Start default network if needed
+sudo virsh net-start default
+
+# Check DHCP leases
+virsh -c qemu:///system net-dhcp-leases default
+```