admin/resf-testing-repo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement Ansible roles for Rocky Linux Testing Framework

Browse Source 
- Added `bootstrap_sparrowdo` role for bootstrapping Sparrowdo on a VM.
- Introduced `cleanup_vm` role for cleaning up VMs and disk images.
- Created `download_image` role to download and cache QCOW2 images.
- Developed `golden_image` role for creating and customizing golden images.
- Implemented `provision_vm` role for provisioning VMs as linked clones.
- Added `run_test` role for executing tests with Sparrowdo.
- Created playbooks for building golden images, running single tests, and running test suites.
- Enhanced documentation with usage examples, configuration details, and troubleshooting tips.
- Added support for multiple cloud providers (AWS, Azure) in the test execution workflow.

Signed-off-by: Stephen Simpson <ssimpson89@users.noreply.github.com>
This commit is contained in:
Stephen Simpson
2025-12-29 16:02:39 -06:00
parent bb829c9b63
commit ec04f0bec5
46 changed files with 2005 additions and 2055 deletions
Download Patch File Download Diff File Expand all files Collapse all files

372
README.md
View File

@@ -6,129 +6,154 @@ Simple, portable Sparrowdo testing framework for Rocky Linux.
This framework provides automated testing for Rocky Linux:
- **Simple Scripts**: 4 standalone bash scripts, easily portable
- **VM Isolation**: Each test runs in a fresh VM
- **Parallel Execution**: Run multiple tests concurrently
- **Fast Provisioning**: Linked clones (copy-on-write) for speed
- **Jenkins Ready**: Simple Jenkinsfile orchestrates scripts
- **Bootstrap Once**: Golden image approach saves time
Two implementations available:
- **Ansible**: Provider-aware, structured roles, better for automation (recommended)
- **Shell Scripts**: Simple scripts, easy to integrate anywhere (libvirt only)
## Prerequisites
```bash
sudo dnf install -y qemu-kvm libvirt virt-install guestfs-tools rakudo
sudo dnf install -y qemu-kvm libvirt virt-install guestfs-tools rakudo ansible
sudo systemctl enable --now libvirtd
sudo usermod -a -G libvirt $(whoami)
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N ""
```
## Quick Start
### Manual Test Run
### Ansible (Recommended)
```bash
# 1. Download base image (cached automatically)
cd ansible
# 1. Add your tests to vars/test-definitions.yml
# 2. Build golden image (interactive prompt for Rocky version)
ansible-playbook playbooks/libvirt/build-golden-image.yml
# 3. Run all tests
ansible-playbook playbooks/run-tests.yml
# 4. Run specific test
ansible-playbook playbooks/run-tests.yml -e "test_filter=Sparky_Knot"
```
### Shell Scripts
```bash
# Download base image
BASE=$(./scripts/download-image.sh \
https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2)
# 2. Build golden image (includes Sparrowdo bootstrap)
# Build golden image
./scripts/build-golden.sh "$BASE" /var/lib/libvirt/images/golden.qcow2
# 3. Run test
# Run test
./scripts/run-test.sh my-test \
https://github.com/your-org/test-repo.git \
/var/lib/libvirt/images/golden.qcow2
# 4. Cleanup
./scripts/cleanup-all.sh
```
### Jenkins
1. Create new Pipeline job in Jenkins
2. Point to this repository
3. Use `Jenkinsfile.simple`
4. Configure parameters:
- **QCOW2_URL**: Rocky Linux base image URL
- **TEST_REPOS**: One test repo URL per line
- **MAX_PARALLEL**: Number of concurrent tests
5. Build with Parameters
## How It Works
```
Download QCOW2 → Build Golden Image → Run Tests in Parallel → Cleanup
(includes bootstrap) (isolated VMs)
Download QCOW2 → Build Golden Image → Run Tests → Cleanup
(bootstrap once) (parallel)
```
**Golden Image Creation:**
1. Copy base QCOW2 image
2. Install Raku, Sparrowdo, dependencies via virt-customize
2. Install Raku, Sparrowdo via virt-customize
3. Create `rocky` user with sudo and SSH keys
4. Boot VM and run `sparrowdo --bootstrap` (once)
5. Shutdown VM, golden image ready
5. Shutdown, save as golden image
**Test Execution:**
1. Provision VM as linked clone (1 second)
1. Provision VM as linked clone (~1 second)
2. Clone test repository
3. Run `sparrowdo --no_sudo` with test
3. Run `sparrowdo --no_sudo`
4. Cleanup VM
## Scripts
### download-image.sh
```bash
./scripts/download-image.sh <url> [output_dir] [force]
```
Downloads and caches QCOW2 images. Returns path to cached image.
### build-golden.sh
```bash
./scripts/build-golden.sh <base_image> <golden_image> [ssh_pub_key]
```
Creates golden image from base QCOW2. Installs Raku/Sparrowdo and bootstraps.
### run-test.sh
```bash
./scripts/run-test.sh <test_name> <test_repo_url> <golden_image> [ssh_key]
```
Provisions VM, clones test repo, runs Sparrowdo test, cleans up.
### provision_vm.sh
```bash
./scripts/provision_vm.sh <vm_name> <golden_image> [timeout]
```
Creates VM as linked clone, starts it, returns IP address.
### cleanup_vm.sh
```bash
./scripts/cleanup_vm.sh <vm_name>
```
Destroys VM and removes disk image.
### cleanup-all.sh
```bash
./scripts/cleanup-all.sh [pattern]
```
Emergency cleanup for orphaned VMs matching pattern.
## Directory Structure
```
.
├── Jenkinsfile.simple # Simple Jenkins pipeline
├── README.md # This file
├── scripts/
│ ├── download-image.sh # Download/cache images
│ ├── build-golden.sh # Create golden image
│ ├── run-test.sh # Run single test
│ ├── provision_vm.sh # Provision VM
│ ├── cleanup_vm.sh # Cleanup VM
│ └── cleanup-all.sh # Emergency cleanup
└── docs/ # Additional documentation
├── ansible/ # Ansible implementation (provider-aware)
│ ├── inventory/
│ │ ├── hosts.yml # Provider groups (libvirt, aws, azure)
│ │ └── group_vars/ # Provider-specific settings
│ ├── vars/
│ │ └── test-definitions.yml # Centralized test list
│ ├── playbooks/
│ │ ├── libvirt/ # Libvirt-specific playbooks
│ │ │ └── build-golden-image.yml
│ │ ├── aws/ # AWS playbooks (future)
│ │ ├── azure/ # Azure playbooks (future)
│ │ └── run-tests.yml # Provider-agnostic test runner
│ ├── tasks/
│ │ └── run-test-with-infrastructure.yml
│ └── roles/ # Reusable roles
│ ├── download_image/
│ ├── golden_image/
│ ├── bootstrap_sparrowdo/
│ ├── provision_vm/
│ ├── run_test/
│ └── cleanup_vm/
├── scripts/ # Shell scripts (libvirt only)
│ ├── download-image.sh
│ ├── build-golden.sh
│ ├── run-test.sh
│ └── cleanup_vm.sh
└── docs/
├── ANSIBLE-GUIDE.md
└── BOOTSTRAP-APPROACH.md
```
## Test Repository Format
## Ansible Roles
| Role | Purpose |
|------|---------|
| `download_image` | Download and cache QCOW2 images |
| `golden_image` | Build golden image with virt-customize |
| `bootstrap_sparrowdo` | Bootstrap Sparrowdo on golden image |
| `provision_vm` | Provision VM as linked clone |
| `run_test` | Execute test workflow |
| `cleanup_vm` | Clean up VMs and disks |
## Shell Scripts
| Script | Purpose |
|--------|---------|
| `download-image.sh` | Download and cache QCOW2 images |
| `build-golden.sh` | Build golden image |
| `bootstrap_golden.sh` | Bootstrap Sparrowdo |
| `run-test.sh` | Run single test |
| `provision_vm.sh` | Create VM |
| `cleanup_vm.sh` | Cleanup single VM |
| `cleanup-all.sh` | Cleanup multiple VMs |
## Test Definitions
### Ansible
Edit `ansible/vars/test-definitions.yml` to add tests:
```yaml
tests:
- name: "Sparky_Knot"
repo_url: "https://git.resf.org/testing/Sparky_Knot.git"
# Optional: branch, timeout
- name: "ssh-test"
repo_url: "https://github.com/your-org/ssh-test.git"
branch: "develop"
timeout: 600
```
### Test Repository Format
Your test repository needs `sparrowfile` or `main.raku`:
@@ -140,183 +165,66 @@ task-run 'check-sshd', %(
plugin => 'systemd-service',
args => ['sshd', 'running']
);
task-run 'verify-rocky-version', %(
plugin => 'shell-command',
args => 'cat /etc/rocky-release'
);
```
## Configuration
### Golden Image Users
### Ansible
- **rocky**: Primary test user (password: `rockypass`, sudo: yes)
- **root**: Root user (password: `rockytesting`)
**Provider Groups:**
- `inventory/hosts.yml` - Provider groups (libvirt, aws, azure)
- `inventory/group_vars/all.yml` - Settings for all providers
- `inventory/group_vars/libvirt.yml` - Libvirt-specific (VM resources, packages)
- `inventory/group_vars/aws.yml` - AWS-specific (future)
- `inventory/group_vars/azure.yml` - Azure-specific (future)
Both have SSH keys injected from `~/.ssh/id_rsa.pub`.
**Test Definitions:**
- `vars/test-definitions.yml` - Centralized test repository list
### VM Resources
Default per VM (edit `provision_vm.sh` to change):
- Memory: 2048 MB
- CPUs: 2
- Disk: Linked clone (only diffs stored)
## Portability
All logic is in standalone shell scripts. Easy to integrate with:
**GitHub Actions:**
```yaml
- name: Run test
run: ./scripts/run-test.sh my-test $REPO $GOLDEN
```
**GitLab CI:**
```yaml
test:
script:
- ./scripts/run-test.sh my-test $REPO $GOLDEN
```
**Windmill:**
```typescript
await bash.run(`./scripts/run-test.sh ${name} ${repo} ${golden}`)
```
**Manually:**
**Example override:**
```bash
./scripts/run-test.sh my-test https://github.com/org/test.git golden.qcow2
ansible-playbook playbooks/run-tests.yml -e "vm_memory=4096"
```
### Shell Scripts
Pass parameters directly:
```bash
./scripts/provision_vm.sh <vm_name> <golden_image> [timeout]
```
## Documentation
- [Ansible Guide](docs/ANSIBLE-GUIDE.md) - Complete Ansible documentation
- [Bootstrap Approach](docs/BOOTSTRAP-APPROACH.md) - How golden image works
## Key Features
### Ansible Implementation
- **Provider-aware** - Supports libvirt, AWS, Azure (cloud providers in progress)
- **Interactive prompts** - No need to remember image URLs
- **Test definitions** - Centralized test list in `vars/test-definitions.yml`
- **Result tracking** - Shows passed/failed tests with summary report
- **Automatic cleanup** - Infrastructure cleaned up after each test
## Troubleshooting
### List VMs
```bash
# List VMs
virsh -c qemu:///system list --all
```
### Get VM IP
```bash
# Get VM IP
virsh -c qemu:///system domifaddr <vm-name>
```
### SSH to VM
```bash
ssh -i ~/.ssh/id_rsa rocky@<vm-ip>
```
### View Console
```bash
virsh -c qemu:///system console <vm-name>
# Press Ctrl+] to exit
```
### Check Network
```bash
virsh -c qemu:///system net-list --all
virsh -c qemu:///system net-start default # If stopped
```
### Force Cleanup
```bash
./scripts/cleanup-all.sh
rm -f /var/lib/libvirt/images/golden-*.qcow2
```
### Bootstrap Fails
```bash
# Check if VM is accessible
ssh -i ~/.ssh/id_rsa rocky@<vm-ip>
# Manually bootstrap
sparrowdo --host <vm-ip> --ssh_user rocky --bootstrap --color
```
## Performance
### Image Caching
- First download: ~5-10 minutes (2GB)
- Subsequent builds: 2 seconds (cached)
### Golden Image Build
- Base preparation: ~2-3 minutes (virt-customize)
- Bootstrap: ~5-10 minutes (sparrowdo --bootstrap)
- Total: ~7-13 minutes (once per build)
### Test Execution
- VM provision: ~30 seconds (boot + IP)
- Test runtime: Varies by test
- VM cleanup: ~2 seconds
### Example: 70 Tests
- Golden image: 10 minutes (once)
- 70 tests @ 3 concurrent: ~15 minutes
- **Total: 25 minutes**
Compare to bootstrapping each VM: 70 × 7 min = 490 minutes (8+ hours)!
## Advanced Usage
### Custom Golden Image
```bash
# Create custom prep script
cat > custom-prep.sh << 'EOF'
#!/bin/bash
dnf install -y postgresql-server nginx
systemctl enable postgresql nginx
EOF
# Use virt-customize directly
sudo virt-customize -a golden.qcow2 \
--run custom-prep.sh \
--selinux-relabel
```
### Testing Beta Images
```bash
BASE=$(./scripts/download-image.sh \
https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud-Base-9.5-beta.x86_64.qcow2 \
/var/lib/libvirt/images true) # Force re-download
```
### Running Subset of Tests
```bash
# In Jenkins, edit TEST_REPOS parameter
https://github.com/org/test1.git
https://github.com/org/test3.git
# (omit test2)
```
### Debugging Test Failures
```bash
# Run test manually and keep VM alive
VM_NAME="debug-vm-$$"
VM_IP=$(./scripts/provision_vm.sh "$VM_NAME" golden.qcow2 60 | tail -1)
# SSH to VM
ssh -i ~/.ssh/id_rsa rocky@$VM_IP
ssh -i ~/.ssh/id_rsa rocky@<vm-ip>
# Manually run test steps
cd /tmp
git clone https://github.com/org/test.git
cd test
sparrowdo --host localhost --ssh_user rocky --no_sudo --sparrowfile main.raku
# Ansible verbose mode
ansible-playbook playbooks/run-tests.yml -vvv
# Cleanup when done
./scripts/cleanup_vm.sh "$VM_NAME"
# Run specific test
ansible-playbook playbooks/run-tests.yml -e "test_filter=Sparky_Knot"
```
## Contributing
This framework is designed to be simple and portable. Keep it that way:
- Scripts should be standalone bash
- Minimal dependencies
- Easy to read, no over-commenting
- Portable across CI/CD platforms
## License
[Your License Here]