276 lines
6.6 KiB
Markdown
276 lines
6.6 KiB
Markdown
# virt-customize Guide for Golden Image Preparation
|
|
|
|
## Overview
|
|
|
|
The `virt-customize` tool modifies QCOW2 images **offline** (without booting them). This means certain commands that require a running system won't work.
|
|
|
|
## What Works in virt-customize
|
|
|
|
### ✅ Package Management
|
|
```bash
|
|
# Install packages
|
|
dnf install -y perl git wget
|
|
|
|
# Update packages (works but can be slow)
|
|
dnf update -y
|
|
|
|
# Remove packages
|
|
dnf remove -y packagename
|
|
```
|
|
|
|
### ✅ File Operations
|
|
```bash
|
|
# Create/modify files
|
|
echo "content" > /etc/myfile
|
|
cat > /etc/config << EOF
|
|
config_line=value
|
|
EOF
|
|
|
|
# Copy files
|
|
cp /source /destination
|
|
|
|
# Set permissions
|
|
chmod 755 /path/to/file
|
|
chown user:group /path/to/file
|
|
```
|
|
|
|
### ✅ User Management
|
|
```bash
|
|
# Create users
|
|
useradd -m username
|
|
|
|
# Set passwords
|
|
echo "user:password" | chpasswd
|
|
|
|
# Modify sudoers
|
|
echo "user ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/user
|
|
chmod 0440 /etc/sudoers.d/user
|
|
```
|
|
|
|
### ✅ Enable Services (Offline Mode)
|
|
```bash
|
|
# systemctl enable creates symlinks - works offline!
|
|
systemctl enable sshd
|
|
systemctl enable chronyd
|
|
systemctl enable httpd
|
|
```
|
|
|
|
## What DOESN'T Work
|
|
|
|
### ❌ D-Bus Dependent Commands
|
|
|
|
These commands require D-Bus and will fail with:
|
|
```
|
|
DBUS_ERROR: Failed to connect to socket /run/dbus/system_bus_socket
|
|
```
|
|
|
|
**Examples:**
|
|
```bash
|
|
# ❌ Don't use these in prep scripts:
|
|
systemctl start sshd # Requires running system
|
|
systemctl restart sshd # Requires running system
|
|
systemctl status sshd # Requires running system
|
|
firewall-cmd --add-service=ssh # Requires firewalld running
|
|
hostnamectl set-hostname test # Requires D-Bus
|
|
timedatectl set-timezone UTC # Requires D-Bus
|
|
localectl set-locale LANG=en_US # Requires D-Bus
|
|
```
|
|
|
|
## Workarounds for Common Tasks
|
|
|
|
### Setting Hostname
|
|
**Instead of:** `hostnamectl set-hostname test-node`
|
|
|
|
**Use virt-customize command line:**
|
|
```bash
|
|
sudo virt-customize -a image.qcow2 --hostname test-node
|
|
```
|
|
|
|
**Or in script:**
|
|
```bash
|
|
echo "test-node" > /etc/hostname
|
|
```
|
|
|
|
### Firewall Configuration
|
|
**Instead of:** `firewall-cmd --add-service=ssh`
|
|
|
|
**Use virt-customize command line:**
|
|
```bash
|
|
sudo virt-customize -a image.qcow2 \
|
|
--run-command 'firewall-offline-cmd --add-service=ssh'
|
|
```
|
|
|
|
**Or modify firewalld config directly:**
|
|
```bash
|
|
# In prep script - add SSH to default zone
|
|
cat > /etc/firewalld/zones/public.xml << 'EOF'
|
|
<?xml version="1.0" encoding="utf-8"?>
|
|
<zone>
|
|
<short>Public</short>
|
|
<service name="ssh"/>
|
|
<service name="dhcpv6-client"/>
|
|
</zone>
|
|
EOF
|
|
```
|
|
|
|
### Timezone Configuration
|
|
**Instead of:** `timedatectl set-timezone America/New_York`
|
|
|
|
**Use:**
|
|
```bash
|
|
ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
|
|
```
|
|
|
|
### SELinux Relabeling
|
|
**Always relabel after file modifications:**
|
|
```bash
|
|
# In virt-customize command (not in script):
|
|
sudo virt-customize -a image.qcow2 \
|
|
--run custom-prep.sh \
|
|
--selinux-relabel
|
|
```
|
|
|
|
## Example: Good Prep Script
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
set -e
|
|
|
|
echo "Preparing golden image..."
|
|
|
|
# ✅ Install packages
|
|
dnf install -y \
|
|
perl \
|
|
git \
|
|
openssh-server
|
|
|
|
# ✅ Enable services (creates symlinks only)
|
|
systemctl enable sshd
|
|
|
|
# ✅ Create users
|
|
useradd -m testuser
|
|
echo "testuser:testpass" | chpasswd
|
|
|
|
# ✅ Configure sudoers
|
|
echo "testuser ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/testuser
|
|
chmod 0440 /etc/sudoers.d/testuser
|
|
|
|
# ✅ Set timezone (manual symlink)
|
|
ln -sf /usr/share/zoneinfo/UTC /etc/localtime
|
|
|
|
# ✅ Configure network (direct file edit)
|
|
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 << 'EOF'
|
|
DEVICE=eth0
|
|
BOOTPROTO=dhcp
|
|
ONBOOT=yes
|
|
EOF
|
|
|
|
echo "Preparation complete!"
|
|
```
|
|
|
|
## Example: Bad Prep Script (Will Fail)
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
set -e
|
|
|
|
# ❌ These will all fail:
|
|
systemctl start sshd # No running systemd
|
|
systemctl restart network # No running systemd
|
|
firewall-cmd --add-service=ssh # No D-Bus
|
|
hostnamectl set-hostname test-node # No D-Bus
|
|
timedatectl set-timezone America/New_York # No D-Bus
|
|
```
|
|
|
|
## Using setup_base.sh Script
|
|
|
|
The `setup_base.sh` script uses both:
|
|
1. Your custom prep script (runs inside image)
|
|
2. virt-customize command-line options (runs outside)
|
|
|
|
**Example call:**
|
|
```bash
|
|
./scripts/setup_base.sh \
|
|
/path/to/base.qcow2 \
|
|
/path/to/prep-script.sh \
|
|
/path/to/golden.qcow2 \
|
|
~/.ssh/id_rsa.pub
|
|
```
|
|
|
|
This automatically:
|
|
- Runs your prep script inside the image
|
|
- Injects SSH key (via `--ssh-inject`)
|
|
- Sets root password (via `--root-password`)
|
|
- Relabels SELinux (via `--selinux-relabel`)
|
|
|
|
## Advanced: Direct virt-customize Commands
|
|
|
|
For complex setups, skip the prep script and use virt-customize directly:
|
|
|
|
```bash
|
|
sudo virt-customize -a golden.qcow2 \
|
|
--install 'perl,git,wget,openssh-server' \
|
|
--run-command 'systemctl enable sshd' \
|
|
--run-command 'useradd -m testuser' \
|
|
--run-command 'echo "testuser:testpass" | chpasswd' \
|
|
--write '/etc/sudoers.d/testuser:testuser ALL=(ALL) NOPASSWD:ALL' \
|
|
--chmod '0440:/etc/sudoers.d/testuser' \
|
|
--ssh-inject root:file:~/.ssh/id_rsa.pub \
|
|
--root-password password:rockytesting \
|
|
--hostname test-node \
|
|
--timezone UTC \
|
|
--selinux-relabel
|
|
```
|
|
|
|
## Debugging virt-customize Issues
|
|
|
|
If your prep script fails, run with debugging:
|
|
|
|
```bash
|
|
export LIBGUESTFS_DEBUG=1
|
|
export LIBGUESTFS_TRACE=1
|
|
|
|
sudo virt-customize -v -x \
|
|
-a golden.qcow2 \
|
|
--run prep-script.sh \
|
|
--selinux-relabel
|
|
```
|
|
|
|
This will show:
|
|
- Exact commands being run
|
|
- Output from each command
|
|
- Error messages with full context
|
|
|
|
## Best Practices
|
|
|
|
1. **Keep prep scripts simple** - Install packages and create users, that's it
|
|
2. **Use virt-customize options** - For hostname, timezone, SSH keys
|
|
3. **Test incrementally** - Add one command at a time to find issues
|
|
4. **Avoid system state** - Don't start services or query running processes
|
|
5. **Always relabel SELinux** - After modifying files
|
|
6. **Comment out dnf update** - It's slow; only use when needed
|
|
|
|
## Common Errors and Solutions
|
|
|
|
### Error: "DBUS_ERROR: Failed to connect to socket"
|
|
**Problem:** Script uses D-Bus dependent command
|
|
**Solution:** Remove firewall-cmd, hostnamectl, timedatectl commands
|
|
|
|
### Error: "systemctl: command not found"
|
|
**Problem:** Minimal image doesn't have systemd
|
|
**Solution:** Check base image has systemd, or install it
|
|
|
|
### Error: "SELinux is preventing..."
|
|
**Problem:** Files created without proper SELinux context
|
|
**Solution:** Add `--selinux-relabel` to virt-customize command
|
|
|
|
### Error: "dnf: command not found"
|
|
**Problem:** Image uses yum not dnf (older RHEL/CentOS)
|
|
**Solution:** Use `yum` instead of `dnf` in scripts
|
|
|
|
## Resources
|
|
|
|
- [virt-customize man page](http://libguestfs.org/virt-customize.1.html)
|
|
- [libguestfs FAQ](http://libguestfs.org/guestfs-faq.1.html)
|
|
- Rocky Linux Cloud Image docs
|