resf-testing-repo/docs/virt-customize-guide.md

virt-customize Guide for Golden Image Preparation

Overview

The virt-customize tool modifies QCOW2 images offline (without booting them). This means certain commands that require a running system won't work.

What Works in virt-customize

✅ Package Management

# Install packages
dnf install -y perl git wget

# Update packages (works but can be slow)
dnf update -y

# Remove packages
dnf remove -y packagename

✅ File Operations

# Create/modify files
echo "content" > /etc/myfile
cat > /etc/config << EOF
config_line=value
EOF

# Copy files
cp /source /destination

# Set permissions
chmod 755 /path/to/file
chown user:group /path/to/file

✅ User Management

# Create users
useradd -m username

# Set passwords
echo "user:password" | chpasswd

# Modify sudoers
echo "user ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/user
chmod 0440 /etc/sudoers.d/user

✅ Enable Services (Offline Mode)

# systemctl enable creates symlinks - works offline!
systemctl enable sshd
systemctl enable chronyd
systemctl enable httpd

What DOESN'T Work

❌ D-Bus Dependent Commands

These commands require D-Bus and will fail with:

DBUS_ERROR: Failed to connect to socket /run/dbus/system_bus_socket

Examples:

# ❌ Don't use these in prep scripts:
systemctl start sshd      # Requires running system
systemctl restart sshd    # Requires running system
systemctl status sshd     # Requires running system
firewall-cmd --add-service=ssh  # Requires firewalld running
hostnamectl set-hostname test   # Requires D-Bus
timedatectl set-timezone UTC    # Requires D-Bus
localectl set-locale LANG=en_US # Requires D-Bus

Workarounds for Common Tasks

Setting Hostname

Instead of: hostnamectl set-hostname test-node

Use virt-customize command line:

sudo virt-customize -a image.qcow2 --hostname test-node

Or in script:

echo "test-node" > /etc/hostname

Firewall Configuration

Instead of: firewall-cmd --add-service=ssh

Use virt-customize command line:

sudo virt-customize -a image.qcow2 \
    --run-command 'firewall-offline-cmd --add-service=ssh'

Or modify firewalld config directly:

# In prep script - add SSH to default zone
cat > /etc/firewalld/zones/public.xml << 'EOF'
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
</zone>
EOF

Timezone Configuration

Instead of: timedatectl set-timezone America/New_York

Use:

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime

SELinux Relabeling

Always relabel after file modifications:

# In virt-customize command (not in script):
sudo virt-customize -a image.qcow2 \
    --run custom-prep.sh \
    --selinux-relabel

Example: Good Prep Script

#!/bin/bash
set -e

echo "Preparing golden image..."

# ✅ Install packages
dnf install -y \
    perl \
    git \
    openssh-server

# ✅ Enable services (creates symlinks only)
systemctl enable sshd

# ✅ Create users
useradd -m testuser
echo "testuser:testpass" | chpasswd

# ✅ Configure sudoers
echo "testuser ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/testuser
chmod 0440 /etc/sudoers.d/testuser

# ✅ Set timezone (manual symlink)
ln -sf /usr/share/zoneinfo/UTC /etc/localtime

# ✅ Configure network (direct file edit)
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 << 'EOF'
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
EOF

echo "Preparation complete!"

Example: Bad Prep Script (Will Fail)

#!/bin/bash
set -e

# ❌ These will all fail:
systemctl start sshd                        # No running systemd
systemctl restart network                   # No running systemd
firewall-cmd --add-service=ssh              # No D-Bus
hostnamectl set-hostname test-node          # No D-Bus
timedatectl set-timezone America/New_York   # No D-Bus

Using setup_base.sh Script

The setup_base.sh script uses both:

1. Your custom prep script (runs inside image)
2. virt-customize command-line options (runs outside)

Example call:

./scripts/setup_base.sh \
    /path/to/base.qcow2 \
    /path/to/prep-script.sh \
    /path/to/golden.qcow2 \
    ~/.ssh/id_rsa.pub

This automatically:

• Runs your prep script inside the image
• Injects SSH key (via --ssh-inject)
• Sets root password (via --root-password)
• Relabels SELinux (via --selinux-relabel)

Advanced: Direct virt-customize Commands

For complex setups, skip the prep script and use virt-customize directly:

sudo virt-customize -a golden.qcow2 \
    --install 'perl,git,wget,openssh-server' \
    --run-command 'systemctl enable sshd' \
    --run-command 'useradd -m testuser' \
    --run-command 'echo "testuser:testpass" | chpasswd' \
    --write '/etc/sudoers.d/testuser:testuser ALL=(ALL) NOPASSWD:ALL' \
    --chmod '0440:/etc/sudoers.d/testuser' \
    --ssh-inject root:file:~/.ssh/id_rsa.pub \
    --root-password password:rockytesting \
    --hostname test-node \
    --timezone UTC \
    --selinux-relabel

Debugging virt-customize Issues

If your prep script fails, run with debugging:

export LIBGUESTFS_DEBUG=1
export LIBGUESTFS_TRACE=1

sudo virt-customize -v -x \
    -a golden.qcow2 \
    --run prep-script.sh \
    --selinux-relabel

This will show:

• Exact commands being run
• Output from each command
• Error messages with full context

Best Practices

1. Keep prep scripts simple - Install packages and create users, that's it
2. Use virt-customize options - For hostname, timezone, SSH keys
3. Test incrementally - Add one command at a time to find issues
4. Avoid system state - Don't start services or query running processes
5. Always relabel SELinux - After modifying files
6. Comment out dnf update - It's slow; only use when needed

Common Errors and Solutions

Error: "DBUS_ERROR: Failed to connect to socket"

Problem: Script uses D-Bus dependent command Solution: Remove firewall-cmd, hostnamectl, timedatectl commands

Error: "systemctl: command not found"

Problem: Minimal image doesn't have systemd Solution: Check base image has systemd, or install it

Error: "SELinux is preventing..."

Problem: Files created without proper SELinux context Solution: Add --selinux-relabel to virt-customize command

Error: "dnf: command not found"

Problem: Image uses yum not dnf (older RHEL/CentOS) Solution: Use yum instead of dnf in scripts

Resources

• virt-customize man page
• libguestfs FAQ
• Rocky Linux Cloud Image docs