rocky-man/README.md

Rocky Man 📚

Rocky Man is a tool for generating searchable HTML documentation from Rocky Linux man pages across BaseOS and AppStream repositories for Rocky Linux 8, 9, and 10.

Features

• Fast & Efficient: Uses filelists.xml to pre-filter packages with man pages
• Complete Coverage: All packages from BaseOS and AppStream repositories
• Container Ready: Works on x86_64, aarch64, arm64, etc.
• Smart Cleanup: Automatic cleanup of temporary files (configurable)
• Parallel Processing: Concurrent downloads and conversions for maximum speed
• Multi-version: Support for Rocky Linux 8, 9, and 10 simultaneously

Quick Start

Podman (Recommended)

# Build the image
podman build -t rocky-man .

# Generate man pages for Rocky Linux 9.6 (using defaults, no custom args)
podman run --rm -v $(pwd)/html:/data/html:Z rocky-man

# Generate for specific versions (requires explicit paths)
podman run --rm -v $(pwd)/html:/app/html:Z rocky-man \
  --versions 8.10 9.6 10.0 --output-dir /app/html

# With verbose logging
podman run --rm -v $(pwd)/html:/app/html:Z rocky-man \
  --versions 9.6 --output-dir /app/html --verbose

# Keep downloaded RPMs (mount the download directory)
podman run --rm -it \
  -v $(pwd)/html:/app/html:Z \
  -v $(pwd)/downloads:/app/tmp/downloads:Z \
  rocky-man --versions 9.6 --keep-rpms \
  --output-dir /app/html --download-dir /app/tmp/downloads --verbose

Docker

# Build the image
docker build -t rocky-man .

# Generate man pages (using defaults, no custom args)
docker run --rm -v $(pwd)/html:/data/html rocky-man

# Generate for specific versions (requires explicit paths)
docker run --rm -v $(pwd)/html:/app/html rocky-man \
  --versions 9.6 --output-dir /app/html

# Interactive mode for debugging
docker run --rm -it -v $(pwd)/html:/app/html rocky-man \
  --versions 9.6 --output-dir /app/html --verbose

# Keep downloaded RPMs (mount the download directory)
docker run --rm -it \
  -v $(pwd)/html:/app/html \
  -v $(pwd)/downloads:/app/tmp/downloads \
  rocky-man --versions 9.6 --keep-rpms \
  --output-dir /app/html --download-dir /app/tmp/downloads --verbose

Directory Structure in Container

The container uses different paths depending on whether you pass custom arguments:

Without custom arguments (using Dockerfile CMD defaults):

• /data/html - Generated HTML output
• /data/tmp/downloads - Downloaded RPM files
• /data/tmp/extracts - Extracted man page files

With custom arguments (argparse defaults from working directory /app):

• /app/html - Generated HTML output
• /app/tmp/downloads - Downloaded RPM files
• /app/tmp/extracts - Extracted man page files

Important: When passing custom arguments, the container's CMD is overridden and the code falls back to relative paths (./html = /app/html). You must explicitly specify --output-dir /app/html --download-dir /app/tmp/downloads to match your volume mounts. Without this, files are written inside the container and lost when it stops (especially with --rm).

Local Development

Prerequisites

• Python 3.9+
• pip (Python package manager)
• mandoc (man page converter)
• Rocky Linux system or container (for DNF)

Installation

# On Rocky Linux, install system dependencies
dnf install -y python3 python3-pip python3-dnf mandoc rpm-build dnf-plugins-core

# Install Python dependencies
pip3 install -e .

Usage

# Generate man pages for Rocky 9.6
python -m rocky_man.main --versions 9.6

# Generate for multiple versions (default)
python -m rocky_man.main --versions 8.10 9.6 10.0

# Custom output directory
python -m rocky_man.main --output-dir /var/www/html/man --versions 9.6

# Keep downloaded RPMs for debugging
python -m rocky_man.main --keep-rpms --verbose

# Adjust parallelism for faster processing
python -m rocky_man.main --parallel-downloads 10 --parallel-conversions 20

# Use a different mirror
python -m rocky_man.main --mirror https://mirrors.example.com/

# Only BaseOS (faster)
python -m rocky_man.main --repo-types BaseOS --versions 9.6

Architecture

Rocky Man is organized into clean, modular components:

rocky-man/
├── src/rocky_man/
│   ├── models/              # Data models (Package, ManFile)
│   ├── repo/                # Repository management
│   ├── processor/           # Man page processing
│   ├── web/                 # Web page generation
│   ├── utils/               # Utilities
│   └── main.py              # Main entry point and orchestration
├── templates/               # Jinja2 templates
├── Dockerfile               # Multi-stage, arch-independent
└── pyproject.toml           # Python project configuration

How It Works

1. Package Discovery - Parse repository filelists.xml to identify packages with man pages
2. Smart Download - Download only packages containing man pages with parallel downloads
3. Extraction - Extract man page files from RPM packages
4. Conversion - Convert troff format to HTML using mandoc
5. Web Generation - Wrap HTML in templates and generate search index
6. Cleanup - Automatically remove temporary files (configurable)

Command Line Options

usage: rocky-man [-h] [--versions VERSIONS [VERSIONS ...]]
                 [--repo-types REPO_TYPES [REPO_TYPES ...]]
                 [--output-dir OUTPUT_DIR] [--download-dir DOWNLOAD_DIR]
                 [--extract-dir EXTRACT_DIR] [--keep-rpms] [--keep-extracts]
                 [--parallel-downloads N] [--parallel-conversions N]
                 [--mirror URL] [--template-dir DIR] [-v]

Generate HTML documentation for Rocky Linux man pages

optional arguments:
  -h, --help            show this help message and exit
  --versions VERSIONS [VERSIONS ...]
                        Rocky Linux versions to process (default: 8.10 9.6 10.0)
  --repo-types REPO_TYPES [REPO_TYPES ...]
                        Repository types to process (default: BaseOS AppStream)
  --output-dir OUTPUT_DIR
                        Output directory for HTML files (default: ./html)
  --download-dir DOWNLOAD_DIR
                        Directory for downloading packages (default: ./tmp/downloads)
  --extract-dir EXTRACT_DIR
                        Directory for extracting man pages (default: ./tmp/extracts)
  --keep-rpms           Keep downloaded RPM files after processing
  --keep-extracts       Keep extracted man files after processing
  --parallel-downloads PARALLEL_DOWNLOADS
                        Number of parallel downloads (default: 5)
  --parallel-conversions PARALLEL_CONVERSIONS
                        Number of parallel HTML conversions (default: 10)
  --mirror MIRROR       Rocky Linux mirror URL (default: http://dl.rockylinux.org/)
  --vault               Use vault directory instead of pub (vault/rocky instead of pub/rocky)
  --existing-versions [VERSION ...]
                        List of existing versions to include in root index (e.g., 8.10 9.7)
  --template-dir TEMPLATE_DIR
                        Template directory (default: ./templates)
  -v, --verbose         Enable verbose logging
  --skip-sections [SKIP_SECTIONS ...]
                        Man sections to skip (default: 3 3p 3pm). Use empty list to skip none.
  --skip-packages [SKIP_PACKAGES ...]
                        Package names to skip (default: lapack dpdk-devel gl-manpages). Use empty list to skip none.
  --skip-languages      Skip non-English man pages (default: enabled)
  --keep-languages      Keep all languages (disables --skip-languages)
  --allow-all-sections  Include all man sections (overrides --skip-sections)

Troubleshooting

DNF Errors

Problem: dnf module not found or repository errors

Solution: Ensure you're running on Rocky Linux or in a Rocky Linux container:

# Run in Rocky Linux container
podman run --rm -it -v $(pwd):/app rockylinux:9 /bin/bash
cd /app

# Install dependencies
dnf install -y python3 python3-dnf mandoc rpm-build dnf-plugins-core

# Run the script
python3 -m rocky_man.main --versions 9.6

Mandoc Not Found

Problem: mandoc: command not found

Solution: Install mandoc:

dnf install -y mandoc

Permission Errors in Container

Problem: Cannot write to mounted volume

Solution: Use the :Z flag with podman for SELinux contexts:

podman run --rm -v $(pwd)/html:/data/html:Z rocky-man

For Docker, ensure the volume path is absolute:

docker run --rm -v "$(pwd)/html":/data/html rocky-man

Out of Memory

Problem: Process killed due to memory

Solution: Reduce parallelism:

python -m rocky_man.main --parallel-downloads 2 --parallel-conversions 5

Slow Downloads

Problem: Downloads are very slow

Solution: Use a closer mirror:

# Find mirrors at: https://mirrors.rockylinux.org/mirrormanager/mirrors
python -m rocky_man.main --mirror https://mirror.example.com/rocky/

Performance Tips

1. Use closer mirrors - Significant speed improvement for downloads
2. Increase parallelism - If you have bandwidth: --parallel-downloads 15
3. Process one repo at a time - Use --repo-types BaseOS first, then --repo-types AppStream
4. Keep RPMs for re-runs - Use --keep-rpms if testing
5. Run in container - More consistent performance

License

This project is licensed under the MIT License - see the LICENSE file for details.

Third-Party Software

This project uses several open source components. See THIRD-PARTY-LICENSES.md for complete license information and attributions.

Trademark Notice

Rocky Linux™ is a trademark of the Rocky Enterprise Software Foundation (RESF). This project is not officially affiliated with or endorsed by RESF. All trademarks are the property of their respective owners. This project complies with RESF's trademark usage guidelines.

Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes with proper documentation
4. Test thoroughly
5. Commit with clear messages (git commit -m 'feat: add amazing feature')
6. Push to your branch (git push origin feature/amazing-feature)
7. Open a Pull Request