rocky-man/README.md

# Rocky Man 📚

**Rocky Man** is a tool for generating searchable HTML documentation from Rocky Linux man pages across BaseOS and AppStream repositories for Rocky Linux 8, 9, and 10.

## Features

- **Fast & Efficient**: Uses filelists.xml to pre-filter packages with man pages
- **Complete Coverage**: All packages from BaseOS and AppStream repositories
- **Container Ready**: Works on x86_64, aarch64, arm64, etc.
- **Smart Cleanup**: Automatic cleanup of temporary files (configurable)
- **Parallel Processing**: Concurrent downloads and conversions for maximum speed
- **Multi-version**: Support for Rocky Linux 8, 9, and 10 simultaneously

## Quick Start

### Podman (Recommended)

```bash
# Build the image
podman build -t rocky-man .

# Generate man pages for Rocky Linux 9.6 (using defaults, no custom args)
podman run --rm -v $(pwd)/html:/data/html:Z rocky-man

# Generate for specific versions (requires explicit paths)
podman run --rm -v $(pwd)/html:/app/html:Z rocky-man \
  --versions 8.10 9.6 10.0 --output-dir /app/html

# With verbose logging
podman run --rm -v $(pwd)/html:/app/html:Z rocky-man \
  --versions 9.6 --output-dir /app/html --verbose

# Keep downloaded RPMs (mount the download directory)
podman run --rm -it \
  -v $(pwd)/html:/app/html:Z \
  -v $(pwd)/downloads:/app/tmp/downloads:Z \
  rocky-man --versions 9.6 --keep-rpms \
  --output-dir /app/html --download-dir /app/tmp/downloads --verbose
```

### Docker

```bash
# Build the image
docker build -t rocky-man .

# Generate man pages (using defaults, no custom args)
docker run --rm -v $(pwd)/html:/data/html rocky-man

# Generate for specific versions (requires explicit paths)
docker run --rm -v $(pwd)/html:/app/html rocky-man \
  --versions 9.6 --output-dir /app/html

# Interactive mode for debugging
docker run --rm -it -v $(pwd)/html:/app/html rocky-man \
  --versions 9.6 --output-dir /app/html --verbose

# Keep downloaded RPMs (mount the download directory)
docker run --rm -it \
  -v $(pwd)/html:/app/html \
  -v $(pwd)/downloads:/app/tmp/downloads \
  rocky-man --versions 9.6 --keep-rpms \
  --output-dir /app/html --download-dir /app/tmp/downloads --verbose
```

### Directory Structure in Container

The container uses different paths depending on whether you pass custom arguments:

**Without custom arguments** (using Dockerfile CMD defaults):
- `/data/html` - Generated HTML output
- `/data/tmp/downloads` - Downloaded RPM files
- `/data/tmp/extracts` - Extracted man page files

**With custom arguments** (argparse defaults from working directory `/app`):
- `/app/html` - Generated HTML output
- `/app/tmp/downloads` - Downloaded RPM files
- `/app/tmp/extracts` - Extracted man page files

**Important**: When passing custom arguments, the container's CMD is overridden and the code falls back to relative paths (`./html` = `/app/html`). You must explicitly specify `--output-dir /app/html --download-dir /app/tmp/downloads` to match your volume mounts. Without this, files are written inside the container and lost when it stops (especially with `--rm`).

### Local Development

#### Prerequisites

- Python 3.9+
- pip (Python package manager)
- mandoc (man page converter)
- Rocky Linux system or container (for DNF)

#### Installation

```bash
# On Rocky Linux, install system dependencies
dnf install -y python3 python3-pip python3-dnf mandoc rpm-build dnf-plugins-core

# Install Python dependencies
pip3 install -e .
```

#### Usage

```bash
# Generate man pages for Rocky 9.6
python -m rocky_man.main --versions 9.6

# Generate for multiple versions (default)
python -m rocky_man.main --versions 8.10 9.6 10.0

# Custom output directory
python -m rocky_man.main --output-dir /var/www/html/man --versions 9.6

# Keep downloaded RPMs for debugging
python -m rocky_man.main --keep-rpms --verbose

# Adjust parallelism for faster processing
python -m rocky_man.main --parallel-downloads 10 --parallel-conversions 20

# Use a different mirror
python -m rocky_man.main --mirror https://mirrors.example.com/

# Only BaseOS (faster)
python -m rocky_man.main --repo-types BaseOS --versions 9.6
```

## Architecture

Rocky Man is organized into clean, modular components:

```
rocky-man/
├── src/rocky_man/
│   ├── models/              # Data models (Package, ManFile)
│   ├── repo/                # Repository management
│   ├── processor/           # Man page processing
│   ├── web/                 # Web page generation
│   ├── utils/               # Utilities
│   └── main.py              # Main entry point and orchestration
├── templates/               # Jinja2 templates
├── Dockerfile               # Multi-stage, arch-independent
└── pyproject.toml           # Python project configuration
```

### How It Works

1. **Package Discovery** - Parse repository `filelists.xml` to identify packages with man pages
2. **Smart Download** - Download only packages containing man pages with parallel downloads
3. **Extraction** - Extract man page files from RPM packages
4. **Conversion** - Convert troff format to HTML using mandoc
5. **Web Generation** - Wrap HTML in templates and generate search index
6. **Cleanup** - Automatically remove temporary files (configurable)

## Command Line Options

```
usage: rocky-man [-h] [--versions VERSIONS [VERSIONS ...]]
                 [--repo-types REPO_TYPES [REPO_TYPES ...]]
                 [--output-dir OUTPUT_DIR] [--download-dir DOWNLOAD_DIR]
                 [--extract-dir EXTRACT_DIR] [--keep-rpms] [--keep-extracts]
                 [--parallel-downloads N] [--parallel-conversions N]
                 [--mirror URL] [--template-dir DIR] [-v]

Generate HTML documentation for Rocky Linux man pages

Options:
  -h, --help            Show this help message and exit

  --versions VERSIONS [VERSIONS ...]
                        Rocky Linux versions to process (default: 8.10 9.6 10.0)

  --repo-types REPO_TYPES [REPO_TYPES ...]
                        Repository types to process (default: BaseOS AppStream)

  --output-dir OUTPUT_DIR
                        HTML output directory (default: ./html)

  --download-dir DOWNLOAD_DIR
                        Package download directory (default: ./tmp/downloads)

  --extract-dir EXTRACT_DIR
                        Extraction directory (default: ./tmp/extracts)

  --keep-rpms           Keep downloaded RPM files after processing

  --keep-extracts       Keep extracted man files after processing

  --parallel-downloads N
                        Number of parallel downloads (default: 5)

  --parallel-conversions N
                        Number of parallel HTML conversions (default: 10)

  --mirror URL          Rocky Linux mirror URL
                        (default: http://dl.rockylinux.org/)

  --template-dir DIR    Custom template directory

  -v, --verbose         Enable verbose logging
```

## Troubleshooting

### DNF Errors

**Problem**: `dnf` module not found or repository errors

**Solution**: Ensure you're running on Rocky Linux or in a Rocky Linux container:

```bash
# Run in Rocky Linux container
podman run --rm -it -v $(pwd):/app rockylinux:9 /bin/bash
cd /app

# Install dependencies
dnf install -y python3 python3-dnf mandoc rpm-build dnf-plugins-core

# Run the script
python3 -m rocky_man.main --versions 9.6
```

### Mandoc Not Found

**Problem**: `mandoc: command not found`

**Solution**: Install mandoc:

```bash
dnf install -y mandoc
```

### Permission Errors in Container

**Problem**: Cannot write to mounted volume

**Solution**: Use the `:Z` flag with podman for SELinux contexts:

```bash
podman run --rm -v $(pwd)/html:/data/html:Z rocky-man
```

For Docker, ensure the volume path is absolute:

```bash
docker run --rm -v "$(pwd)/html":/data/html rocky-man
```

### Out of Memory

**Problem**: Process killed due to memory

**Solution**: Reduce parallelism:

```bash
python -m rocky_man.main --parallel-downloads 2 --parallel-conversions 5
```

### Slow Downloads

**Problem**: Downloads are very slow

**Solution**: Use a closer mirror:

```bash
# Find mirrors at: https://mirrors.rockylinux.org/mirrormanager/mirrors
python -m rocky_man.main --mirror https://mirror.example.com/rocky/
```

## Performance Tips

1. **Use closer mirrors** - Significant speed improvement for downloads
2. **Increase parallelism** - If you have bandwidth: `--parallel-downloads 15`
3. **Process one repo at a time** - Use `--repo-types BaseOS` first, then `--repo-types AppStream`
4. **Keep RPMs for re-runs** - Use `--keep-rpms` if testing
5. **Run in container** - More consistent performance

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

### Third-Party Software

This project uses several open source components. See [THIRD-PARTY-LICENSES.md](THIRD-PARTY-LICENSES.md) for complete license information and attributions.

### Trademark Notice

Rocky Linux™ is a trademark of the Rocky Enterprise Software Foundation (RESF). This project is not officially affiliated with or endorsed by RESF. All trademarks are the property of their respective owners. This project complies with RESF's trademark usage guidelines.

## Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes with proper documentation
4. Test thoroughly
5. Commit with clear messages (`git commit -m 'feat: add amazing feature'`)
6. Push to your branch (`git push origin feature/amazing-feature`)
7. Open a Pull Request