CLI Ergonomics
The Goldentooth CLI underwent a fundamental transformation, evolving from a verbose, Ansible-heavy interface into a sleek, ergonomic command suite optimized for both human operators and programmatic consumption. This architectural revolution introduced direct SSH operations, intelligent MOTD systems, distributed computing integration, and performance improvements that deliver 3x faster execution times.
The Transformation
From Ansible-Heavy to SSH-Native Operations
The original CLI relied exclusively on Ansible playbooks for every operation, creating unnecessary overhead for simple tasks. The new architecture introduces direct SSH operations that bypass Ansible entirely for appropriate use cases:
Before: Every command required Ansible overhead
# Old approach - always through Ansible
goldentooth command all "systemctl status consul" # ~10-15 seconds
After: Direct SSH with intelligent routing
# New approach - direct SSH operations
goldentooth shell bettley # Instant interactive session
goldentooth command all "systemctl status consul" # ~3-5 seconds with parallel
Revolutionary SSH-Based Command Suite
Interactive Shell Sessions
The shell
command provides seamless access to cluster nodes with intelligent behavior:
# Single node - direct SSH session with beautiful MOTD
goldentooth shell bettley
# Multiple nodes - broadcast mode with synchronized output
goldentooth shell all
Smart Behavior:
- Single node: Interactive SSH session with full MOTD display
- Multiple nodes: Broadcast mode with synchronized command execution
- Automatic host resolution from Ansible inventory groups
Stream Processing with Pipe
The pipe
command transforms stdin into distributed execution:
# Stream commands to multiple nodes
echo "df -h" | goldentooth pipe storage_nodes
echo "systemctl status consul" | goldentooth pipe consul_server
Advanced Features:
- Comment filtering (lines starting with
#
are ignored) - Empty line skipping for clean script processing
- Parallel execution across multiple hosts
- Clean error handling and output formatting
File Transfer with CP
Node-aware file transfer using intuitive syntax:
# Copy from cluster to local
goldentooth cp bettley:/var/log/consul.log ./logs/
# Copy from local to cluster
goldentooth cp ./config.yaml allyrion:/etc/myapp/
# Inter-node transfers
goldentooth cp allyrion:/tmp/data.json bettley:/opt/processing/
Batch Script Execution
Execute shell scripts across the cluster:
# Run maintenance script on storage nodes
goldentooth batch maintenance.sh storage_nodes
# Execute deployment script on all nodes
goldentooth batch deploy.sh all
Multi-line Command Execution
The heredoc
command enables complex multi-line operations:
goldentooth heredoc consul_server <<'EOF'
consul kv put config/database/host "db.goldentooth.net"
consul kv put config/database/port "5432"
systemctl reload myapp
EOF
Performance Architecture
GNU Parallel Integration
The CLI intelligently detects and leverages GNU parallel
for concurrent operations:
Automatic Parallelization:
- Single host: Direct SSH connection
- Multiple hosts: GNU parallel with job control (
-j0
for optimal concurrency) - Fallback: Sequential execution if parallel unavailable
Performance Improvements:
- 3x faster execution for multi-host operations
- Optimal resource utilization across cluster nodes
- Tagged output for clear host identification
Intelligent SSH Configuration
Optimized SSH behavior for different use cases:
Clean Command Output:
ssh_opts="-T -o StrictHostKeyChecking=no -o LogLevel=ERROR -q"
Features:
-T
flag disables pseudo-terminal allocation (suppresses MOTD for commands)- Error suppression for clean programmatic consumption
- Connection optimization for repeated operations
MOTD System Overhaul
Visual Node Identification
Each cluster node features unique ASCII art MOTD for instant visual recognition:
Implementation:
- Node-specific colorized ASCII artwork stored in
/etc/motd
- Beautiful visual identification during interactive SSH sessions
- SSH
PrintMotd yes
configuration for proper display
Examples:
bettley
: Distinctive golden-colored ASCII art designallyrion
: Unique visual signature for immediate recognition- Each node: Custom artwork matching cluster theme and node personality
Smart MOTD Behavior
The system provides context-appropriate MOTD display:
Interactive Sessions: Full MOTD display with ASCII art Command Execution: Suppressed MOTD for clean output Programmatic Access: No visual interference with data processing
Technical Implementation:
- Removed complex PAM-based conditional MOTD system
- Leveraged SSH's built-in
PrintMotd
behavior - Clean separation between interactive and programmatic access
Inventory Integration System
Ansible Group Compatibility
The CLI seamlessly integrates Ansible inventory definitions with SSH operations:
Inventory Parsing:
# parse-inventory.py converts YAML inventory to bash functions
def generate_bash_variables(groups):
# Creates goldentooth:resolve_hosts() function
# Generates case statements for each group
# Maintains compatibility with existing Ansible workflows
Generated Functions:
function goldentooth:resolve_hosts() {
case "$expression" in
"consul_server")
echo "allyrion bettley cargyll"
;;
"storage_nodes")
echo "jast karstark lipps"
;;
# ... all inventory groups
esac
}
Installation Integration:
- Inventory parsing during CLI installation (
make install
) - Automatic generation of
/usr/local/bin/goldentooth-inventory.sh
- Dynamic loading of inventory groups into CLI
Distributed LLaMA Integration
Cross-Platform Compilation
Advanced cross-compilation support for ARM64 distributed computing:
Architecture:
- x86_64 Velaryon node: Cross-compilation host
- ARM64 Pi nodes: Deployment targets
- Automated binary distribution and service management
Commands:
# Model management
goldentooth dllama_download_model meta-llama/Llama-3.2-1B
# Service lifecycle
goldentooth dllama_start_workers
goldentooth dllama_stop
# Cluster status
goldentooth dllama_status
# Distributed inference
goldentooth dllama_inference "Explain quantum computing"
Technical Features:
- Automatic model download and conversion
- Distributed worker node management
- Cross-architecture binary deployment
- Performance monitoring and status reporting
Command Line Interface Enhancements
Bash Completion System
Comprehensive tab completion for all operations:
Features:
- Command completion for all CLI functions
- Host and group name completion
- Context-aware parameter suggestions
- Integration with existing shell environments
Error Handling and Output Management
Professional error management with proper stream handling:
Implementation:
- Error messages directed to stderr
- Clean stdout for programmatic consumption
- Consistent exit codes for automation integration
- Detailed error reporting with actionable suggestions
Help and Documentation
Built-in documentation system:
# List available commands
goldentooth help
# Command-specific help
goldentooth help shell
goldentooth help dllama_inference
# Show available inventory groups
goldentooth list_groups
Integration with Existing Infrastructure
Ansible Compatibility
The new CLI maintains full compatibility with existing Ansible workflows:
Hybrid Approach:
- SSH operations for simple, fast tasks
- Ansible playbooks for complex configuration management
- Seamless switching between approaches based on task requirements
Examples:
# Quick status check - SSH
goldentooth command all "uptime"
# Complex configuration - Ansible
goldentooth setup_consul
Monitoring and Observability
CLI operations integrate with existing monitoring systems:
Features:
- Command execution logging
- Performance metrics collection
- Integration with Prometheus/Grafana monitoring
- Audit trail for security compliance
User Experience Improvements
Intuitive Command Syntax
Natural, memorable command patterns:
# Intuitive file operations
goldentooth cp source destination
# Clear service management
goldentooth dllama_start_workers
# Obvious interactive access
goldentooth shell hostname