Advanced AI-Powered Optimization

Explore how LLM embeddings revolutionize per-job power prediction, achieving higher accuracy than state-of-the-art methods.

The study's novel semantic retrieval model (SR) demonstrates a 15% reduction in Mean Absolute Error (MAE) for per-job power prediction compared to the current state-of-the-art baseline (UoPC). This significant improvement is primarily driven by gains in the high-volume, low-to-mid power consumption regions (250-500 W), where 40% of all jobs reside. The semantic approach, leveraging Large Language Model (LLM) embeddings of enriched job scripts, captures nuanced domain- and workflow-specific cues, leading to more accurate predictions without manual feature engineering. A similar trend is observed for runtime prediction, with a 12% reduction in mean runtime error.

Discover the lightweight, energy-aware scheduling algorithm and the high-fidelity FastSim simulator that enables its optimization.

The energy-aware scheduling algorithm leverages predicted power usage and real-time renewable energy availability to dynamically adjust job priorities within Slurm's multifactor priority framework. This lightweight heuristic approach avoids core scheduler modifications, making it practical for production deployment. The rigorous validation of the FastSim simulator against historical job traces ensures that simulation results accurately reflect real-world performance, providing a reliable framework for evaluating and optimizing new scheduling policies. The optimization process uses Optuna to maximize renewable power utilization, achieving a deliberate balance between minimizing job wait times and maximizing clean energy use.

Learn about the practical, Slurm-native integration pathway that allows for incremental rollout without core scheduler changes.

This blueprint outlines a Slurm-native deployment strategy utilizing existing plugin interfaces (Job-Submit and SiteFactor). The approach ensures that core scheduler modifications are avoided, mitigating risks and simplifying adoption. Predictions are generated by a lightweight inference service, which allows for fast, non-blocking operations. The system is designed for incremental rollout, enabling administrators to gradually increase the influence of energy-aware scheduling and periodically tune parameters based on real-world performance, ensuring stability and optimal results.