Cluster Admin Guide ($49) · Snippets · GitLab

This GitLab is scheduled to be shut down at the end of this year. Please create new projects/repositories on the kiz-GitLab (https://gitlab.uni-ulm.de/). Existing projects on this GitLab can be imported into the kiz-GitLab over "New project" > "Import project" > "Repository by URL" > https://gitlab-mi.informatik.uni-ulm.de/[path_to_project].git (clone project with https)

leonsick @txf95 · Sep 23, 2024
Addition to Troubleshooting

Problem: A worker has status "draining" and jobs won't start anymore

Restart the worker (either reboot or physical restart)

Execute: sudo scontrol update nodename=<worker_id> state=idle

Edited Sep 23, 2024 by leonsick

**Addition to Troubleshooting** **Problem:** A worker has status "draining" and jobs won't start anymore 1. Restart the worker (either reboot or physical restart) 2. Execute: sudo scontrol update nodename=<worker_id> state=idle
leonsick @txf95 · Oct 07, 2024
Problem: A worker has status "draining" and Reason SlurmSpoolDir is full.

ssh onto the worker

Execute docker system prune -af

Check what is in /var/spool/slurmd and clean up what can be removed

Set worker to status idle

Edited Oct 07, 2024 by leonsick

**Problem:** A worker has status "draining" and Reason SlurmSpoolDir is full. 1. ssh onto the worker 2. Execute docker system prune -af 3. Check what is in /var/spool/slurmd and clean up what can be removed 3. Set worker to status idle

Please register or to comment