We strive to perform all planned and unplanned maintenance with minimal impact to the end user. However on occasion we need to conduct planned or unplanned maintenance on our systems which requires us to take the system completely down or significantly degrade performance.
Currently the supercomputer Sisu needs 3-4 regular software updates per year for which a planned break needs to be scheduled.
Planned breaks are announced at least several weeks in advance of the break allowing sufficient time for all running jobs to complete safely.
We always try to schedule our maintenance breaks in the mainenance window on the first Monday of every month. Sometimes we require several days to perform the maintenance. We will always announce the planned duration in advance and send updates before and during the break if there are changes to the duration.
The system is drained from jobs typically using an advanced reservation. This means that any job that can safely finish before the break is eligible to run. Thus very short jobs can still run even moments before the break.
On occasion we may need to perform maintenance urgently due to technical or security reasons. During these cases we will announce the break and set the advanced reservation (see above) as soon as possible.