Service Desk
Mapping out the future of billing units
Big changes are coming to CSC resource allocations in computing, data and cloud services. Last year, an internal development project to create and implement a new model for managing our resources was conducted. We are switching from an all-encompassing billing unit to more specific units for HPC, HPC GPU, cloud, and storage services.
Additionally, there will be changes to quota application processes, and the lifetimes of unused billing units and extra quotas will be limited. Instead of applying for one kind of billing units, researchers need to specify their resource needs for HPC, HPC GPU, cloud computing, and storage separately. This means more detailed resource planning and resource applications. Many users have a good idea of what resources and how much of them they need already. If the choice of resources and their quantities present problems, we are happy to help with choosing the right service and assessing the resource needs. In general, it’s good to start by testing with a small amount of units and apply for more once the research needs are clear.
In the future, capacity management for the different billing units could look like this:
New billing units will reflect the different kinds of resources and usage
CSC’s services for research need many different kinds of infrastructure. Different research projects have varying requirements: Some need a lot of CPUs or GPUs for a short while, some need more permanent resources, and some need vast amounts of data storage. These requirements can evolve independently over time.
The billing units that CSC has used since 1994 do not differentiate between these needs, however. Only ten years ago cloud computing was just a small part of our service offering, but its importance has grown quickly, and the need for storage capacity has grown even quicker. The rule-of-thumb for large resource applications has been to apply for resources to cover the needs for the next six months. Hence, some of the allocations have been large and their effect on services has been difficult to assess, leading to capacity shortages and long queues for some resources.
The growing user base creates another reason to streamline processes: Ten years ago CSC had about 6000 users and now there are more than 12 000. In the same time, the amount of academic projects has more than tripled from 1300 to 4500.
Another reason for the improved capacity management is the change of CSC’s funding: In the future the ministry funding for research infrastructure is continuous, hence the need for better usage data and usage control, which in turn enables better continuous ecosystem and capacity development.
Capacity management the key motivation for billing unit renewal
The main objectives for the change are to improve the capacity management and to unify resource management practices while keeping the model as simple and clear as possible for the end-users. We have examined how resources are managed at other HPC centers and what we can learn from them. As part of the project, quota applications will be centralized to MyCSC, aiming to develop it into a one-stop-shop for all computing project management needs.
In addition to simplified resource management, we strive to ensure that researchers have access to the resources they have been granted. Since the grants will be more specific to services, the granted, but unused, billing units deprecate if the project doesn’t use them as planned, and with more accurate usage prediction we can invest in the right kind of capacity to match future needs. We are also planning to implement user-friendly visualizations of the capacities in various CSC services.
The billing unit renewal will be implemented in the autumn of 2025 and will possibly be piloted in the spring. The change will affect the services for research: Roihu, Puhti, and Mahti supercomputers, Pouta and Rahti cloud services, SD Connect and SD Desktop services, as well as Allas, Shared fileservice, and Pukki data services – essentially all services that use billing units. For any questions, feel free to contact CSC Service Desk for more information.
Include publications for resource application directly from Research Hub
CSC has also made it possible for researchers to provide publication and research funding information from Research Hub while filling in the resource application in MyCSC customer portal. This kind of additional information gives the CSC resource allocation process more information about the qualifications and impact of the research, and hopefully raises the level of resource allocations. The results of using CSC’s services (publications, positive funding decisions) are an integral part of CSC’s impact assessments. We strive to allocate our resources according to the guidelines from the Ministry of Education and Culture as well as the EU, fitting together the needs of top research, the feedback from our users, and the usage and application data we collect.
Billing unit renewal supports the implementation of updated resource allocation policy
The billing unit renewal described in this blog is closely related to other improvements CSC has done recently. CSC resource allocation policy, a high-level policy to guide the scientific resource allocation, has been updated to meet the needs of a wider service catalog. Other policies and practices are being updated at the same time as well. Requiring a machine-actionable data management plan for large storage quota applications in the future is planned. This is to make more informed decisions based on the plans for the data during and after the research project.
We will communicate the changes and their schedule later. Summary of planned accounting models for services for research:
Authors: Joonas Nurmi, Jussi Tella, Atte Sillanpää
This blog was originally posted on CSC’s main website.