{"id":15081,"date":"2023-08-08T16:05:44","date_gmt":"2023-08-08T14:05:44","guid":{"rendered":"https:\/\/ucit.fr\/?p=15041"},"modified":"2023-09-07T16:12:42","modified_gmt":"2023-09-07T14:12:42","slug":"integrating-the-oka-suite-with-your-aws-cluster","status":"publish","type":"post","link":"https:\/\/ucit.fr\/index.php\/2023\/08\/08\/integrating-the-oka-suite-with-your-aws-cluster\/","title":{"rendered":"Integrating the OKA Suite with your AWS cluster"},"content":{"rendered":"

<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Integrating the OKA Suite with your AWS cluster<\/h1><\/div><\/div><\/div><\/div><\/div>

Efficiently running an HPC infrastructure is complex, and often lacks the proper tools to track down and get insights on how the users are behaving and how the cluster is responding to the demand. This is even more complicated when using Cloud clusters. Due to their transient and dynamic nature, information about instances types, location and costs are important assets to monitor, especially when you pay for what you use. The task of managing and presenting these metrics becomes increasingly difficult when the scale of the infrastructure grows or undergoes changes over time, which is a common situation in Cloud environments.<\/p>\n

“Standard” metrics and information provided by the job scheduler might not be sufficient to efficiently manage the Cloud clusters. For example, tracking the cost of running jobs becomes even more important in order to monitor and manage your budget, and redistribute the costs to your users\/departments. Due to the wide variety of compute instance types in the cloud, it can also be interesting to track on which instance types the jobs have run, in order to check their performances and associated costs, and further improve the placement and instance selection of the jobs.<\/p>\n

OKA offers many benefits to monitor your jobs, and deep-dive into how your clusters behave and how they are being used by your end-users. Accessing in OKA information about the cloud is straightforward, if you have configured your environment properly. In this article, we present a simple integration that can be made in a Slurm cluster in AWS to retrieve the type of instances jobs runs on, their pricing information (on-demand\/spot, per hour price…), the AWS region, and virtually any information about the AWS environment you are using.
\nThe scripts provided below are given as examples, and can easily be adapted to retrieve more detailed information, or be adapted to work with other job schedulers (e.g., LSF, PBS…).<\/p>\n<\/div>

There are many ways to create a cluster in AWS, the details are out of the scope of this article, but you can use for example AWS ParallelCluster<\/a><\/span> or CCME<\/a><\/span>.<\/p>\n<\/div>

\n

Note: the solution presented here is extracted from CCME where it is available out of the box.<\/p>\n<\/blockquote>\n<\/div>

<\/div>

The principle depicted here is very simple, and relies on two components:<\/p>\n

\n
    \n
  1. A Slurm epilog script<\/a><\/span> that will gather information about the AWS environment on which the job runs, and store this information as comma separated values (CSV) in the Comment<\/a><\/span> field of the job. The gathered information are:\n
      \n
    • instance type<\/li>\n
    • instance id of the “main” job node<\/li>\n
    • availability zone<\/li>\n
    • region<\/li>\n
    • instance price<\/li>\n
    • cost type: ondemand or spot<\/li>\n
    • tenancy: shared, reserved…<\/li>\n<\/ul>\n<\/li>\n
    • An OKA Data Enhancer that will parse the values of the Comment<\/code> field, and store them as additional information with each job.<\/li>\n<\/ol>\n<\/div>\n<\/div>
      <\/div>
      \"\"<\/div>
      \"\"<\/div>