Batch scheduler management of virtual machines

Number: 9946563
Published: 2018-04-17
Filed: 2012-02-21
Assignee: Disney Enterprises, Inc.
Inventors: Geibel; Jonathan Eric et al.
CPC: G06F9/45558; G06F9/5088; G06F9/5077
Verdict: Set aside virtual-machine batch scheduling, cloud infra
Source: Google Patents · FreePatentsOnline

Abstract

A batch scheduler receives a task. Further, a processor, at the batch scheduler, generates a virtual machine corresponding to the task. In addition, the virtual machine is deployed from the batch scheduler to a compute node. The task is also dispatched from the batch scheduler to the virtual machine at the compute node.

Background

BACKGROUND(1) 1. Field(2) This disclosure generally relates to the field of computer systems. More particularly, the disclosure relates to batch scheduling.(3) 2. General Background(4) Many current computer systems utilize batch scheduling to execute a series of programs without manual intervention. Batch scheduling allows large sets of data to be processed in batches. Current batch scheduling systems typically submit tasks directly to the base operating system of a compute node that is to run the task or to a virtual machine (“VM”) that is not directly managed by the batch scheduling system itself. The compute node may be a computing device, a program executed on a computing device, an operating system, or the like. Further, a computing device, e.g., a server, may have one or more compute nodes. When a compute node is resource constrained, e.g., by limitations regarding a central processing unit (“CPU”), memory, or the like, tasks may be discarded to free up resources and rerun on another compute node at a later point. Most batch scheduling systems in high performance computing are utilized for processing batches of large amounts of data, which does not adequately scale the batch processing to optimize utilization of resources. For example, the current batch processing systems often overutilize certain resources, but underutilize other resources. Many servers perform intensive processing tasks while other processors perform very little processing.SUMMARY(5) In one aspect of

Claims

1. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: receive a task at a batch scheduler; generate, at the batch scheduler, a virtual machine corresponding to the task; deploy the virtual machine from the batch scheduler to a compute node; dispatch the task from the batch scheduler to the virtual machine at the compute node so that the virtual machine performs the task; receive, at the batch scheduler, global system information of the compute node from a hypervisor monitoring module that monitors a hypervisor running at the compute node; monitor, with the batch scheduler, a probability of the task at the compute node exceeding a resource reservation for the virtual machine and an additional probability of one or more additional tasks at one or more additional compute nodes exceeding an additional resource reservation for one or more additional virtual machines deployed to the one or more additional compute nodes; and dynamically rebalance, with the batch scheduler, a total load of the virtual machine and the one or more additional virtual machines, based on the global system information of the compute node, the probability of the task at the compute node exceeding the resource reservation, and the probability of the one or more additional compute nodes exceeding the additional resource reservation, by reallocating the task from the virtual machine to one of the one or more additional virtual machines to optimize resource allocation during active processing of at least a portion of the total load. 9. A method comprising: receiving a task at a batch scheduler; generating with a processor, at the batch scheduler, a virtual machine corresponding to the task; deploying the virtual machine from the batch scheduler to a compute node; dispatching the task from the batch scheduler to the virtual machine at the compute node so that the virtual machine performs the task; receiving, at the batch scheduler, global system information of the compute node from a hypervisor monitoring module that monitors a hypervisor running at the compute node; monitoring, with the batch scheduler, a probability of the task at the compute node exceeding a resource reservation for the virtual machine and an additional probability of one or more additional tasks at one or more additional compute nodes exceeding an additional resource reservation for one or more additional virtual machines deployed to the one or more additional compute nodes; and dynamically rebalancing, with the batch scheduler, a total load of the virtual machine and the one or more additional virtual machines, based on the global system information of the compute node, the probability of the task at the compute node exceeding the resource reservation, and the probability of the one or more additional compute nodes exceeding the additional resource reservation, by reallocating the task from the virtual machine to one of the one or more additional virtual machines to optimize resource allocation during active processing of at least a portion of the total load. 17. A system comprising: a processor having a batch scheduler that receives a task, generates a virtual machine corresponding to the task, deploys the virtual machine to a compute node, dispatches the task to the virtual machine at the compute node, receives global system information of the compute node from a hypervisor monitoring module that monitors a hypervisor running at the compute node, monitors a probability of the task at the compute node exceeding a resource reservation for the virtual machine, and an additional probability of one or more additional tasks at one or more additional compute nodes exceeding an additional resource reservation for one or more additional virtual machines deployed to the one or more additional compute nodes, and dynamically rebalances a total load of the virtual machine and the one or more additional virtual machines by reallocating the task from the virtual machine to one of the one or more additional virtual machines, based on the global system information of the compute node, the probability of the task at the compute node exceeding the resource reservation, and the probability of the one or more additional compute nodes exceeding the additional resource reservation, to optimize resource allocation during active processing of at least a portion of the total load, the compute node being a computing device at which the virtual machine performs the task. 18. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: receive, at a compute node, a virtual machine from a batch scheduler; receive, at the compute node, a task from the batch scheduler, the task corresponding to the virtual machine; provide, at the compute node, resources to the virtual machine to process the task; receive global system information of the compute node from a hypervisor monitoring module that monitors a hypervisor running at the compute node; and reallocate the task from the virtual machine to one of one or more additional virtual machines based on a dynamic rebalancing determined by the batch scheduler during active processing of the task based on the global system information of the compute node, wherein the batch scheduler monitors a probability of the task at the compute node exceeding a resource reservation for the virtual machine and an additional probability of one or more additional tasks at one or more additional compute nodes exceeding an additional resource reservation for one or more additional virtual machines deployed to the one or more additional compute nodes, wherein the dynamic rebalancing is further based on the probability of the task at the compute node exceeding the resource reservation and the probability of the one or more additional compute nodes exceeding the additional resource reservation. 21. A method comprising: receiving, at a compute node, a virtual machine from a batch scheduler; receiving, at the compute node, a task from the batch scheduler, the task corresponding to the virtual machine; providing, at the compute node, resources to the virtual machine to process the task; receiving global system information of the compute node from a hypervisor monitoring module that monitors a hypervisor running at the compute node; and reallocating the task from the virtual machine to one of one or more additional virtual machines based on a dynamic rebalancing determined by the batch scheduler during active processing of the task based on the global system information of the compute node, wherein the batch scheduler monitors a probability of the task at the compute node exceeding a resource reservation for the virtual machine and an additional probability of one or more additional tasks at one or more additional compute nodes exceeding an additional resource reservation for one or more additional virtual machines deployed to the one or more additional compute nodes, wherein the dynamic rebalancing is further based on the probability of the task at the compute node exceeding the resource reservation and the probability of the one or more additional compute nodes exceeding the additional resource reservation.