TECHNIQUES FOR AUTOMATICALLY INTERPRETING METRIC VALUES TO EVALUATE THE HEALTH OF A COMPUTER-BASED SERVICE

Number: 20190394283
Published: 2019-12-26
Filed: 2018-06-21
Assignee: DISNEY ENTERPRISES, INC.
Inventors: MORRISON; Vincent
CPC: G06F7/50; G06N20/00; G06N3/045; G06N3/0455; G06N3/08; G06N3/09; G06N5/01; H04L41/5009; H04L67/51
Verdict: Set aside server health metric evaluation, cloud plumbing
Source: Google Patents · FreePatentsOnline

Abstract

In various embodiments, a health evaluation application automatically monitors and evaluates the health of one or more computer-based services. The health evaluation application computes deviation values based on one or more machine-learned expected variations associated with multiple metrics. The metrics are associated with the computer-based service(s). The health evaluation application then performs classification operation(s) based on the deviation values and machine-learned classification criteria to compute anomaly indicators associated with a first service included in the one or more computer-based services. Subsequently, the health evaluation application computes a score that indicates the overall health of the first service based on the anomaly indicators. Advantageously, because the health evaluation application automatically computes the score, the time required to monitor and evaluate the health of the service is reduced compared to the time required to manually monitor and evaluate the health of the service.

Background

BACKGROUNDField of the Various Embodiments

Embodiments of the present invention relate generally to performance evaluation technology for monitoring and controlling the allocation of resources in a computer-based service.Description of the Related Art

Ensuring that a service is available and performing as intended or “healthy” is an important aspect of providing an effective service. As referred to herein, a “service” executes on one or more devices capable of executing instructions to perform a function for any number of users. A service is typically distributed via a network architecture. To evaluate the health of a service, oftentimes a service provider monitors a variety of metrics associated with the service and, based on those metrics, attempts to determine whether the service is healthy. If, at any point in time, the service provider determines that the performance of the service is subpar, then the service provider can take measures to restore the health of the service. For example, a web service provider could monitor requests per second, memory utilization, disk space, etc. to evaluate the performance of the web service. If the web service provider were to determine that the performance of the web service was subpar based on the metrics, then the web service provider could adjust the amount of resources (e.g., increase the memory) allocated to the web service in an attempt to resolve the performance issues and restore the “health” of the service.

Claims

1. A computer-implemented method, comprising: computing a first plurality of deviation values based on one or more machine-learned expected variations associated with a plurality of metrics, wherein the plurality of metrics is associated with one or more computer-based services; performing one or more classification operations based on the first plurality of deviation values and a first set of machine-learned classification criteria to compute a first plurality of anomaly indicators associated with a first service included in the one or more computer-based services; and computing a first score that indicates the overall health of the first service based on the first plurality of anomaly indicators. 2. The computer-implemented method of claim 1, wherein computing the first plurality of deviation values comprises: generating a first identification feature matrix based a plurality of metric values stored in a metric value database; generating a smoothed matrix based on the first identification feature matrix and an anomaly identification model that includes a mapping from the first identification feature matrix to the smoothed matrix based on the one or more machine-learned expected variations; and performing one or more subtraction operations between the first identification feature matrix and the smoothed matrix to generate the first plurality of deviation values. 3. The computer-implemented method of claim 2, wherein the anomaly identification model comprises a trained noise reducing autoencoder. 4. The computer-implemented method of claim 1, wherein performing the one or more classification operations comprises: generating a first plurality of interdependent inputs based on the first plurality of deviation values, wherein each interdependent input is associated with the first service and each metric included in the plurality of metrics; and for each interdependent input included in the first plurality of interdependent inputs, generating a different anomaly indicator included in the first plurality of anomaly indicators based on an anomaly model, wherein the anomaly model includes a mapping of the interdependent input to the different anomaly indicator that is based on the first set of machine-learned classification criteria. 5. The computer-implemented method of claim 4, wherein the anomaly model comprises a trained isolation forest. 6. The computer-implemented method of claim 1, wherein generating the first score comprises performing one or more weighted averaging operations on the first plurality of anomaly indicators. 7. The computer-implemented method of claim 1, further comprising: performing one or more classification operations based on the first plurality of deviation values, a first metric included in the plurality of metrics, and a second set of machine-learned classification criteria to compute a second plurality of anomaly indicators associated with the first service; and computing a second score that indicates a health of the first service with respect to the first metric based on the second plurality of anomaly indicators. 8. The computer-implemented method of claim 1, further comprising: generating a first plurality of independent inputs based on the plurality of deviation values, wherein each independent input is associated with the first service and a first metric included in the plurality of metrics; for each interdependent input included in the first plurality of interdependent inputs, generating a different anomaly indicator included in a second plurality of anomaly indicators based on an anomaly model, wherein the anomaly model includes a mapping of the independent input to the different anomaly indicator that is based on a second set of machine-learned classification criteria; and computing a second score that indicates a health of the first service with respect to the first metric based on the second plurality of anomaly indicators. 9. The computer-implemented method of claim 8, wherein the anomaly model comprises a trained isolation forest. 10. The computer-implemented method of claim 1, further comprising: performing one or more classification operations based on the first plurality of deviation values and the first set of machine-learned classification criteria to compute a second plurality of anomaly indicators associated with a second service included in the one or more computer-based services; and computing a second score that indicates the overall health of the second service based on the second plurality of anomaly indicators. 11. The computer-implemented method of claim 1, further comprising automating a mitigating action based on the first score. 12. The computer implemented method of claim 11, wherein the mitigating action is an allocation of computing resources to the one or more computer-based services. 13. The computer implemented method of claim 11, wherein the mitigating action is a computing operation associated with a second score. 14. A computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform the steps of: computing a first plurality of deviation values based on one or more machine-learned expected variations associated with a plurality of metrics, wherein the plurality of metrics is associated with one or more computer-based services; performing one or more classification operations based on the first plurality of deviation values and a first set of machine-learned classification criteria to compute a first plurality of anomaly indicators associated with a first service included in the one or more computer-based services; and computing a first score that indicates the overall health of the first service based on the first plurality of anomaly indicators. 15. The computer-readable storage medium of claim 14, wherein computing the first plurality of deviation values comprises: generating a first identification feature matrix based a plurality of metric values stored in a metric value database; generating a smoothed matrix based on the first identification feature matrix and an anomaly identification model that includes a mapping from the first identification feature matrix to the smoothed matrix based on the one or more machine-learned expected variations; and performing one or more subtraction operations between the first identification feature matrix and the smoothed matrix to generate the first plurality of deviation values. 16. The computer-readable storage medium of claim 15, further comprising training a noise reducing autoencoder using a second identification feature matrix associated with the one or more computer-based services to generate the anomaly identification model. 17. The computer-readable storage medium of claim 14, wherein performing the one or more classification operations comprises: generating a first plurality of interdependent inputs based on the first plurality of deviation values, wherein each interdependent input is associated with the first service and each metric included in the plurality of metrics; and for each interdependent input included in the first plurality of interdependent inputs, generating a different anomaly indicator included in the first plurality of anomaly indicators based on an anomaly model, wherein the anomaly model includes a mapping of the interdependent input to the different anomaly indicator that is based on the first set of machine-learned classification criteria. 18. The computer-readable storage medium of claim 17, further comprising training an isolation forest using at least a second plurality of interdependent inputs and at least one expected percentage of anomalies to generate the anomaly model. 19. The computer-readable storage medium of claim 14, wherein generating the first score comprises performing one or more weighted averaging operations on the first plurality of anomaly indicators. 20. The computer-readable storage medium of claim 14, further comprising: generating a first plurality of independent inputs based on the plurality of deviation values, wherein each independent input is associated with the first service and a first metric included in the plurality of metrics; for each interdependent input included in the first plurality of interdependent inputs, generating a different anomaly indicator included in a second plurality of anomaly indicators based on an anomaly model, wherein the anomaly model includes a mapping of the independent input to the different anomaly indicator that is based on a second set of machine-learned classification criteria; and computing a second score that indicates a health of the first service with respect to the first metric based on the second plurality of anomaly indicators. 21. The computer-readable storage medium of claim 20, further comprising training an isolation forest using at least a second plurality of independent inputs associated with the first metric and a third plurality of independent inputs associated with a second metric included in the plurality of metrics to generate the anomaly model. 22. The computer-readable storage medium of claim 14, further comprising generating a graphical user interface that depicts the first score and a plurality of metric health scores associated with the plurality of metrics. 23. A system, comprising: a memory storing instructions; and a processor that is coupled to the memory and, when executing the instructions, is configured to: compute a first plurality of deviation values based on one or more machine-learned expected variations associated with a plurality of metrics, wherein the plurality of metrics is associated with one or more computer-based services; perform one or more classification operations based on the first plurality of deviation values and a first set of machine-learned classification criteria to compute a first plurality of anomaly indicators associated with a first service included in the one or more computer-based services; and compute a first score that indicates the overall health of the first service based on the first plurality of anomaly indicators.