Interface for visualizing and improving model performance
A method includes monitoring performance of a generated model while the generated model is being used for classification on live data, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time; rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric; and rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value. Related apparatus, systems, techniques and articles are also described.
1. A method comprising:
monitoring performance of a generated model while the generated model is being used for classification on live data, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time;
rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric; and
rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value.
2. The method of claim 1 , further comprising:
rendering a first line indicative of random model performance, a second line indicative of constant accuracy, and/or a third line indicative of constant cost.
3. The method of claim 1 , wherein the first performance metric includes rate of false positive, count of false positive, cost of false positive, benefit missed by false positive, true positive, benefit of true positive, benefit of minimizing false positive, benefit of maximizing true positive, or a combination thereof;
wherein the second performance metric includes rate of false negative, count of false negative, cost of false negative, benefit missed by false negative, true negative, benefit of true negative, benefit of minimizing false negative, benefit of maximizing true negative, or a combination thereof.
4. The method of claim 1 , further comprising:
rendering a line between the first graphical object and the second graphical object.
5. The method of claim 1 , wherein the monitoring includes determining a third performance value of the generated model at a third point in time; the method further comprising:
rendering, within the graphical user interface and the plot, a third graphical object at a third location characterizing the third performance value.
6. The method of claim 1 , wherein the first graphical object includes a shape and/or color indicative of a characteristic of the generated model, the characteristic including a complexity metric.
7. The method of claim 1 , further comprising:
monitoring performance of a second generated model while the second generated model is being used for classification on live data, the monitoring including determining a third performance value of the second generated model at the first point in time and determining a fourth performance value of the second generated model at the second point in time; and
rendering, within the graphical user interface and the plot, a third graphical object at a third location characterizing the third performance value and a fourth graphical object at a fourth location characterizing the fourth performance value.
8. A system comprising:
at least one processor;
memory storing instructions which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:
monitoring performance of a generated model while the generated model is being used for classification on live data, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time;
rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric; and
rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value.
9. The system of claim 8 , the operations further comprising:
rendering a first line indicative of random model performance, a second line indicative of constant accuracy, and/or a third line indicative of constant cost.
10. The system of claim 8 , wherein the first performance metric includes rate of false positive, count of false positive, cost of false positive, benefit missed by false positive, true positive, benefit of true positive, benefit of minimizing false positive, benefit of maximizing true positive, or a combination thereof;
wherein the second performance metric includes rate of false negative, count of false negative, cost of false negative, benefit missed by false negative, true negative, benefit of true negative, benefit of minimizing false negative, benefit of maximizing true negative, or a combination thereof.
11. The system of claim 8 , the operations further comprising:
rendering a line between the first graphical object and the second graphical object.
12. The system of claim 8 , wherein the monitoring includes determining a third performance value of the generated model at a third point in time; the method further comprising:
rendering, within the graphical user interface and the plot, a third graphical object at a third location characterizing the third performance value.
13. The system of claim 8 , wherein the first graphical object includes a shape and/or color indicative of a characteristic of the generated model, the characteristic including a complexity metric.
14. The system of claim 8 , the operations further comprising:
monitoring performance of a second generated model while the second generated model is being used for classification on live data, the monitoring including determining a third performance value of the second generated model at the first point in time and determining a fourth performance value of the second generated model at the second point in time; and
rendering, within the graphical user interface and the plot, a third graphical object at a third location characterizing the third performance value and a fourth graphical object at a fourth location characterizing the fourth performance value.
15. A non-transitory computer program product storing instructions, which when executed by at least one processor of at least one computing system, cause the at least one processor to perform operations comprising:
monitoring performance of a generated model while the generated model is being used for classification on live data, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time;
rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric; and
rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value.