Firenze: Model Evaluation Using Weak Signals. (arXiv:2207.00827v1 [cs.CR])

Data labels in the security field are frequently noisy, limited, or biased
towards a subset of the population. As a result, commonplace evaluation methods
such as accuracy, precision and recall metrics, or analysis of performance
curves computed from labeled datasets do not provide sufficient confidence in
the real-world performance of a machine learning (ML) model. This has slowed
the adoption of machine learning in the field. In the industry today, we rely
on domain expertise and lengthy manual evaluation to …