Performance Comparison of Classifiers: Frequentist and Bayesian
- Speaker: Dr Dell Zhang, Department of Computer Science and Information Systems, Birkbeck, University of London.
- Date: from 16:00 to 17:00
- Location: Room 151
Suppose that you have developed a new classification algorithm and it
achieves a higher performance score than the state of the art on a
test dataset. How can you know whether the performance improvement is
genuine but not because the new classifier happens to work better on
that particular test dataset by luck? The standard method to address
this problem is to carry out null hypothesis significance testing
(NHST), e.g., using the t-test. In this talk, I would like to discuss
the limitations of such a frequentist approach, and introduce our
recently proposed Bayesian models which can supersede NHST.