Ebook: Optimal Bayesian classification

"The most basic problem of engineering is the design of optimal operators. Design takes different forms depending on the random process constituting the scientific model and the operator class of interest. This book treats classification, where the underlying random process is a feature-label distribution, and an optimal operator is a Bayes classifier, which is a classifier minimizing the classification error. With sufficient knowledge we can construct the feature-label distribution and thereby find a Bayes classifier. Rarely, do we possess such knowledge. On the other hand, if we had unlimited data, we could accurately estimate the feature-label distribution and obtain a Bayes classifier. Rarely do we possess sufficient data. The aim of this book is to best use whatever knowledge and data are available to design a classifier. The book takes a Bayesian approach to modeling the feature-label distribution and designs an optimal classifier relative to a posterior distribution governing an uncertainty class of feature-label distributions. In this way it takes full advantage of knowledge regarding the underlying system and the available data. Its origins lie in the need to estimate classifier error when there is insufficient data to hold out test data, in which case an optimal error estimate can be obtained relative to the uncertainty class. A natural next step is to forgo classical ad hoc classifier design and simply find an optimal classifier relative to the posterior distribution over the uncertainty class-this being an optimal Bayesian classifier"