Bayesian model-based clustering procedures

John Lau, P.J. Green

    Research output: Contribution to journalArticle

    101 Citations (Scopus)

    Abstract

    This article establishes a general formulation for Bayesian model-based clustering, in which subset labels are exchangeable, and items are also exchangeable, possibly up to covariate effects. The notational framework is rich enough to encompass a variety of existing procedures, including some recently discussed methods involving stochastic search or hierarchical clustering, but more importantly allows the formulation of clustering procedures that are optimal with respect to a specified loss function. Our focus is on loss functions based on pairwise coincidences, that is, whether pairs of items are clustered into the same subset or not.Optimization of the posterior expected loss function can be formulated as a binary integer programming problem, which can be readily solved by standard software when clustering a modest number of items, but quickly becomes impractical as problem scale increases. To combat this, a new heuristic item-swapping algorithm is introduced. This performs well in our numerical experiments, on both simulated and real data examples. The article includes a comparison of the statistical performance of the (approximate) optimal clustering with earlier methods that are rnodel-based but ad hoc in their detailed definition.
    Original languageEnglish
    Pages (from-to)526-558
    JournalJournal of Computational and Graphical Statistics
    Volume16
    Issue number3
    DOIs
    Publication statusPublished - 2007

    Fingerprint Dive into the research topics of 'Bayesian model-based clustering procedures'. Together they form a unique fingerprint.

    Cite this