### Abstract

Friedman proposed a regularization technique (RDA) of discriminant analysis in the Gaussian framework. RDA uses two regularization parameters to design an intermediate classifier between the linear, the quadratic, and the nearest-means classifiers. In this article we propose an alternative approach, called EDDA, that is based on the reparameterization of the covariance matrix [Σ_{k}] of a group G_{k} in terms of its eigenvalue decomposition Σ_{k} = λ_{k}D_{k}A_{k}D′_{k}, where λ_{k} specifies the volume of density contours of G_{k}, the diagonal matrix of eigenvalues specifies its shape, and the eigenvectors specify its orientation. Variations on constraints concerning volumes, shapes, and orientations λ_{k}, A_{k}, and D_{k} lead to 14 discrimination models of interest. For each model, we derived the normal theory maximum likelihood parameter estimates. Our approach consists of selecting a model by minimizing the sample-based estimate of future misclassification risk by cross-validation. Numerical experiments on simulated and real data show favorable behavior of this approach compared to RDA.

Original language | English |
---|---|

Pages (from-to) | 1743-1748 |

Number of pages | 6 |

Journal | Journal of the American Statistical Association |

Volume | 91 |

Issue number | 436 |

Publication status | Published - 1 Dec 1996 |

Externally published | Yes |

### Fingerprint

### Keywords

- Covariance matrix
- Maximum likelihood
- Normal-based classification
- Spectral decomposition

### ASJC Scopus subject areas

- Mathematics(all)
- Statistics and Probability

### Cite this

*Journal of the American Statistical Association*,

*91*(436), 1743-1748.

**Regularized Gaussian discriminant analysis through eigenvalue decomposition.** / Bensmail, Halima; Celeux, Gilles.

Research output: Contribution to journal › Article

*Journal of the American Statistical Association*, vol. 91, no. 436, pp. 1743-1748.

}

TY - JOUR

T1 - Regularized Gaussian discriminant analysis through eigenvalue decomposition

AU - Bensmail, Halima

AU - Celeux, Gilles

PY - 1996/12/1

Y1 - 1996/12/1

N2 - Friedman proposed a regularization technique (RDA) of discriminant analysis in the Gaussian framework. RDA uses two regularization parameters to design an intermediate classifier between the linear, the quadratic, and the nearest-means classifiers. In this article we propose an alternative approach, called EDDA, that is based on the reparameterization of the covariance matrix [Σk] of a group Gk in terms of its eigenvalue decomposition Σk = λkDkAkD′k, where λk specifies the volume of density contours of Gk, the diagonal matrix of eigenvalues specifies its shape, and the eigenvectors specify its orientation. Variations on constraints concerning volumes, shapes, and orientations λk, Ak, and Dk lead to 14 discrimination models of interest. For each model, we derived the normal theory maximum likelihood parameter estimates. Our approach consists of selecting a model by minimizing the sample-based estimate of future misclassification risk by cross-validation. Numerical experiments on simulated and real data show favorable behavior of this approach compared to RDA.

AB - Friedman proposed a regularization technique (RDA) of discriminant analysis in the Gaussian framework. RDA uses two regularization parameters to design an intermediate classifier between the linear, the quadratic, and the nearest-means classifiers. In this article we propose an alternative approach, called EDDA, that is based on the reparameterization of the covariance matrix [Σk] of a group Gk in terms of its eigenvalue decomposition Σk = λkDkAkD′k, where λk specifies the volume of density contours of Gk, the diagonal matrix of eigenvalues specifies its shape, and the eigenvectors specify its orientation. Variations on constraints concerning volumes, shapes, and orientations λk, Ak, and Dk lead to 14 discrimination models of interest. For each model, we derived the normal theory maximum likelihood parameter estimates. Our approach consists of selecting a model by minimizing the sample-based estimate of future misclassification risk by cross-validation. Numerical experiments on simulated and real data show favorable behavior of this approach compared to RDA.

KW - Covariance matrix

KW - Maximum likelihood

KW - Normal-based classification

KW - Spectral decomposition

UR - http://www.scopus.com/inward/record.url?scp=0030326891&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030326891&partnerID=8YFLogxK

M3 - Article

VL - 91

SP - 1743

EP - 1748

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 436

ER -