Kernel spectral clustering and applications

Rocco Langone, RaghvenPhDa Mall, Carlos Alzate, Johan A.K. Suykens

Research output: Chapter in Book/Report/Conference proceedingChapter

13 Citations (Scopus)

Abstract

In this chapter we review the main literature related to kernel spectral clustering (KSC), an approach to clustering cast within a kernel-based optimization setting. KSC represents a least-squares support vector machine-based formulation of spectral clustering described by a weighted kernel PCA objective. Just as in the classifier case, the binary clustering model is expressed by a hyperplane in a high dimensional space induced by a kernel. In addition, the multi-way clustering can be obtained by combining a set of binary decision functions via an Error Correcting Output Codes (ECOC) encoding scheme. Because of its model-based nature, the KSC method encompasses three main steps: training, validation, testing. In the validation stage model selection is performed to obtain tuning parameters, like the number of clusters present in the data. This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics. Once a KSC model is trained on a small subset of the entire data, it is able to generalize well to unseen test points. Beyond the basic formulation, sparse KSC algorithms based on the Incomplete Cholesky Decomposition (ICD) and L 0, L1, L0+L1, Group Lasso regularization are reviewed. In that respect, we show how it is possible to handle large-scale data. Also, two possible ways to perform hierarchical clustering and a soft clustering method are presented. Finally, real-world applications such as image segmentation, power load time-series clustering, document clustering, and big data learning are considered.

Original languageEnglish
Title of host publicationUnsupervised Learning Algorithms
PublisherSpringer International Publishing
Pages135-161
Number of pages27
ISBN (Electronic)9783319242118
ISBN (Print)9783319242095
DOIs
Publication statusPublished - 1 Jan 2016
Externally publishedYes

Fingerprint

Image segmentation
Clustering algorithms
Support vector machines
Time series
Classifiers
Tuning
Decomposition
Testing
Big data

ASJC Scopus subject areas

  • Engineering(all)
  • Computer Science(all)

Cite this

Langone, R., Mall, R., Alzate, C., & Suykens, J. A. K. (2016). Kernel spectral clustering and applications. In Unsupervised Learning Algorithms (pp. 135-161). Springer International Publishing. https://doi.org/10.1007/978-3-319-24211-8_6

Kernel spectral clustering and applications. / Langone, Rocco; Mall, RaghvenPhDa; Alzate, Carlos; Suykens, Johan A.K.

Unsupervised Learning Algorithms. Springer International Publishing, 2016. p. 135-161.

Research output: Chapter in Book/Report/Conference proceedingChapter

Langone, R, Mall, R, Alzate, C & Suykens, JAK 2016, Kernel spectral clustering and applications. in Unsupervised Learning Algorithms. Springer International Publishing, pp. 135-161. https://doi.org/10.1007/978-3-319-24211-8_6
Langone R, Mall R, Alzate C, Suykens JAK. Kernel spectral clustering and applications. In Unsupervised Learning Algorithms. Springer International Publishing. 2016. p. 135-161 https://doi.org/10.1007/978-3-319-24211-8_6
Langone, Rocco ; Mall, RaghvenPhDa ; Alzate, Carlos ; Suykens, Johan A.K. / Kernel spectral clustering and applications. Unsupervised Learning Algorithms. Springer International Publishing, 2016. pp. 135-161
@inbook{ac8086d153784432b09c57cb29717140,
title = "Kernel spectral clustering and applications",
abstract = "In this chapter we review the main literature related to kernel spectral clustering (KSC), an approach to clustering cast within a kernel-based optimization setting. KSC represents a least-squares support vector machine-based formulation of spectral clustering described by a weighted kernel PCA objective. Just as in the classifier case, the binary clustering model is expressed by a hyperplane in a high dimensional space induced by a kernel. In addition, the multi-way clustering can be obtained by combining a set of binary decision functions via an Error Correcting Output Codes (ECOC) encoding scheme. Because of its model-based nature, the KSC method encompasses three main steps: training, validation, testing. In the validation stage model selection is performed to obtain tuning parameters, like the number of clusters present in the data. This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics. Once a KSC model is trained on a small subset of the entire data, it is able to generalize well to unseen test points. Beyond the basic formulation, sparse KSC algorithms based on the Incomplete Cholesky Decomposition (ICD) and L 0, L1, L0+L1, Group Lasso regularization are reviewed. In that respect, we show how it is possible to handle large-scale data. Also, two possible ways to perform hierarchical clustering and a soft clustering method are presented. Finally, real-world applications such as image segmentation, power load time-series clustering, document clustering, and big data learning are considered.",
author = "Rocco Langone and RaghvenPhDa Mall and Carlos Alzate and Suykens, {Johan A.K.}",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-24211-8_6",
language = "English",
isbn = "9783319242095",
pages = "135--161",
booktitle = "Unsupervised Learning Algorithms",
publisher = "Springer International Publishing",

}

TY - CHAP

T1 - Kernel spectral clustering and applications

AU - Langone, Rocco

AU - Mall, RaghvenPhDa

AU - Alzate, Carlos

AU - Suykens, Johan A.K.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - In this chapter we review the main literature related to kernel spectral clustering (KSC), an approach to clustering cast within a kernel-based optimization setting. KSC represents a least-squares support vector machine-based formulation of spectral clustering described by a weighted kernel PCA objective. Just as in the classifier case, the binary clustering model is expressed by a hyperplane in a high dimensional space induced by a kernel. In addition, the multi-way clustering can be obtained by combining a set of binary decision functions via an Error Correcting Output Codes (ECOC) encoding scheme. Because of its model-based nature, the KSC method encompasses three main steps: training, validation, testing. In the validation stage model selection is performed to obtain tuning parameters, like the number of clusters present in the data. This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics. Once a KSC model is trained on a small subset of the entire data, it is able to generalize well to unseen test points. Beyond the basic formulation, sparse KSC algorithms based on the Incomplete Cholesky Decomposition (ICD) and L 0, L1, L0+L1, Group Lasso regularization are reviewed. In that respect, we show how it is possible to handle large-scale data. Also, two possible ways to perform hierarchical clustering and a soft clustering method are presented. Finally, real-world applications such as image segmentation, power load time-series clustering, document clustering, and big data learning are considered.

AB - In this chapter we review the main literature related to kernel spectral clustering (KSC), an approach to clustering cast within a kernel-based optimization setting. KSC represents a least-squares support vector machine-based formulation of spectral clustering described by a weighted kernel PCA objective. Just as in the classifier case, the binary clustering model is expressed by a hyperplane in a high dimensional space induced by a kernel. In addition, the multi-way clustering can be obtained by combining a set of binary decision functions via an Error Correcting Output Codes (ECOC) encoding scheme. Because of its model-based nature, the KSC method encompasses three main steps: training, validation, testing. In the validation stage model selection is performed to obtain tuning parameters, like the number of clusters present in the data. This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics. Once a KSC model is trained on a small subset of the entire data, it is able to generalize well to unseen test points. Beyond the basic formulation, sparse KSC algorithms based on the Incomplete Cholesky Decomposition (ICD) and L 0, L1, L0+L1, Group Lasso regularization are reviewed. In that respect, we show how it is possible to handle large-scale data. Also, two possible ways to perform hierarchical clustering and a soft clustering method are presented. Finally, real-world applications such as image segmentation, power load time-series clustering, document clustering, and big data learning are considered.

UR - http://www.scopus.com/inward/record.url?scp=84978238265&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978238265&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-24211-8_6

DO - 10.1007/978-3-319-24211-8_6

M3 - Chapter

SN - 9783319242095

SP - 135

EP - 161

BT - Unsupervised Learning Algorithms

PB - Springer International Publishing

ER -