Dealing with collinearity in finite impulse response models using Bayesian shrinkage

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.

Original languageEnglish
Pages (from-to)292-298
Number of pages7
JournalIndustrial and Engineering Chemistry Research
Volume45
Issue number1
DOIs
Publication statusPublished - 4 Jan 2006
Externally publishedYes

Fingerprint

Impulse response
Chemical industry
Model structures
Probability density function
Redundancy

ASJC Scopus subject areas

  • Chemistry(all)
  • Chemical Engineering(all)
  • Industrial and Manufacturing Engineering

Cite this

Dealing with collinearity in finite impulse response models using Bayesian shrinkage. / Nounou, Mohamed.

In: Industrial and Engineering Chemistry Research, Vol. 45, No. 1, 04.01.2006, p. 292-298.

Research output: Contribution to journalArticle

@article{1c175f88447b45789d8f26848887f344,
title = "Dealing with collinearity in finite impulse response models using Bayesian shrinkage",
abstract = "Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.",
author = "Mohamed Nounou",
year = "2006",
month = "1",
day = "4",
doi = "10.1021/ie048897m",
language = "English",
volume = "45",
pages = "292--298",
journal = "Industrial and Engineering Chemistry Research",
issn = "0888-5885",
publisher = "American Chemical Society",
number = "1",

}

TY - JOUR

T1 - Dealing with collinearity in finite impulse response models using Bayesian shrinkage

AU - Nounou, Mohamed

PY - 2006/1/4

Y1 - 2006/1/4

N2 - Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.

AB - Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.

UR - http://www.scopus.com/inward/record.url?scp=30444435938&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=30444435938&partnerID=8YFLogxK

U2 - 10.1021/ie048897m

DO - 10.1021/ie048897m

M3 - Article

VL - 45

SP - 292

EP - 298

JO - Industrial and Engineering Chemistry Research

JF - Industrial and Engineering Chemistry Research

SN - 0888-5885

IS - 1

ER -