On-line multiscale filtering of random and gross errors without process models

Mohamed Nounou, Bhavik R. Bakshi

Research output: Contribution to journalReview article

98 Citations (Scopus)

Abstract

Data Rectification by univariate filtering is popular for processes lacking an accurate model. Linear filters are most popular for online filtering; however, they are single-scale best suited for rectifying data containing features and noise that are at the same resolution in time and frequency. Consequently, for multiscale data, linear filters are forced to trade off the extent of noise removal with the accuracy of the features retained. In contrast, nonlinear filtering methods, such as FMH and wavelet thresholding, are multiscale, but they cannot be used for online rectification. A technique is presented for online nonlinear filtering based on wavelet thresholding. OLMS rectification applies wavelet thresholding to data in a moving window of dyadic length to remove random errors. Gross errors are removed by combining wavelet thresholding with multiscale median filtering. Theoretical analysis shows that OLMS rectification using Haar wavelets subsumes mean filters of dyadic length, while rectification with smoother boundary corrected wavelets is analogous to adaptive exponential smoothing. If the rectified measurements are not needed online, the quality of rectification can be further improved by averaging the rectified signals in each window, overcoming the boundary effects encountered in TI rectification. Synthetic and industrial data show the benefits of the online multiscale and boundary corrected translation invariant rectification methods.

Original languageEnglish
Pages (from-to)1041-1058
Number of pages18
JournalAICHE Journal
Volume45
Issue number5
DOIs
Publication statusPublished - 1 Jan 1999
Externally publishedYes

Fingerprint

Nonlinear filtering
Noise
Random errors

ASJC Scopus subject areas

  • Biotechnology
  • Environmental Engineering
  • Chemical Engineering(all)

Cite this

On-line multiscale filtering of random and gross errors without process models. / Nounou, Mohamed; Bakshi, Bhavik R.

In: AICHE Journal, Vol. 45, No. 5, 01.01.1999, p. 1041-1058.

Research output: Contribution to journalReview article

@article{9b953242661246599a14b0993d4d136e,
title = "On-line multiscale filtering of random and gross errors without process models",
abstract = "Data Rectification by univariate filtering is popular for processes lacking an accurate model. Linear filters are most popular for online filtering; however, they are single-scale best suited for rectifying data containing features and noise that are at the same resolution in time and frequency. Consequently, for multiscale data, linear filters are forced to trade off the extent of noise removal with the accuracy of the features retained. In contrast, nonlinear filtering methods, such as FMH and wavelet thresholding, are multiscale, but they cannot be used for online rectification. A technique is presented for online nonlinear filtering based on wavelet thresholding. OLMS rectification applies wavelet thresholding to data in a moving window of dyadic length to remove random errors. Gross errors are removed by combining wavelet thresholding with multiscale median filtering. Theoretical analysis shows that OLMS rectification using Haar wavelets subsumes mean filters of dyadic length, while rectification with smoother boundary corrected wavelets is analogous to adaptive exponential smoothing. If the rectified measurements are not needed online, the quality of rectification can be further improved by averaging the rectified signals in each window, overcoming the boundary effects encountered in TI rectification. Synthetic and industrial data show the benefits of the online multiscale and boundary corrected translation invariant rectification methods.",
author = "Mohamed Nounou and Bakshi, {Bhavik R.}",
year = "1999",
month = "1",
day = "1",
doi = "10.1002/aic.690450513",
language = "English",
volume = "45",
pages = "1041--1058",
journal = "AICHE Journal",
issn = "0001-1541",
publisher = "American Institute of Chemical Engineers",
number = "5",

}

TY - JOUR

T1 - On-line multiscale filtering of random and gross errors without process models

AU - Nounou, Mohamed

AU - Bakshi, Bhavik R.

PY - 1999/1/1

Y1 - 1999/1/1

N2 - Data Rectification by univariate filtering is popular for processes lacking an accurate model. Linear filters are most popular for online filtering; however, they are single-scale best suited for rectifying data containing features and noise that are at the same resolution in time and frequency. Consequently, for multiscale data, linear filters are forced to trade off the extent of noise removal with the accuracy of the features retained. In contrast, nonlinear filtering methods, such as FMH and wavelet thresholding, are multiscale, but they cannot be used for online rectification. A technique is presented for online nonlinear filtering based on wavelet thresholding. OLMS rectification applies wavelet thresholding to data in a moving window of dyadic length to remove random errors. Gross errors are removed by combining wavelet thresholding with multiscale median filtering. Theoretical analysis shows that OLMS rectification using Haar wavelets subsumes mean filters of dyadic length, while rectification with smoother boundary corrected wavelets is analogous to adaptive exponential smoothing. If the rectified measurements are not needed online, the quality of rectification can be further improved by averaging the rectified signals in each window, overcoming the boundary effects encountered in TI rectification. Synthetic and industrial data show the benefits of the online multiscale and boundary corrected translation invariant rectification methods.

AB - Data Rectification by univariate filtering is popular for processes lacking an accurate model. Linear filters are most popular for online filtering; however, they are single-scale best suited for rectifying data containing features and noise that are at the same resolution in time and frequency. Consequently, for multiscale data, linear filters are forced to trade off the extent of noise removal with the accuracy of the features retained. In contrast, nonlinear filtering methods, such as FMH and wavelet thresholding, are multiscale, but they cannot be used for online rectification. A technique is presented for online nonlinear filtering based on wavelet thresholding. OLMS rectification applies wavelet thresholding to data in a moving window of dyadic length to remove random errors. Gross errors are removed by combining wavelet thresholding with multiscale median filtering. Theoretical analysis shows that OLMS rectification using Haar wavelets subsumes mean filters of dyadic length, while rectification with smoother boundary corrected wavelets is analogous to adaptive exponential smoothing. If the rectified measurements are not needed online, the quality of rectification can be further improved by averaging the rectified signals in each window, overcoming the boundary effects encountered in TI rectification. Synthetic and industrial data show the benefits of the online multiscale and boundary corrected translation invariant rectification methods.

UR - http://www.scopus.com/inward/record.url?scp=0032943498&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032943498&partnerID=8YFLogxK

U2 - 10.1002/aic.690450513

DO - 10.1002/aic.690450513

M3 - Review article

VL - 45

SP - 1041

EP - 1058

JO - AICHE Journal

JF - AICHE Journal

SN - 0001-1541

IS - 5

ER -