具有输入数据注意力机制的卷积神经网络用于氟化工产品质量预测

doi:10.16085/j.issn.1000-6613.2021-0611

化工进展 ›› 2022, Vol. 41 ›› Issue (2): 593-600.DOI: 10.16085/j.issn.1000-6613.2021-0611

具有输入数据注意力机制的卷积神经网络用于氟化工产品质量预测

李欣铜¹^,²(), 陈志冰³, 魏志强⁴, 李苏桐⁴, 陈旭¹, 宋凯¹()

^1.天津大学化工学院，天津 300350
^2.航天长征化学工程股份有限公司，北京 100176
^3.巨化清安检测科技有限公司，浙江衢州 324004
^4.巨化股份有限公司，浙江衢州 324004

收稿日期:2021-03-26 修回日期:2021-08-10 出版日期:2022-02-05 发布日期:2022-02-23
通讯作者: 宋凯
作者简介:李欣铜（1995—），男，硕士研究生，研究方向为先进控制。E-mail：3014207218@tju.edu.cn。
基金资助:
国家重点研发计划(2018YFC0808600)

Convolution neural network with attention mechanism of input data for quality prediction of fluorine chemical products

LI Xintong¹^,²(), CHEN Zhibing³, WEI Zhiqiang⁴, LI Sutong⁴, CHEN Xu¹, SONG Kai¹()

^1.School of Chemical Engineering, Tianjin University, Tianjin 300350, China
^2.Changzheng Engineering Co. , Ltd. , Beijing 100176, China
^3.Zhejiang Juhua Qing’an Testing Technology Co. , Ltd. , Quzhou 324004, Zhejiang, China
^4.Zhejiang Juhua Co. , Ltd. , Quzhou 324004, Zhejiang, China

Received:2021-03-26 Revised:2021-08-10 Online:2022-02-05 Published:2022-02-23
Contact: SONG Kai

摘要/Abstract

摘要：

在氟化工等复杂的化工过程中，具有不同时间尺度的时变特性同时存在并作用于系统运行。这类复杂的强时变特性严重制约着各种先进控制策略的广泛应用。为了克服关键质量变量测量滞后所带来的不利因素，进一步提高氟化工过程先进控制系统的控制精度，本文提出了一种具有输入数据注意力机制的卷积神经网络（ACNN）并用于产品质量预测。通过引入注意力机制自适应地提取不同时间跨度输入数据的时间特性，来克服常规卷积神经网络因输入数据窗口固定而无法充分利用各类时变尺度特性的弊端，从而更为精准地提取氟化工过程复杂的强时变特性，更加准确地预测产品质量，辅助工业生产。应用氟化工过程真实数据和TE（Tennessee Eastman）模拟数据验证了方法的有效性和泛化性，结果表明对于强时变或同时具有长时间跨度的漂移波动而言，ACNN的质量预测模型具有更高的可靠性。

关键词: 注意力机制, 卷积神经网络, 氟化工过程, 质量预测

Abstract:

For a complicated chemical process, i.e. the fluorochemical process, the simultaneous existence of the time-varying processes with different time characteristics makes regular machine learning methods unable to predict product quality precisely. In this study, a convolutional neural network with attention mechanism of input data (ACNN) was proposed to improve the prediction of the product quality. By introducing the adaptive attention mechanism on input data at different-time span, this method can simultaneously extract the characteristics of time-varying processes. Therefore, it can overcome the abovementioned drawbacks of the regular convolution neural network. This advantage allows the possibility for ACNN to accurately extract the features of strong and complicated time-varying fluorine chemical process, and to further precisely predict the quality of products to improve the performance of the industrial control system. The performance of ACNN was strongly proved by the application in quality prediction of the fluorine chemical process located in East China. The application of it in the TE (Tennessee Eastman) benchmark also proved its generalization in the applications on other chemical processes. The results showed that the accuracy of ACNN was higher for strong time-varying or long time-span fluctuations as compared to the conventional methods.

Key words: attention mechanism, convolution neural network, fluorochemical process, quality prediction

中图分类号:

TQ12

李欣铜, 陈志冰, 魏志强, 李苏桐, 陈旭, 宋凯. 具有输入数据注意力机制的卷积神经网络用于氟化工产品质量预测[J]. 化工进展, 2022, 41(2): 593-600.

LI Xintong, CHEN Zhibing, WEI Zhiqiang, LI Sutong, CHEN Xu, SONG Kai. Convolution neural network with attention mechanism of input data for quality prediction of fluorine chemical products[J]. Chemical Industry and Engineering Progress, 2022, 41(2): 593-600.

图/表 9

参考文献 22

1	ALBERGARIA J T, MARTINS F G, ALVIM-FERRAZ M C M, et al. Multiple linear regression and artificial neural networks to predict time and efficiency of soil vapor extraction[J]. Water, Air, & Soil Pollution, 2014, 225(8): 1-9.
2	MEMARIAN A, VARANASI S K, HUANG B. Mixture robust semi-supervised probabilistic principal component regression with missing input data[J]. Chemometrics and Intelligent Laboratory Systems, 2021, 214: 104315.
3	SUGIYAMA M, IDÉ T, NAKAJIMA S, et al. Semi-supervised local Fisher discriminant analysis for dimensionality reduction[J]. Machine Learning, 2010, 78(1/2): 35-61.
4	KANEKO H, ARAKAWA M, FUNATSU K. Erratum for “development of a new regression analysis method using independent component analysis”[J]. Journal of Chemical Information and Modeling, 2013, 53(11): 3113.
5	ADEBIYI O A, CORRIPIO A B. Dynamic neural networks partial least squares (DNNPLS) identification of multivariable processes[J]. Computers & Chemical Engineering, 2003, 27(2): 143-155.
6	WILSON D J H, IRWIN G W, LIGHTBODY G. Nonlinear PLS using radial basis functions[J]. Transactions of the Institute of Measurement and Control, 1997, 19(4): 211-220.
7	XIE Y, ZHANG Y W, JIA Q L, et al. Fault detection based on probabilistic kernel partial least square regression for industrial processes[J]. Journal of Chemical Engineering of Japan, 2018, 51(1): 89-99.
8	HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
9	HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.
10	ZHANG W, LI C H, PENG G L, et al. A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load[J]. Mechanical Systems and Signal Processing, 2018, 100: 439-453.
11	HOANG D T, KANG H J. Rolling element bearing fault diagnosis using convolutional neural network and vibration image[J]. Cognitive Systems Research, 2019, 53: 42-50.
12	FUKUSHIMA K. Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4): 193-202.
13	GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377.
14	KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL]. 2014: arXiv: 1412.6980 [cs.LG].
15	ZEILER M D. ADADELTA: an adaptive learning rate method[EB/OL]. 2012: arXiv: 1212.5701[cs.LG].
16	DAUPHIN Y N, DE VRIES H, CHUNG J, et al. RMSProp and equilibrated adaptive learning rates for non-convex optimization[EB/OL]. 2015: arXiv: 1502.04390[cs.LG].
17	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
18	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
19	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 2818-2826.
20	CHAKRABARTY S, HABETS E A P. Multi-speaker DOA estimation using deep convolutional networks trained with noise signals[J]. IEEE Journal of Selected Topics in Signal Processing, 2019, 13(1): 8-21.
21	BATHELT A, RICKER N L, JELALI M. Revision of the Tennessee Eastman process model[J]. IFAC-PapersOnLine, 2015, 48(8): 309-314.
22	ZHANG Y W, TENG Y D, ZHANG Y. Complex process quality prediction using modified kernel partial least squares[J]. Chemical Engineering Science, 2010, 65(6): 2153-2158.

模型	MSE（训练）	MSE（测试）
KPLS	0.3734	0.5260
O-ICA-KPLS	0.1271	0.2285
CNN	0.0113	0.028
SE-NET	0.0107	0.026
ACNN	0.0094	0.013

模型	MSE（训练）	MSE（测试）
KPLS	0.3734	0.5260
O-ICA-KPLS	0.1271	0.2285
CNN	0.0113	0.028
SE-NET	0.0107	0.026
ACNN	0.0094	0.013

序号	类型	标签号
1	PV	2FI0023.PV
2	PV	2FI0072.PV
3	PV	2TI0090B.PV
4	PV	2TIC0160.PV
5	PV	2PI0182.PV
6	PV	2LIC0210.PV
7	PV	2TI0168.PV
8	PV	2TI0024.PV
9	PV	2TIC0184.PV
10	PV	2PIC0184.PV

序号	类型	标签号
1	PV	2FI0023.PV
2	PV	2FI0072.PV
3	PV	2TI0090B.PV
4	PV	2TIC0160.PV
5	PV	2PI0182.PV
6	PV	2LIC0210.PV
7	PV	2TI0168.PV
8	PV	2TI0024.PV
9	PV	2TIC0184.PV
10	PV	2PIC0184.PV

模型	结构	MSE
1	Conv（64）-Pool-FC（1024）-FC（1）	0.7479
2	Conv（32）-Conv（64）-Pool-FC（1024）-FC（1）	0.4223
3	Conv（32）-Conv（64）-Conv（128）-Pool-FC（1024）-FC（1）	0.0210
4	Conv（32）-Conv（64）-Pool-Conv（128）-Conv（128）-Pool-FC（1024）-FC（1）	0.0051
5	Conv（32）-Conv（64）-Conv（128）-Pool-Conv（128）-Conv（128）-Pool-FC（1024）-FC（1）	0.0063
6	Conv（32）-Conv（64）-Conv（128）-Pool-Conv（128）-Conv（128）- Conv（128）-Pool-FC（1024）-FC（1）	0.0103

具有输入数据注意力机制的卷积神经网络用于氟化工产品质量预测

Convolution neural network with attention mechanism of input data for quality prediction of fluorine chemical products

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 22

相关文章 3

编辑推荐

Metrics

本文评价

模型	MSE（训练）	MSE（测试）
KPLS	0.1941	0.3130
O-ICA-KPLS	0.0376	0.0975
CNN	0.0044	0.0114
SE-NET	0.0041	0.0117
ACNN	0.0029	0.0051

[1]	张建伟, 许蕊, 张忠闯, 董鑫, 冯颖. 基于卷积神经网络的撞击流反应器浓度场混合特性[J]. 化工进展, 2023, 42(2): 658-668.
[2]	徐日辛, 周骛, 张翔云. 基于深度学习的显微离焦图像法颗粒深度测量[J]. 化工进展, 2021, 40(12): 6499-6504.
[3]	李艳红1，2，赵文波2，常丽萍1，王美君1，王平艳2，杨荣2. 炼焦机理和焦炭质量预测的研究进展[J]. 化工进展, 2014, 33(05): 1142-1150.