Improved residual network based on attention mechanism for flame temperature field reconstruction

doi:10.16085/j.issn.1000-6613.2023-1354

Abstract

Abstract:

The method of reconstructing the flame temperature field based on convolutional neural network has been widely used in recent years, but the traditional convolutional neural network model is prone to overfitting or model degradation as the number of network layers increases, resulting in large reconstruction errors. This paper proposed an improved method, which used the ResNet18 network to reconstruct the flame temperature field, and introduced the attention mechanism and local importance pooling to optimize the extracted content, realized the full use of known information, and reduced the reconstruction error. The experimental results showed that after introducing the local importance pooling and attention mechanism at the same time, the average relative error of temperature field reconstruction was 0.13%, and the maximum relative error was 0.75%. Compared with the initial ResNet18 network, the average relative error was reduced by 31.58%. The maximum relative error was reduced by 34.21%. The influence of the two factors on the reconstruction accuracy was verified by ablation experiments. The results showed that the temperature field reconstruction accuracy after adding two improved modules at the same time was better than that after adding a single improved module, and the local importance pooling module had a significant effect on the accuracy improvement.

Key words: temperature field, residual network, attention mechanism, pooling

摘要：

基于卷积神经网络重建火焰温度场的方法近年来已被广泛采用，但是传统卷积神经网络模型随着其网络层数的增加极易出现过拟合或者模型退化的现象，导致重建误差较大。本文提出一种改进的方法，使用ResNet18网络进行火焰温度场重建，并引入注意力机制和局部重要性池化，优化提取内容，实现已知信息的充分利用，减少重建误差。实验结果表明，同时引入局部重要性池化和注意力机制后，温度场重建的平均相对误差为0.13%，最大相对误差为0.75%；相较于初始ResNet18网络，平均相对误差减少了31.58%，最大相对误差减少了34.21%。通过消融实验验证了两种因素对重建精度的影响，结果表明：同时加入两个改进模块后的温度场重建精度要优于加入单个改进模块后的精度，局部重要性池化模块对精度提升的作用更大。

关键词: 温度场, 残差网络, 注意力机制, 池化

CLC Number:

TK311

SHAN Liang, ZHOU Rongxing, HONG Bo, YANG Wenqi, KONG Ming. Improved residual network based on attention mechanism for flame temperature field reconstruction[J]. Chemical Industry and Engineering Progress, 2024, 43(2): 688-695.

单良, 周荣幸, 洪波, 仰文淇, 孔明. 基于注意力机制的改进残差网络火焰温度场重建[J]. 化工进展, 2024, 43(2): 688-695.

Figures/Tables 8

References 27

1	王志宁, 杨协和, 张扬, 等. 内/外烟气再循环对天然气燃烧NO_x生成的影响[J]. 化工进展, 2019, 38(9): 4327-4334.
	WANG Zhining, YANG Xiehe, ZHANG Yang, et al. I-/e-FGR effect on NO _x emission of natural gas combustion[J]. Chemical Industry and Engineering Progress, 2019, 38(9): 4327-4334.
2	韩徳琳, 李丹, 王天天, 等. 位移钝体稳燃的旋流预混燃烧污染物生成特性[J]. 化工进展, 2022, 41(6): 2915-2923.
	HAN Delin, LI Dan, WANG Tiantian, et al. Emission characteristics of swirl premixed combustion stabilized using a displacing bluff body[J]. Chemical Industry and Engineering Progress, 2022, 41(6): 2915-2923.
3	刘煜东. 光场层析成像火焰三维温度场测量方法与系统研究[D]. 南京: 东南大学, 2020.
	LIU Yudong. Study of light field sectioning pyrometry for three-dimensional flame temperature measurement[D]. Nanjing:SoutheastUniversity, 2020.
4	闫慧博, 唐广通, 李路江, 等. 热辐射成像法测量大型炉膛内三维温度场的算法新进展[J]. 洁净煤技术, 2022, 28(5): 97-108.
	YAN Huibo, TANG Guangtong, LI Lujiang, et al. New progress of algorithm for three-dimensional temperature field in large scale furnace measured by thermal radiative imaging[J]. Clean Coal Technology, 2022, 28(5): 97-108.
5	孙俊, 许传龙, 张彪, 等. 多焦距微透镜阵列光场成像火焰三维温度场测量[J]. 工程热物理学报, 2017, 38(10): 2164-2170.
	SUN Jun, XU Chuanlong, ZHANG Biao, et al. Three-dimensional temperature measurement of the flame using the light field camera with a multiple focus microlens array[J]. Journal of Engineering Thermophysics, 2017, 38(10): 2164-2170.
6	庞维煦, 李宁, 黄孝龙, 等. 基于分数阶Tikhonov正则化的激光吸收光谱燃烧场二维重建光路优化研究[J]. 物理学报, 2023, 72(3): 314-324.
	PANG Weixu, LI Ning, HUANG Xiaolong, et al. Optimization of beam arrangement for tunable diode laser absorption tomography reconstruction based on fractional Tikhonov regularization[J]. Acta Physica Sinica, 2023, 72(3): 314-324.
7	谢正超, 王飞, 严建华, 等. 炉膛三维温度场重建中Tikhonov正则化和截断奇异值分解算法比较[J]. 物理学报, 2015, 64(24): 21-28.
	XIE Zhengchao, WANG Fei, YAN Jianhua, et al. Comparative studies of Tikhonov regularization and truncated singular value decomposition in the three-dimensional flame temperature field reconstruction[J]. Acta Physica Sinica, 2015, 64(24): 21-28.
8	单良, 赵腾飞, 黄荟云, 等. 基于阻尼LSQR-LMBC的火焰三维温度场重建[J]. 物理学报, 2022, 71(4): 21-32.
	SHAN Liang, ZHAO Tengfei, HUANG Huiyun, et al. Flame 3D temperature field reconstruction based on Damped LSQR-LMBC[J]. Acta Physica Sinica, 2022, 71(4): 21-32.
9	LEE En-Jui, HUANG He, DENNIS John M, et al. An optimized parallel LSQR algorithm for seismic tomography[J]. Computers & Geosciences, 2013, 61: 184-197.
10	HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
11	JIN Kyong Hwan, MCCANN Michael T, FROUSTEY Emmanuel, et al. Deep convolutional neural network for inverse problems in imaging[J]. IEEE Transactions on Image Processing, 2017, 26(9): 4509-4522.
12	DENG Andong, HUANG Jianqing, LIU Hecong, et al. Deep learning algorithms for temperature field reconstruction of nonlinear tomographic absorption spectroscopy[J]. Measurement: Sensors, 2020, 10/11/12: 100024.
13	张杰, 齐琪, 韩哲哲, 等. 基于深度学习和光场成像的火焰三维温度场重建算法[J]. 东南大学学报(自然科学版), 2021, 51(6): 1060-1067.
	ZHANG Jie, QI Qi, HAN Zhezhe, et al. Reconstruction algorithm of flame 3D temperature distribution based on deep learning and light field imaging[J]. Journal of Southeast University (Natural Science Edition), 2021, 51(6): 1060-1067.
14	孙安泰. 基于深度学习的火焰三维温度场层析重建及预测研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
	SUN Antai. Tomography reconstruction and prediction of three-dimensional temperature field of flame based on deep learning[D]. Harbin: Harbin Institute of Technology, 2021.
15	VEIT A, WILBER M J, BELONGIE S J. Residual networks behave like ensembles of relatively shallow networks[C]// Advances in Neural Information Processing Systems, December 5-10,2016,Barvelona,Spain.New York: Curran Associates, 2016:550-558.
16	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 770-778.
17	WANG Fei, JIANG Mengqing, QIAN Chen, et al. Residual attention network for image classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 6450-6458.
18	HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7132-7141.
19	WANG Xiaolong, GIRSHICK Ross, GUPTA Abhinav, et al. Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7794-7803.
20	CAO Yue, XU Jiarui, LIN Stephen, et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). October 27-28, 2019, Seoul, Korea (South). IEEE, 2020: 1971-1980.
21	刘梦雅, 毛剑琳. 一种改进池化模型对卷积神经网络性能影响的研究[J]. 电子测量技术, 2019, 42(5): 34-38.
	LIU Mengya, MAO Jianlin. Research on the influence of an improved pooling model on the performance of convolutional neural networks[J]. Electronic Measurement Technology, 2019, 42(5): 34-38.
22	LIN Min, CHEN Qiang, YAN Shuicheng. Network in network[EB/OL]. (2013-12-16) [2021-01-10]. .
23	GAO Ziteng, WANG Limin, WU Gangshan. LIP: Local importance-based pooling[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019, Seoul, Korea (South). IEEE, 2020: 3354-3363.
24	LI Yanghao, WANG Naiyan, SHI Jianping, et al. Adaptive Batch Normalization for practical domain adaptation[J]. Pattern Recognition, 2018, 80: 109-117.
25	胡嵽, 冯子亮. 基于深度学习的轻量级道路图像语义分割算法[J]. 计算机应用, 2021, 41(5): 1326-1331.
	HU Di, FENG Ziliang. Light-weight Road image semantic segmentation algorithm based on deep learning[J]. Journal of Computer Applications, 2021, 41(5): 1326-1331.
26	CRAMMER Koby, KULESZA Alex, DREDZE Mark. Adaptive regularization of weight vectors[J]. Machine Learning, 2013, 91(2): 155-187.
27	KINGMA Diederik P, BA Jimmy. Adam: A method for stochastic optimization[EB/OL]. (2014-12-22) [2021-02-04]. .

层数	名称	步长	核大小	激活函数	尺寸
1	输入层	—	—	—	360×360
2	卷积层	2	7×7	ReLU	180×180×32
3	最大池化层	2	3×3	—	90×90×32
4	残差块1	1	3×3	ReLU	90×90×32
5	残差块1	1	3×3	ReLU	90×90×32
6	残差块1	1	3×3	ReLU	90×90×32
7	残差块1	1	3×3	ReLU	90×90×32
8	残差块2	2	3×3	ReLU	45×45×64
9	残差块2	1	3×3	ReLU	45×45×64
10	残差块2	1	3×3	ReLU	45×45×64
11	残差块2	1	3×3	ReLU	45×45×64
12	残差块3	2	3×3	ReLU	22×22×128
13	残差块3	1	3×3	ReLU	22×22×128
14	残差块3	1	3×3	ReLU	22×22×128
15	残差块3	1	3×3	ReLU	22×22×128
16	残差块4	2	3×3	ReLU	11×11×256
17	残差块4	1	3×3	ReLU	11×11×256
18	残差块4	1	3×3	ReLU	11×11×256
19	残差块4	1	3×3	ReLU	11×11×256
20	注意力机制层	1	1×1	ReLU/Softmax	11×11×256
21	局部重要性池化层	1	3×3	sigmoid	11×11×256
22	全连接层	—	—	—	2704
23	输出层	—	—	—	13×13×16

层数	名称	步长	核大小	激活函数	尺寸
1	输入层	—	—	—	360×360
2	卷积层	2	7×7	ReLU	180×180×32
3	最大池化层	2	3×3	—	90×90×32
4	残差块1	1	3×3	ReLU	90×90×32
5	残差块1	1	3×3	ReLU	90×90×32
6	残差块1	1	3×3	ReLU	90×90×32
7	残差块1	1	3×3	ReLU	90×90×32
8	残差块2	2	3×3	ReLU	45×45×64
9	残差块2	1	3×3	ReLU	45×45×64
10	残差块2	1	3×3	ReLU	45×45×64
11	残差块2	1	3×3	ReLU	45×45×64
12	残差块3	2	3×3	ReLU	22×22×128
13	残差块3	1	3×3	ReLU	22×22×128
14	残差块3	1	3×3	ReLU	22×22×128
15	残差块3	1	3×3	ReLU	22×22×128
16	残差块4	2	3×3	ReLU	11×11×256
17	残差块4	1	3×3	ReLU	11×11×256
18	残差块4	1	3×3	ReLU	11×11×256
19	残差块4	1	3×3	ReLU	11×11×256
20	注意力机制层	1	1×1	ReLU/Softmax	11×11×256
21	局部重要性池化层	1	3×3	sigmoid	11×11×256
22	全连接层	—	—	—	2704
23	输出层	—	—	—	13×13×16

方法	平均相对误差/%	最大相对误差/%
ResNet18	0.19	1.14
ResNet18+LIP	0.16	0.81
ResNet18+GCNet	0.18	1.01
ResNet18+LIP+GCNet	0.13	0.75

方法	平均相对误差/%	最大相对误差/%
ResNet18	0.19	1.14
ResNet18+LIP	0.16	0.81
ResNet18+GCNet	0.18	1.01
ResNet18+LIP+GCNet	0.13	0.75

[1]	SHAN Liang, YANG Wenqi, HONG Bo, ZHOU Rongxing, KONG Ming. Three-dimensional temperature field reconstruction of flame based on LSMR algorithm [J]. Chemical Industry and Engineering Progress, 2024, 43(2): 659-666.
[2]	HAN Delin, LI Dan, WANG Tiantian, ZHANG Hai, ZHANG Yang, WANG Suilin. Emission characteristics of swirl premixed combustion stabilized using a displacing bluff body [J]. Chemical Industry and Engineering Progress, 2022, 41(6): 2915-2923.
[3]	LI Xintong, CHEN Zhibing, WEI Zhiqiang, LI Sutong, CHEN Xu, SONG Kai. Convolution neural network with attention mechanism of input data for quality prediction of fluorine chemical products [J]. Chemical Industry and Engineering Progress, 2022, 41(2): 593-600.
[4]	QI Qi, WANG Changjian, YI Jianhua, HUANG Yizhi, LI Jinjian, ZHANG Biao, XU Chuanlong. Approach to optimize the sampling of multi-light field camera system based on feature rays selection [J]. Chemical Industry and Engineering Progress, 2021, 40(12): 6581-6589.
[5]	Zhong XU, Jing HOU, Jun LI, Enhui WU, Ping HUANG, Qianshu LIU, Dawei XU. Preparation and performances of expanded graphite/organic matter composite phase change materials [J]. Chemical Industry and Engineering Progress, 2020, 39(7): 2758-2767.
[6]	Xiaoyan BAI,Sijian QU,Yang ZHANG,Yan WANG,Xianfeng PEI. Research of new inner-rotary moving bed technology for low rank coal pyrolysis [J]. Chemical Industry and Engineering Progress, 2020, 39(3): 984-991.
[7]	Shaowu YIN, Jiawei HAN, Yongle SHI, Lige TONG, Li WANG. Heat charge and discharge characteristics of non-phase change solid heat storage materials [J]. Chemical Industry and Engineering Progress, 2020, 39(12): 5161-5169.
[8]	Pengfei ZHAO,Xiaohui ZHANG,Han ZHANG,Peng FENG,Jiarui XU. Impact surface temperature distribution uniformity of non-isothermal liquid-liquid opposite impinging process [J]. Chemical Industry and Engineering Progress, 2019, 38(12): 5297-5305.
[9]	GUO Chongzhi，XIAO Le. A sequence coupling method for numerical simulation of temperature [J]. Chemical Industry and Engineering Progree, 2010, 29(9): 1615-.