Development of BP neural network and its application in chemistry and chemical engineering

doi:10.16085/j.issn.1000-6613.2018-2045

Abstract

Abstract:

Artificial neural network (ANN) is in universe application because of intrinsic favorable nonlinear mapping ability, fault tolerance and self-learning ability. Backpropagation (BP) neural network, an important part of ANN, has great advantages over traditional reaction mechanism modeling which deals with nonlinear multi-factor system. Currently it has been successfully applied in varying fields after experiencing intermittent prosperous and fading development historically. Herein the principle of mapping process, the shortcomings and the corresponding improvement methods of BP neural network are briefly summarized, and its applications in catalyst designing, kinetic simulation, physical and chemical properties prediction, process control and optimization, chemical synthesis and reaction performance prediction were introduced in detail. The prediction accuracy and efficiency in experiment design and process optimization based on BP neural network could be enhanced remarkably by improved algorithm. Finally, it was pointed out that BP neural network, further combined with data depth mining and machine learning techniques, could be a powerful tool for future research in chemical field.

Key words: artificial neural network, BP model, chemical application

摘要：

人工神经网络（ANN）由于本身具有极强的非线性映射能力、容错性、自学习能力得到广泛的应用。基于反向传播算法（BP）的神经网络作为ANN重要组成部分，在涉及多种非线性因素建模时，相对于传统的反应机理建模显示出巨大的优势。虽然神经网络的发展几经繁荣与冷落，但目前在不同领域已经获得成功的应用。本文概述了BP神经网络的映射原理、缺点以及相应的改进方法，介绍其在催化剂设计、动力学模拟、理化特性估算、过程控制与优化、化学合成与反应性能预测的应用现状，展示了使用不同优化方法的改进模型在实验设计与优化方面取得的成果。最后指出未来BP神经网络的发展要进一步结合数据深度挖掘与机器学习等技术，为今后化学化工领域的研究提供强有力的工具。

关键词: 人工神经网络, 反向传播算法, 化工应用

CLC Number:

TQ015.9

Fang LIU, Long XU, Xiaoxun MA. Development of BP neural network and its application in chemistry and chemical engineering[J]. Chemical Industry and Engineering Progress, 2019, 38(06): 2559-2573.

刘方, 徐龙, 马晓迅. BP神经网络的发展及其在化学化工中的应用[J]. 化工进展, 2019, 38(06): 2559-2573.

Figures/Tables 14

References 86

1	邹蕾, 张先锋 . 人工智能及其发展应用[J]. 信息网络安全, 2012 (2): 11-13.
	ZOU Lei , ZHANG Xianfeng . Development and application of artificial intelligence[J]. Netinfor Security, 2012(2): 11-13.
2	吕伟, 钟臻怡, 张伟 . 人工智能技术综述[J]. 上海电气技术, 2018, 11(1): 62-64.
	LÜ Wei, ZHONG Zhenyi , ZHANG Wei . Review of artificial intelligence technology[J]. Journal of Shanghai Electric Technology, 2018, 11(1): 62-64.
3	TURING A M . Computing machinery and intelligence[J]. Mind, 1950, 59(236): 433-460.
4	顾险峰 . 人工智能的历史回顾和发展现状[J]. 自然杂志, 2016, 38(3): 157-166.
	GU Xianfeng . Historical review and development of artificial intelligence[J]. Chinese Journal of Nature, 2016, 38(3): 157-166.
5	HEBB D O . The organization of behavior[M]. New York: John Wiley & Sons Inc., 1949: 18-37.
6	TUMELLHAR D E , MCCLELLAND J L . Parallel distributed processing[M]. Cambridge, MA: MIT Press, 1987: 235-283.
7	ECCLES J C , FATT P , KOKETSU K . Cholinergic and inhibitory synapses in a pathway from motor-axon collaterals to motoneurones[J]. The Journal of Physiology, 1954, 3(126): 524-562.
8	ROSENBLATT F . The perceptron: A probabilistic model for information storage and organization in the brain[J]. Psychological Review, 1958, 6(65): 386-408.
9	GROSSBERG S . On the serial learning of lists[J]. Mathematical Biosciences, 1969, 4(1/2): 201-253.
10	GROSSBERG S . Some networks that can learn, remember, and reproduce any number of complicated space-time[J]. Indiana University Mathematics Journal, 1970, 19(1): 135-166.
11	MINSKY M , PAPERT S . Perceptions[M]. Oxford: MIT Press, 1969: 57-89.
12	HOLLAND J H . Genetic algorithms and the optimal allocation of trials[J]. Journal on Computing, 1973, 2(2): 88-105.
13	HOLLAND J H . Adaptation in natural and artificial systems[M]. Ann Arbor: The University of Michigan Press, 1975: 118-147.
14	WERBOS P . Beyond regression: new tools for prediction and analysis in the behavioral sciences[D]. Boston: Harvard University, 1974: 89-135.
15	GROSSBERG S . Adaptive pattern classification and universal recoding: part I: parallel development and coding of neural feature detectors[J]. Biological Cybernetics, 1976, 23(3): 121-134.
16	FUKUSHIMA K . Neocognitron: a self-organizing multilayered neural network[J]. Biological Cybernetics, 1980, 37(4): 193-202.
17	HOPFIELD J J . Neural networks and physical systems with emergent collective computational abilities[J]. Proceedings of the National Academy of Science, 1982, 79(8): 2254-2558.
18	KIRKPATRICK S , GELLAT C D , VEECHI M P . Optimization by simulated annealing[J]. Science, 1983, 220(4598): 671-681.
19	ACKLEY D H , HINTON G E , SEJNOWSKI T J . A learning algorithm for Boltzmann machines[J]. Cognitive Science, 1985, 9(1): 147-169.
20	HINTON G E , SEJNOWSKI T J . Learning and relearning in Boltzmann machines[M]. Cambridge, MA: MIT Press, 1986: 282-317.
21	WERBOS P J . Backpropagation through time: what it does and how to do it[J]. Proceedings of the IEEE, 1990, 78(10): 1550-1560.
22	BROOMHEAD D S , LOWE D . Multivariable functional interpolation and adaptive networks[J]. Complex Systems, 1988, 2(3): 321-355.
23	LI T Y , YORKE J A . Period three implies chaos[J]. America Mathematics Monthly, 1975, 82(10): 985-992.
24	ZHANG Q , BENVENISTE A .Wavelet networks[J].IEEE Transactions on Neural Networks,1992,3(6):889-898.
25	韩力群 . 人工神经网络理论、设计及应用[M]. 北京:化学工业出版社, 2007: 1-4.
	HAN Liqun . Artificial neural network theory, design and application[M]. Beijing: Chemical Industry Press, 2007: 1-4.
26	谢良才 . 基于BP神经网络的煤热解特性及煤灰熔融特性研究[D]. 西安: 西北大学, 2018.
	XIE Liangcai . Study on pyrolysis and ash fusion characteristics of coal based on BP neural network[D]. Xi’an: Northwestern University, 2018.
27	RUMELHART D E , HINTON G E , WILLIAMS R J . Learning representations by back-propagating errors[J]. Nature, 1986, 323: 533-536.
28	LEONARD J , KRAMER M A .Improvement of the backpropagation algorithm for training neural networks[J].Computers&Chemical Engineering,1990,14(3):337-341
29	FORESEE F D , HAGAN M T .Gauss-newton approximation to bayesian learning [C]//International Conference on Neural Networks.1997.
30	EBERHART R , KENNEDY J . A new optimizer using particle swarm theory[C]//Proceedings of the 6th International Symposium on Micro Machine and Human Science. Nagoya, Japan: IEEE Service Center, 1995: 39-43.
31	胥良, 白晓磊 . 模拟退火BP神经网络在矿用通风机故障诊断中的应用[J].黑龙江科技大学学报, 2018, 28(5): 582-586.
	XU Liang , BAI Xiaolei . Application of simulated annealing BP neural network in fault diagnosis of mine ventilato[J]. Journal of Heilongjiang University of Science and Technology, 2018, 28(5): 582-586.
32	SHAO D , NONG X , TAN X ,et al . Daily water quality forecast of the south-to-north water diversion project of China based on the cuckoo search-back propagation neural network[J]. Water, 2018, 10(10): 1471-1499.
33	CHEN S , FANG G , HUANG X , et al . Water quality prediction model of a water diversion project based on the improved artificial bee colony–backpropagation neural network[J]. Water, 2018, 10(6): 806-825.
34	WANG Deyun , LUO Hongyuan , Grunder OLIVIER , et al . Multi-step ahead electricity price forecasting using a hybrid model based on two-layer decomposition technique and BP neural network optimized by firefly algorithm[J]. Applied Energy, 2017, 190: 390-407.
35	YANG Li , ZHAO Ji , SHIJUN Ji . Thermal positioning error modeling of machine tools using a bat algorithm-based back propagation neural network[J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(5/6/7/8): 2575-2586.
36	MIRJALILI S , MIRJALILI S M , LEWIS A . Grey wolf optimizer[J]. Advances in Engineering Software, 2014, 69: 46-61.
37	ZHUO L , ZHANG J , DONG P , et al . An SA-GA-BP neural network-based color correction algorithm for TCM tongue images[J]. Neurocomputing, 2014, 134: 111-116.
38	冯少荣 . 决策树算法的研究与改进[J].厦门大学学报(自然科学版), 2007(4): 496-500.
	FENG Shaorong . Research and improvement of decision tree algorithms[J]. Journal of Xiamen University (Natural Science) , 2007(4): 496-500.
39	方匡南, 吴见彬, 朱建平, 等 .随机森林方法研究综述[J]. 统计与信息论坛, 2011, 26(3): 32-38.
	FANG Kuangnan , WU Jianbian , ZHU Jianping , et al . A review of technologies on random forests[J]. Statistics & Information Forum, 2011, 26(3): 32-38.
40	尾崎萃 . 催化剂手册[M]. 北京: 化学工业出版社, 1982: 45-87.
	WEIQI Cui . Handbook of catalyst[M]. Beijing: Chemical Industry Press, 1982: 45-87.
41	金松寿 . 有机催化[M]. 上海: 上海科技出版社, 1986: 26-73.
	JIN Songshou . Organic catalysis[M]. Shanghai: Shanghai Scientific & Technical Publishers, 1986: 26-73.
42	CHINMOY B , DALAI A K . Esterification of free fatty acids (FFA) of green seed canola (GSC) oil using H-Y zeolite supported 12-tungstophosphoric acid (TPA)[J]. Applied Catalysis A: General, 2014, 485: 99-107.
43	吴彩霞, 方敏, 余贤旺, 等 .基于正交设计与BP神经网络优化制备纳米WC-MgO复合粉末[J]. 世界有色金属, 2017(16): 1-5.
	WU Caixia , FANG Min , YU Xianwang , et al . Optimizing preparation of nanocomposite WC-MgO powders based on orthogonal design and back-propagation BP neural network [J]. World Nonferrous Metals, 2017(16): 1-5.
44	黄凯, 陈勇 ,母志为, 等 .基于人工神经网络和遗传算法的甲烷制氢催化剂设计[J]. 化工学报, 2016, 67(8): 3481-3490.
	HUANG Kai , CHEN Yong , MU Zhiwei , et al . Catalyst design for production of hydrogen from methane based on artificial neural network and genetic algorithm[J]. CIESC Journal, 2016, 67(8): 3481-3490.
45	ABBASI M , NIAEI A , SALARI D , et al . Modeling and optimization of synthesis parameters in nanostructure La₁₋ _x Ba _x Ni₁₋ _y Cu _y O₃, catalysts used in the reforming of methane with CO₂ [J]. Journal of the Taiwan Institute of Chemical Engineers, 2017, 74: 187-195.
46	HADI N , NIAEI A , NABAVI S R , et al . An intelligent approach to design and optimization of M-Mn/H-ZSM-5 (M: Ce, Cr, Fe, Ni) catalysts in conversion of methanol to propylene[J]. Journal of the Taiwan Institute of Chemical Engineers, 2016, 59: 173-185.
47	SUNPHORKA S , CHALERMSINSUWAN B , PIUMSOMBOON P . Artificial neural network model for the prediction of kinetic parameters of biomass pyrolysis from its constituents[J]. Fuel, 2017, 193: 142-158.
48	XING J , LUO K , PITSCH H , et al . Predicting kinetic parameters for coal devolatilization by means of artificial neural networks[J]. Proceedings of the Combustion Institute, 2018, 37(3): 2943-2950.
49	MURAVYEV N V , PIVKINA A N . New concept of thermokinetic analysis with artificial neural networks[J]. Thermochimica Acta, 2016, 637: 69-73.
50	FATTAHI M , KAZEMEINI M , KHORASHEH F , et al . Kinetic modeling of oxidative dehydrogenation of propane (ODHP) over a vanadium–graphene catalyst: application of the DOE and ANN methodologies[J]. Journal of Industrial & Engineering Chemistry, 2014, 20(4): 2236-2247.
51	SUN Y , YANG G , ZHANG L , et al . Fischer-Trospch synthesis using iron-based catalyst in a microchannel reactor: hybrid lump kinetic with ANNs/RSM[J]. Chemical Engineering & Processing, 2017, 122: 181-189.
52	高保娇, 薛永强 . 对应状态原理及其应用[J]. 中北大学学报(自然科学版), 1997(4): 369-373.
	GAO Baojiao , XUE Yongqiang . Corresponding state principle and its application[J]. Journal of North China Institute of Technology (Natural Science Edition), 1997(4): 369-373.
53	高保娇, 薛永强 . 基团贡献法估算物性的进展[J]. 山西化工, 1997(3): 21-25.
	GAO Baojiao , XUE Yongqiang . Development of estimating physical properties by group contribution method[J]. Shanxi Chemical Industry, 1997 (3): 21-25.
54	潘勇, 蒋军成, 曹洪印, 等 .基于神经网络的定量结构-性质相关性研究预测有机物燃烧特性[J]. 化工进展, 2008, 27(3): 378-384.
	PAN Yong , JIANG Juncheng , CAO Hongyin ,et al . Prediction of flammability characteristics by using quantitative structure-property relationship study based on neural network[J]. Chemical Industry and Engineering Progress, 2008, 27(3): 378-384.
55	GUO Z , LIM K H, CHEN M , et al . Predicting cetane numbers of hydrocarbons and oxygenates from highly accessible descriptors by using artificial neural networks[J]. Fuel, 2017, 207: 344-351.
56	KESSLER T , SACIA E R , BELL A T , et al . Artificial neural network based predictions of cetane number for furanic biofuel additives[J]. Fuel, 2017, 206: 171-179.
57	ROCABRUNOVALDÉS C I , RAMÍREZVERDUZCO L F , HERNÁNDEZ J A . Artificial neural network models to predict density, dynamic viscosity, and cetane number of biodiesel[J]. Fuel, 2015, 147: 9-17.
58	ASANTE-OKYERE S , QIANG X , MENSAH R A , et al . Generalized regression and feed forward back propagation neural networks in modelling flammability characteristics of polymethyl methacrylate (PMMA)[J]. Thermochimica Acta, 2018, 667: 79-92.
59	PADUSZYŃSKI K . Extensive evaluation of the conductor-like screening model for real solvents method in predicting liquid–liquid equilibria in ternary systems of ionic liquids with molecular compounds[J]. The Journal of Physical Chemistry B, 2018, 122(14): 4016-4028.
60	谭均权,涂永善,刘子媛 .人工神经网络法预测轻质油品的闪点[J]. 石化技术与应用, 2016, 34(1): 24-28.
	TAN Junquan , TU Yongshan , LIU Ziyuan . Prediction of flash point of light fractions by artificial neural network model[J]. Petrochemical Technology & Application, 2016, 34(1): 24-28.
61	谢良才, 李风海, 薛兆民,等 . 配煤对高熔点煤灰熔融特性影响的研究[J]. 燃料化学学报, 2016, 44(12): 1430-1439.
	XIE Liangcai , LI Fenghai , XUE Zhaomin , et al . Influence of coal blending on ash fusion characteristics for coal with high ash fusion temperature[J]. Journal of Fuel Chemistry and Technology, 2016, 44(12): 1430-1439.
62	CHENG J , WANG X , SI T , et al . Maximum burning rate and fixed carbon burnout efficiency of power coal blends predicted with back-propagation neural network models[J]. Fuel, 2016, 172: 170-177.
63	LIU Hui , MI Xiwei , LI Yanfei . Comparison of two new intelligent wind speed forecasting approaches based on wavelet packet decomposition, complete ensemble empirical mode decomposition with adaptive noise and artificial neural networks[J]. Energy Conversion and Management, 2018, 155: 188-200.
64	孟庆武, 赵宏伟, 孙润诚 . 基于BP神经网络算法的电池组故障诊断研究[J]. 自动化与仪器仪表, 2017(11): 45-47.
	MENG Qingwu , ZHAO Hongwei , SUN Runcheng . Research on battery battery fault diagnosis based on BP neural network[J]. Automation & Instrumentation, 2017(11): 45-47.
65	李为民, 徐春明, 高国生，等 .基于神经网络-遗传算法优化制氢工艺水碳比[J]. 化工进展, 2004, 23(9): 998-1000.
	LI Weimin , XU Chunming , GAO Guosheng , et al . Optimization of water/carbon ratio in hydrogen production process based on neural network-genetic algorithm[J]. Chemical Industry and Engineering Progress, 2004, 23(9): 998-1000.
66	PATIL-SHINDE V , KULKARNI T , KULKARNI R , et al . Artificial intelligence-based modeling of high ash coal gasification in a pilot plant scale fluidized bed gasifier[J]. Industrial & Engineering Chemistry Research, 2014, 53(49): 18678-18689.
67	AZARPOUR A , BORHANI T N G , ALWI S R W , et al . A generic hybrid model development for process analysis of industrial fixed-bed catalytic reactors[J]. Chemical Engineering Research & Design, 2017, 117: 149-167.
68	AZZAM M , ARAMOUNI N A K , AHMAD M N ,et al . Dynamic optimization of dry reformer under catalyst sintering using neural networks[J]. Energy Conversion and Management, 2018, 157: 146-156.
69	ARAMOUNI N A K , ZEAITER J , KWAPINSKI W , et al . Thermodynamic analysis of methane dry reforming: effect of the catalyst particle size on carbon formation[J]. Energy Conversion & Management, 2017, 150: 614-622.
70	MIKULANDRIĆ R , LONČAR D , BÖHNING D , et al . Artificial neural network modelling approach for a biomass gasification process in fixed bed gasifiers[J]. Energy Conversion & Management, 2014, 87: 1210-1223.
71	SHARMA N , SINGH K . Model predictive control and neural network predictive control of TAME reactive distillation column[J]. Chemical Engineering & Processing Process Intensification, 2012, 59: 9-21.
72	LI S , LI Y Y . Neural network based nonlinear model predictive control for an intensified continuous reactor[J]. Chemical Engineering & Processing Process Intensification, 2015, 96: 14-27.
73	SENSENI A Z , MESHKANI F , FATTAHI S M S , et al . A theoretical and experimental study of glycerol steam reforming over Rh/MgAl₂O₄catalysts[J]. Energy Conversion & Management, 2017, 154: 127-137.
74	AVINASH A , MURUGESAN A . Prediction capabilities of mathematical models in producing a renewable fuel from waste cooking oil for sustainable energy and clean environment[J]. Fuel, 2018, 216: 322-329.
75	ESTAHBANATI M R K , FEILIZADEH M , ILIUTA M C . Photocatalytic valorization of glycerol to hydrogen: optimization of operating parameters by artificial neural network [J]. Applied Catalysis B: Environmental, 2017, 209: 483-492.
76	ABDULKADIR K , ATILA C , BAHATTIN A , et al . The pyrolysis process verification of hydrogen rich gas (HerG) production by artificial neural network (ANN)[J]. International Journal of Hydrogen Energy, 2016, 41(8): 4750-4758.
77	HOSSAIN M A , AYODELE B V , CHENG C K , et al . Artificial neural network modeling of hydrogen-rich syngas production from methane dry reforming over novel Ni/CaFe₂O₄ catalysts[J]. International Journal of Hydrogen Energy, 2016, 41(26): 11119-11130.
78	付柯, 谢良才, 闫雨瑗,等 . 改进BP神经网络预测Ni/Al₂O₃催化CH₄-CO₂重整反应[J]. 化工进展, 2017, 36(7): 2393-2399.
	FU Ke , XIE Liangcai , YAN Yuyuan , et al . Predicting model of CH₄-CO₂ reforming on Ni/Al₂O₃ catalyst by improved back propagation (BP) neural network[J]. Chemical Industry and Engineering Progress, 2017, 36(7): 2393-2399.
79	HAJJAR Z , KHODADADI A , MORTAZAVI Y , et al . Artificial intelligence modeling of DME conversion to gasoline and light olefins over modified nano ZSM-5 catalysts[J]. Fuel, 2016, 179: 79-86.
80	MOHAMMED M L , MBELECK R , Patel D , et al . Greener and efficient epoxidation of 4-vinyl-1-cyclohexene with polystyrene 2-(aminomethyl) pyridine supported Mo(VI) catalyst in batch and continuous reactors[J]. Chemical Engineering Research & Design, 2015, 94: 194-203.
81	ZHU X , LIU S , CAI Y , et al . Post-plasma catalytic removal of methanol over Mn–Ce catalysts in an atmospheric dielectric barrier discharge[J]. Applied Catalysis B: Environmental, 2016, 183: 124-132.
82	SENER A N ,GME, EBAA L , et al . Statistical review of dry reforming of methane literature using decision tree and artificial neural network analysis[J]. Catalysis Today, 2018, 299: 289-302.
83	RACCUGLIA P , ELBERT K C , ADLER P D , et al . Machine-learning-assisted materials discovery using failed experiments[J]. Nature, 2016, 533: 73-77.
84	GILMER J , SCHOENHOLZ S S , RILEY P F , et al . Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning, PMLR 70, Sydney, Australia, 2017: 6-11.
85	NOSENGO N .Can artificial intelligence create the next wonder material?[J]. Nature, 2016, 533(7601): 22-25.
86	SEGLER M H S , PREUSS M , WALLER M P . Planning chemical syntheses with deep neural networks and symbolic AI[J]. Nature, 2018, 555(7698): 604.

[1]	LI Chuankun，WANG Chunli，GAO Xinjiang. R&D and application of caprolactam plant safety operation guidance system [J]. Chemical Industry and Engineering Progree, 2014, 33(04): 1060-1066.
[2]	DU Yujing1,2，FAN Yingfang1. QSAR study of triphenylacrylonitrile derivatives using artificial neural network [J]. Chemical Industry and Engineering Progree, 2010, 29(1): 25-.
[3]	YIN Fanghua，LI Weimin，YAO Chao. Optimization for biodiesel production technology based on genetic algorithm-neural network [J]. Chemical Industry and Engineering Progree, 2008, 27(8): 1293-.
[4]	LIU Jianxin . Model and optimization of para-xylene oxidation process based on neural network [J]. Chemical Industry and Engineering Progree, 2006, 25(6): 708-.