Abstract:

Currently, most refinery crude oil scheduling studies adopt static scheduling schemes based on mathematical programming, which cannot adjust and optimize according to environmental change in real-time. This paper established a dynamic real-time scheduling decision model subject to refinery production constraints and designed the corresponding agent interaction environment. The soft actor-critic (SAC) algorithm in deep reinforcement learning solved the model. Firstly, the crude oil resource scheduling problem was transformed into a Markov decision process, and a deep reinforcement learning algorithm based on SAC was proposed to simultaneously determine discrete decisions such as transmission target and continuous decisions such as transmission speed in the scheduling process. Extensive experimental results showed that the strategy learned by the algorithm has better usability, which effectively improved the decision-making efficiency of the algorithm and effectively controlled the influence range of random events on the overall decision-making compared with the baseline algorithm. This algorithm can provide new ideas for rapid decision-making of crude oil storage and transportation scheduling in coastal refineries.

Key words: refinery crude oil storage and transportation, resource scheduling, deep reinforcement learning, soft actor-critic (SAC)

摘要：

目前对于炼厂原油储运调度决策的研究大多采用基于数学规划的静态调度方案，求解时间较长并且无法针对环境的变化进行实时高效的储运调度优化。为此，本文结合深度强化学习算法建立了考虑炼厂生产约束的原油资源储运动态实时调度决策算法。该算法首先将炼厂原油资源调度问题转换为马尔可夫决策过程，其次提出了一种基于软演员-评论家（soft actor-critic，SAC）的深度强化学习算法来同时确定调度过程中的传输目标等离散决策以及传输速度等连续决策。结果表明，算法学习到的策略可行性较好，与基线算法相比，油轮在港时间、调度方案事件数量、加工计划执行率等重要指标方面均得到了较好的效果，在求解时间方面大幅提升至毫秒级，并有效控制随机事件对整体决策的影响范围。该算法可为沿海炼厂原油储运调度快速决策提供新的思路。

关键词: 炼厂原油储运, 资源调度, 深度强化学习, 软演员-评论家