Controlling the Propagation of Mechanical Discontinuity using
Reinforcement Learning
Abstract
Mechanical discontinuity embedded in a material plays an essential role
in determining the bulk mechanical, physical, and chemical properties.
The ability to control mechanical discontinuity is relevant for
industries dependent on natural, synthetic and composite materials, e.g.
construction, aerospace, oil and gas, ceramics, metal, and geothermal
industries, to name a few. The paper is a proof-of-concept development
and deployment of a reinforcement learning framework to control the
propagation of mechanical discontinuity. The reinforcement learning
framework is coupled with an OpenAI-Gym-based environment that uses the
mechanistic equation governing the propagation of mechanical
discontinuity. Learning agent does not explicitly know about the
underlying physics of propagation of discontinuity; nonetheless, the
learning agent can infer the control strategy by continuously
interacting the simulation environment. The Markov decision process,
which includes state, action and reward, had to be carefully designed to
obtain a good control policy. The deep deterministic policy gradient
(DDPG) algorithm is implemented for learning continuous actions for the
desired reinforcement learning. It is also observed that the training
efficiency is strongly determined by the formulation of reward function.
An adaptive reward function involving reward shaping improves the
training. The reward function that forces the learning agent to stay on
the shortest linear path between crack tip and goal point performs much
better than the reward function that aims to reach closest to the goal
point in minimum number of steps. After close to 500 training episodes,
the reinforcement learning framework successfully controlled the
propagation of discontinuity in a material despite the complexity of the
propagation pathway determined by multiple goal points.