Gas furnaces are the most widely used means of heating in industry, and with the growing concern for environmental issues, and a global energy crisis at our doorstep, the optimization of the processes related to them becomes a key challenge. This paper aims at introducing a new way of practicing gas furnace control involving simulations, virtual sensors and deep reinforcement learning (DRL) techniques. In order to do so we designed a set of simulations of conjugate heat transfer systems governed by the coupled Navier-Stokes and heat equations for single-step control. The DRL algorithm used in this paper is the policy-based optimization (PBO) algorithm specialized in singlestep (or open-loop) control. We explore its ability to find global maxima in di erent situations and under various constraints. Therefore, various 2D and 3D cases are tackled, in which the position of the work piece, the flow rate, and other parameters are controlled. The obtained results highlight the potential of the DRL framework combined with computational fluid dynamics (CFD) conjugate heat transfer systems for optimizing searches in large parameter spaces. For the 2D case, PPO achieved an increase of 89% in temperature homogeneity, and for the 3D case an increase of 7% in final temperature with the same total input.