This work explores the performance difference between the approach with and without Battery Energy Storage System (BESS) for an off-grid Hybrid Energy System (HES). For the storage-less case, the solar forecast is used to mitigate the system variability. The evaluation is achieved through a highly configurable Python-based simulation platform and a costbased evaluation approach, where the simulation results are considered as the system performance indicator. In this work, two forecast methods and 6 different BESS sizes from 0 to 50MWh are tested. The results show that the use of BESS can enhance the system stability, increase the PV energy share in the final energy consumption, and reduce the final system cost, especially in a situation with uncertainty. In cases with a high PV penetration rate, a larger BESS provides greater advantages, however, the overall performance-to-cost ratio of an oversized BESS is lower compared to a properly sized BESS. For a high-value offgrid system, BESS proves to be a useful and efficient option. Alternatively, a reasonably accurate forecast is also a viable cost-effective choice.