Can end-to-end data-driven models outperform traditional semi-physical models in separating 1-min irradiance?

As a crucial component of the model chain, which facilitates irradiance-to-power conversion during solar resource assessment and forecasting, separation modeling continues to draw attention in both academia and industry. However, when evaluating even the best separation model today, one can quickly recognize its limited accuracy compared to other energy meteorology models such as transposition models. The task of separating global horizontal irradiance into diffuse and beam components does not seem soluble by any derivative effort aimed at tweaking the existing semi-physical models. As a result, an appealing alternative is to consider end-to-end data-driven models, which have demonstrated predictive capability in scenarios where the volume of data is substantial and the interaction among variables is complex. This work discusses the separation of 1-min irradiance from a data-driven perspective. In this preliminary study, a total of 10 representative data-driven separation models are developed and compared to the state-of-the-art semi-physical models, using a comprehensive 1-min irradiance database that spans five years and covers numerous climate types. The average error of the data-driven models is found to be 15.2% to 22.6% lower than that of the semi-physical models for training locations and 7.9% to 17.6% lower for completely unseen locations. Data-driven models also have significantly lower standard deviations (up to 87.2% even for completely unseen locations), highlighting their robustness. In addition, this work provides a guideline for choosing between data-driven and semi-physical models based on data availability, application needs, computational resources, interpretability, and model adaptability. Furthermore, the study underscores the challenges in accurately predicting the diffuse fraction using available input features and indicates that the incorporation of additional weather-related variables and domain knowledge could enhance the performance of data-driven separation models. © 2023 Elsevier Ltd