Research Article
Solar and wind forecasting by NARX neural networks
National Research Council of Italy, Institute of Intelligent Systems for Automation, Unit of Palermo,
Palermo, Italy
^{⁎} email: gianpaolo.vitale@ieee.org
The nonlinear autoregressive network with exogenous input (NARX) is used to perform hourly solar irradiation and wind speed forecasting, according to a multistep ahead approach. Temperature has been considered as the exogenous variable. The NARX topology selection is supported by a combined use of two techniques: (1) a genetic algorithm (GA)based optimization technique and (2) a method that determines the optimal network architecture by pruning (optimal brain surgeon (OBS) strategy). The considered variables are observed at hourly scale in a seven year dataset and the forecasting is done for several time horizons in the range from 8 to 24 h ahead.
© A. Di Piazza et al., published by EDP Sciences, 2016
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1 Introduction
An accurate prediction of solar energy production is crucial for the effective integration of photovoltaic (PV) and wind generators in smart grids [1,2]. For this reason, modeling solar irradiation by means of time series forecasting techniques is becoming widespread.
In general, artificial neural networks (ANNs) have proven to be more effective for this purpose than other classical autoregressive predictors, such as ARX, ARMAX, and BoxJenkins (BJ) model [3]. On the other hand, some issues are still under discussion. For example, once a given ANN is chosen, the definition of general and reliable criteria for selecting the most appropriate structure of the ANNbased model is needed. In particular, the definition of methods for defining both the optimal weight set and the best network topology is useful to avoid a time consuming trial and error procedure for the network setup.
In this paper, an ANNbased model is used to perform the hourly solar irradiation and wind speed forecasting, according to a multistep ahead approach. Particularly, the nonlinear autoregressive network with exogenous input (NARX) is chosen, where the exogenous variable is the temperature. The choice of temperature is due both to its availability in the database and to the suitability as exogenous input in solar irradiation forecasting. With the available temperature dataset the wind speed forecasting has been performed as well with good results. The reason for the choice of NARX network is due to the good ability of this neural network to handle problems involving the modeling of nonlinear dynamic systems, such as dependencies among meteorological time series [4,5].
To overcome the disadvantage of the trial and error based procedure [6,7], in this work, the NARX structure selection is supported by a combined use of two techniques: (1) a genetic algorithm (GA)based optimization technique that allows the best network weight set to be determined and (2) a pruning method based on the optimal brain surgeon (OBS) strategy that determines the optimal network architecture. In such a way an optimized NARX is obtained.
The considered variable datasets are referred to a seven yearobservation period and their forecasting is done for several time horizons in the range from 8 to 24 h ahead.
2 Geographical context and performance indices
Temperature, global solar irradiation and wind speed data, used in this study, come from Palermo, Sicily (Italy), gauge station (latitude 38° 8′ N, longitude 13° 20′ W, elevation 55 m). The used dataset consists of the hourly global solar irradiation (MJ/m^{2}), hourly wind speed (m/s) measured at two meters above ground level and the hourly maximum and minimum temperature recorded during seven years, from 2002 to 2008. In this paper the mean hourly temperature has been used. All data have been provided by SIAS (Servizio Informativo Agrometeorologico Siciliano). The performance indices used to assess the NARX model are the normalized root mean square error (NRMSE) and the coefficient of variation of the root mean squared error, CV(RMSE). They are defined, respectively as: (1) (2) where Y is the observed time series, is the predicted time series, is the mean of the observed values, Y_{max} is the maximum and Y_{min} is the minimum observed values. In this paper, the performance indices will contain the subscripts e, v, t, r and f that indicate, respectively, estimation set and validation set for time series linear model, training phase, recall phase (application of models to validation set) and forecast phase for neural approach.
3 Forecasting technique
The NARX neural network is derived by a class of discretetime nonlinear systems, i.e., the nonlinear autoregressive with exogenous input (NARX) models. It has been chosen for the proposed analysis since it is well suited to model nonlinear dynamic systems. The NARX model mathematical formulation is the following: (3) where y(t) and u(t) are the past and present independent (exogenous) inputs of the model at a discrete time step t, n_{y} ≥ 1, n_{u} ≥ 1, n_{y} ≥ n_{u} are the input memory and output memory orders (delay) and f is a nonlinear mapping function.
When the function f is approximated by a multilayer perceptron (MLP), the resulting neural network is called NARX network. In other words, a NARX network consists of a MLP that takes as input a window of past independent (exogenous) inputs and past outputs and calculates the current output. Unlike a conventional recurrent neural network, the NARX network has a limited feedback coming only by the output neuron rather than by the hidden states. Actually, only the output of the NARX is fed back to the input of the feedforward neural network. Nevertheless, it has been demonstrated that it is as much computationally powerful as a fully connected recurrent neural network [8]. In order to avoid the trial and error approach for the determination of parameters, the training of NARX has been obtained by GA algorithm in its open form. The multistepahead forecast has been obtained using the NARX in closedloop form (Fig. 1). In particular the procedure followed to find the best set of the NARX parameters, can be summarized as follows: (1) a starting network topology is chosen, (2) a GAbased optimization technique is used for the determination of the best weight set of the network (training phase) and (3) a pruning method based on the OBS strategy is applied to extract the optimal number of the network parameters, reducing the number of connections.
The number of the network parameters represents how many connections or weights are contained in a neural network. For the NARX network under study, this number (N_{p}) is given by (4): (4) where n_{u} and n_{y} are the input memory and output memory orders (delays), N is the number of neurons in the hidden layer and the term “1” is added to account for the bias of the output neuron. Once the optimal number of parameters is found, the optimal structure of the considered neural network is defined.
The number of parameters of the chosen NARX network, according to (4) is 161 for the temperature/solar irradiation and 321 for temperature/wind speed dynamic systems.
As wellestablished, GAs are heuristic, stochastic, combinatorial, optimization techniques based on the biological process of natural evolution [9,10]. The three heuristic processes of selection, crossover and mutation are applied probabilistically to discrete variables that are coded into binary or strings of real numbers. The algorithm starts by creating a random initial population, then, in each generation, creates a sequence of new population using individuals of the current generation to create the next population. To perform this procedure, the algorithm evaluates the fitness value of every individuals and selects member called parents based on their fitness. In this application the individuals of the population are the weights and the biases of the neural network; the error in training of the NARX (i.e. the MSE – mean square error) is used to provide a fitness value. Comparing the performance indices for training and test set (recall phase), it has been observed that the network overfits the data. This means that the selected model structure contains too many weights [11]. In order to find the best structure of NARX, the OBS strategy for pruning the neural network model is then used [12]. In particular, a function that performs the training of the network after each weight elimination is employed. On such a basis, the best number of the network parameters corresponds to the minimum value of test error. Figure 2 shows the trends of the test error, the training error and the Akaike index (final prediction index, FPE) [13] versus the number of the network parameters, for the solar irradiation variable computation. In the considered case, the optimal number of the network parameters is 61 for solar irradiation and 21 for wind speed. Finally in Figure 3, the obtained pruned structures of the feed forward neural network, with the optimum number of connections are shown. In Figure 3 (right side), it can be noted that the starting structure of neural network in the case of wind speed is more complex than the starting structure of neural network for solar irradiation (left side). This due to the fact that, starting from a simpler structure of neural network for wind speed variable, the algorithm of pruning could not converge. In Figure 3, the solid line connections refer to excitation inputs whereas dashed connections refer to inhibitory inputs. Starting from this structure, the optimized NARX is obtained and used to perform the hourly solar irradiation and wind speed forecasting.
Fig. 1 Chosen structures of the NARX network: closedloop. 
Fig. 3 Pruned structure of the feed forward neural network with the optimum number of connections: (a) solar irradiation and (b) wind speed. 
Fig. 2 Best number of parameters by Akaike index. 
4 Results and discussion
As previously said, the hourly solar irradiation forecasting has been performed by using a NARX network whose parameters have been defined by a GA algorithm procedure, starting from a structure with a quite large number of parameters. The performance indices of the optimized NARX, obtained according to the method described in Section 3, have been computed. In particular, the NRMSE_{r} and CV(RMSE)_{r} in recall phase are, for solar irradiation prediction, 6.1% and 32%, respectively and 7% and 47% for wind speed prediction. This result demonstrates that the method proposed in this paper for fixing the NARX structure overcomes the disadvantage of repeated tests, typical of the trial and error procedure, where much time is needed to obtain the optimal network configuration. Moreover, with the proposed method, the performance of network is improved as well [14]. It is worth noting that, in any case, the network performance is measured by the statistical indices in (1) and (2). The solar irradiation recalled by the optimized NARX is given in Figure 4 and the corresponding residuals autocorrelation is sketched in Figure 5. From Figure 5 it is possible to observe that the points corresponding to the autocorrelation of residuals are included in the confidence interval, except for three. For sake of brevity, only graphical results of solar irradiation prediction are shown in this manuscript. From the histogram of the residuals obtained by the NARX network optimized by GAOBS for prediction of the solar irradiation, the mean value is equal to −0.0096 and the variance is equal to 0.20. The ttest performed on these residuals confirms that data are a random samples from a normal distribution with mean equal to −0.0096 and unknown variance. This confirms the good result of the proposed method since residuals are normally distributed around zero with a low variance. The results of the multistepahead forecasting are shown in Figures 6 and 7.
Very similar results for the analysis of residual obtained for the prediction of wind speed are obtained. Starting from the structure sketched in Figure 3 the optimized NARXs has been utilized to forecast hourly solar irradiation and hourly wind speed for five different time horizons ranging from 8 to 24 h. The performance indices are shown in Table 1. The best forecasting results are obtained for 8 and 10 h for solar irradiation and for 18 and 24 h for wind speed.
Fig. 7 10hahead solar irradiation forecast with the NARX network optimized by GAOBS. 
Fig. 6 8hahead solar irradiation forecast with the NARX network optimized by GAOBS. 
Fig. 5 Autocorrelation of residuals from NARX network optimized by GAOBS with 99% confidence interval. 
Fig. 4 Recalled hourly solar irradiation by the NARX network optimized by GAOBS. 
Performance indices for the NARX in forecasting (bold values represent the best results).
5 Conclusions
The nonlinear autoregressive network with exogenous input (NARX) is used to perform a hourly solar irradiation and wind speed forecasting, according to a multistepahead approach. Temperature has been considered as the exogenous variable in the analysis. The NARX optimized by a GA and a OBS strategy overcomes the drawback to set up the network structure by repeated trials. The proposed method allows to improve the forecasting of PV and wind power generation and can be effectively implemented within a smart grid management system.
Acknowledgments
This paper has been funded by the following research projects: (1) RITmare, Ricerca ITaliana per il mare (Italian Research for the sea), CUP: B91J11000740001; (2) TESEO, CUP: B61C12000850005; and (3) CNR per il Mezzogiorno – “Advanced Technologies for Energy Efficiency and Sustainable Mobility”, CUP: B51J10001290001.
References
 V. Lenzi, A. Ulbig, G. Andersson, Impact of forecast accuracy on grid integration of renewable energy sources , in Proc. IEEE Power Tech. Conf., Grenoble ( 2013) (In the text)
 H. Quan, D. Srinivasan, A. Khosravi, Shortterm load and wind power forecasting using neural networkbased prediction intervals, IEEE Trans. Neural Netw. Learn. Syst. 25, 303 ( 2014) [CrossRef] (In the text)
 A. Mellit, S.A. Kalogirou, L. Hontoria, S. Shaari, Artificial intelligence techniques for sizing photovoltaic systems: a review, Renew. Sustain. Energy Rev. 13, 406 ( 2009) [CrossRef] (In the text)
 R.H. Inman, H.T.C. Pedro, C.F. Coimbra, Solar forecasting methods for renewable energy integration, Progr. Energy Combust. Sci. 39, 535 ( 2013) [CrossRef] (In the text)
 S. Gupta, D. Srinivasan, T. Reindl, Forecasting solar and wind data using dynamic neural network architectures for a microgrid ensemble, in Proc. IEEE Symposium on Computational Intelligence Applications in Smart Grid, Singapore ( 2013), Vol. 1, p. 87 (In the text)
 A. Di Piazza, M.C. Di Piazza, G. Vitale, Solar radiation estimate and forecasting by neural networks for smart grid energy management, in Proc. European PV Solar Energy Conference and Exhibition, Paris ( 2013) (In the text)
 A. Di Piazza, M.C. Di Piazza, G. Vitale, Estimation and forecast of wind power generation by FTDNN and NARXnet based models for energy management purpose in smart grids, Renew. Energy Power Qual. J. 12, 560 ( 2014) (In the text)
 T. Hava, B.G. Seigelmann, C. Horne, L. Giles, Computational capabilities of recurrent NARX neural Networks, IEEE Trans. Syst. Man Cybernet. 27, 208 ( 1997) (In the text)
 D. Montana, L. Davis, Training feedforward neural networks using genetic algorithms, in Proc. 11th Int. Joint Conf. Artificial Intelligence, San Mateo ( 1989), p. 762 (In the text)
 J. Holland, Adaptation in natural and artificial systems ( University of Michigan Press, Ann Arbor, MI, 1975) (In the text)
 P.M. Nørgård, The neural network based system identification toolbox: for use with MATLAB, 1997 (In the text)
 H. Hassibi, D.G. Stork, Second order derivatives for network pruning: optimal brain surgeon, in: NIPS 5, edited by S.J. Hanson et al. ( Morgan Kaufmann, San Mateo, 1993), p. 164 (In the text)
 L.K. Hansen, J. Larsen, Linear unlearning for crossvalidation, Adv. Comput. Math. 5, 269 ( 1996) [CrossRef] (In the text)
 A. Di Piazza, M.C. Di Piazza, G. Vitale, Solar radiation forecasting based on artificial neural networks optimized by genetic algorithm for energy management in smart grids, in Proc. European PV Solar Energy Conference and Exhibition, Amsterdam ( 2014) (In the text)
Cite this article as: Annalisa Di Piazza, Maria Carmela Di Piazza, Gianpaolo Vitale, Solar and wind forecasting by NARX neural networks, Renew. Energy Environ. Sustain. 1, 39 (2016)
All Tables
Performance indices for the NARX in forecasting (bold values represent the best results).
All Figures
Fig. 1 Chosen structures of the NARX network: closedloop. 

In the text 
Fig. 3 Pruned structure of the feed forward neural network with the optimum number of connections: (a) solar irradiation and (b) wind speed. 

In the text 
Fig. 2 Best number of parameters by Akaike index. 

In the text 
Fig. 7 10hahead solar irradiation forecast with the NARX network optimized by GAOBS. 

In the text 
Fig. 6 8hahead solar irradiation forecast with the NARX network optimized by GAOBS. 

In the text 
Fig. 5 Autocorrelation of residuals from NARX network optimized by GAOBS with 99% confidence interval. 

In the text 
Fig. 4 Recalled hourly solar irradiation by the NARX network optimized by GAOBS. 

In the text 