© T. Schmidt et al., published by EDP Sciences, 2017

1 Introduction

High resolution very short-term (seconds to minutes ahead) local solar forecasting, based on sky images, is a key technology where variability in solar irradiance is a critical issue, for example when operating electricity grids with high penetration of photovoltaic (PV) systems. Specifically in remote (off-grid) networks, the integration of a higher share of PV generation is mainly limited by the PV power fluctuations caused by passing clouds [1]. Hence, accurate solar forecasting methods are of interest to remote electricity network operators. Well-known forecasting methods, based on numerical weather prediction models or satellite images, however, lack spatial and temporal resolution to resolve and to detect small scale cloud structures needed for accurate high resolution forecasts [2]. On the other hand, time series based statistical methods are not able to predict ramp events. Sky image based very short-term forecasting provides both high temporal (∼seconds) and spatial resolution (∼metres) along with high update rates (∼few seconds) [3]. The ability to detect and track clouds in the field of view of the camera lens enables users to predict cloud events shortly before they take place. Clear sky, overcast or scattered cloud conditions can be identified and then used to provide information about the near-future power output of PV systems (e.g. constant or highly variable). Grid operators will then be able to take measures to enhance the grid stability in the foreseen situation by starting more dispatchable generation, managing or curtailing PV generation, shifting loads or charging/discharging storage elements [4].

In this study, we focus on diesel powered remote network applications and associated generator scheduling. We want to investigate the possibility of reducing spinning reserve requirements under constant clear sky conditions when there is a high share of PV generation in the network. Therefore, we evaluate the reliability of predicting the persistence of clear sky conditions in the next 2–5 min, a period which allows diesel generators to start and to connect.

We developed a solar irradiance forecasting scheme, based on all-sky images taken every 10 s at the University of Oldenburg in Germany. Key components of the scheme are cloud detection, cloud tracking, geometric cloud shadow projection and irradiance modelling. The key principle is the determination of cloud position and motion to estimate future cloud conditions and as a result surface solar irradiance levels. In this paper, a brief overview of the forecasting scheme is provided.

The forecasting scheme is applied to a data set of images and global horizontal irradiance (GHI) measurements acquired from 10th November to 28th December 2016 at Murdoch University, Perth, Western Australia. Sky images used to process forecasts have been recorded every 10 s from sunrise to sunset. An SP-Lite2 pyranometer, mounted close to the camera, provided time synchronized GHI data, which is used for forecasting calibration and validation.

2 Methodology

The present forecasting scheme makes use of hemispheric images (1920 × 1920 pixels) taken by a low-cost Vivotek FE8174V fish eye lens network (security) camera. A geometrical lens calibration for the necessary projection of image coordinates in the three-dimensional space and vice versa has been performed. The calibration process used several snapshots of chessboards with the OcamCalib Toolbox [5] to fit a radially symmetric projection function that maps the pixel coordinates to the three-dimensional coordinate of its origin ray. To validate the calibration, the backward model is used to re-project the sun position to image coordinates (Fig. 1).

Deriving cloud information and surface solar irradiance estimations from sky images is a step-wise procedure. The model is described by [6]. First, a cloud decision scheme using the Red-Blue-Ratio, with a clear sky reference library, is applied to classify a pixel as cloudy or not (Fig. 2b). This binary distinction is a first approximation to separate clouds from clear sky but has weaknesses in the case of optically semi-transparent clouds (e.g. cirrus clouds). Under these conditions, forecast results based on the binary cloud decision will likely be erroneous. In a next step, usually cloud base height information and the lens projection function are applied to project the 2D image coordinates of clouds to their geometrical position in a three-dimensional space. For this study, only forecasts for the location of the camera have been processed and compared to local irradiance measurements. In this case, geometrical cloud and shadow projection using cloud base height is not needed. Cloud and irradiance forecasts are the result of tracking clouds occluding the sun (Fig. 2c). The cloud tracking is based on a sparse optical flow (Lukas-Kanade algorithm [7]). Image pixels are tracked for subsequent images and a global cloud motion is computed as the average of the projected motion vectors. This motion vector is then used to extrapolate the current cloud scene. This approach performs well as long as cloud motion in the image is homogeneous. Problematic conditions are produced by clouds in multiple layers, especially when moving in different directions and from clouds moving slowly, while morphing strongly, as a result of convection. How often these conditions occur depends on the local climate. The latter condition is typical for climates where convection due to solar ground heating dominates the advection (horizontal movement) in determining overall wind patterns. Cloud tracking and forecasting is complicated in this case, especially if clouds form or dissolve on timescales shorter than the forecast horizon of the sky camera system. The final irradiance modelling is based on the binary information from the projected cloud maps (Fig. 2d). A mapping of both states (cloud or no cloud) to the solar irradiance values is performed, based on the clear sky index (normalized measured irradiance with modelled clear sky irradiance) histograms of the past 30 min. Histogram peaks for the cloudy and the non-cloudy state are assigned to generate the irradiance map. Cloud edge transitions are smoothed by a Gaussian filter to obtain more realistic irradiance ramps.

Focusing on PV integration in remote electricity systems with diesel generators, we evaluate the forecast performance regarding the ability of the forecast model to predict solar irradiance in the forthcoming 2–5 min, allowing time to start and synchronise a typical diesel generator. The accurate prediction of changes in solar irradiance in the aforementioned critical time window is of importance rather than the accurate prediction of irradiance at a specific point in time and space. The main question to be answered is: how reliable is a forecast that there will be no cloud event in the time window of interest?

A simplified control strategy of a diesel generator with solar irradiance forecasts has been defined to demonstrate a potential application for the reduction of diesel consumption. We assume a diesel generator to remain off during constant clear sky conditions over the last 10 min with a minimum (10th percentile) insolation of 600 W/m² (about 55% of noon clear sky irradiance). The generator has to be switched on when the conditions are about to change in the next 2–5 min (Fig. 3). Otherwise the generator stays off. We call this a persistent clear sky. A non-persistent clear sky means that clouds will shade the pyranometer (cloud event) and therefore end the clear sky period defined above. The sky conditions change to either mixed (fluctuating insolation) or overcast (down-ramp to homogeneous low insolation) in the next 2–5 min. The opposite case of starting with overcast or mixed sky is not examined in this study, since it is assumed that the generator will be anyway switched on and sharing the load. Therefore, we only investigate the reliability of the forecast to correctly predict a persistent clear sky or how well an erroneous forecast of a non-persistent clear sky can be avoided. The latter case could lead to grid stability issues, if insufficient spinning reserve is available.

The total number of constant clear sky conditions is identified by applying the 600 W/m² threshold to sliding windows of 10 min of the irradiance time series. Next, we do the same for solar irradiance forecasts and measurements for the horizon from 2 to 5 min. Figure 4 illustrates the method. Note, that forecasts are updated every 10 s resulting in 30 forecasts predicting the same event with different horizons. It must be emphasized, that at least one forecast should predict the irradiance drop 2 min in advance to guarantee the generator to have enough time to start. A confusion matrix lists the correct (True Positive, TP and True Negative, TN) and false forecasts (False Positive, FP and False Negative, FN) of the binary classifier. Please note that in this study, the positive condition is an observed “persistent clear sky” and the negative condition is an observed “non-persistent clear sky” (cloud event).

• True Positive (TP): persistent clear sky was observed and predicted correctly;

• False Positive (FP): persistent clear sky was observed, but a cloud event was predicted;

• True Negative (TN): non-persistent clear sky was observed/cloud event happened and predicted correctly;

• False Negative (FN): non-persistent clear sky was observed/cloud event happened but persistent clear sky was predicted.

We call the share of correctly predicted conditions accuracy (Eq. (1)) and the share of incorrectly predicted non-persistent clear sky conditions the miss rate (Eq. (2)). The share of incorrect forecasts of persistent clear sky conditions is called false alarm rate (Eq. (3)). (1) (2) (3)

Both miss rate and false alarm rate should be minimized in order to achieve high potential for reduction of spinning reserves (false alarm rate) and to reduce the probability of grid stability issues due to non-predicted reductions in PV generation (miss rate).

Fig. 1 Lens calibration results showing the projection of viewing angles to hemispheric zenith angles and the back projection of the sun position into the image. The grey line indicates the path of the sun on the given day.

Fig. 2 Basic steps in sky image processing: (a) raw masked image, (b) binary cloud detection, (c) cloud tracking with final cloud motion path and (d) projected irradiance map.

Fig. 3 Diesel generator scheduling used in the study.

Fig. 4 Illustration of non-persistent clear sky conditions. If the irradiance (black dashed line) drops below the threshold at any time in the forthcoming 2–5 min, actual conditions are flagged as non-persistent. While instances at the green diamonds represent persistent conditions, red diamonds indicate non-persistence. The past 10 min are considered in the definition of a constant clear sky period.

3 Results

About half of the days from 10th November to 28th December 2016 were almost completely cloud-free leading to about 61% constant clear sky periods. The other days were dominated by mixed clouds (scattered or broken) moving from different directions with varying speeds. Several situations where constant clear sky conditions are replaced by fluctuating conditions could be identified from irradiance time series and images. A total number of 84 cloud events indicating the ending of a constant clear sky period could be identified.

Figure 5 shows an example period with five cloud events. The colour shaded area indicates the constant clear sky periods considered for potential generator scheduling and the forecast evaluation. The yellow colour represents persistent conditions and blue indicates non-persistence due to upcoming clouds. The chosen generator scheduling is also given in this figure. In this simplified case, the generator is switched off, when clear sky lasted for at least 5 min and remains off when no cloud event is predicted (TP and FN). It can be noted that all events in the shown period could be predicted at least by the latest forecast 2 min in advance (light blue area or TN). The first event was not yet predicted 5 min beforehand (dark blue or FN), but was predicted in the subsequent forecasts.

To summarize the results for the whole period, only the constant clear sky periods are evaluated as mentioned before. For these periods, Table 1 shows the confusion matrix of the binary forecast results. Overall, for 98.2% of the time clear sky conditions persist in the next 2–5 min (TP + FP). For only about 1.8% of the time, conditions change from clear sky to cloudy (TN + FN) in the next 2–5 min. For 2.8% of the time a change from clear sky to cloudy was predicted but clear sky conditions persisted (false alarm rate). Nevertheless, focusing on the rare, but more important non-persistent clear sky conditions due to cloud events, we assert a miss rate of 28.2% (FN/(FP + TN)). The total number of 1892 forecast instances, where the next 2–5 min observations are classified as non-persistent clear sky, covered a period with 84 actual cloud events, as mentioned above. While 76 events were predicted, at least by one forecast in advance, 8 events (∼10%) were still not foreseen. This is a slightly better result, but not yet satisfactory.

Investigating the rare number of falsely predicted cloud events, the lessons to be learnt for improving the forecast model can be exposed. A number of problematic situations for non-predicted cloud events were identified:

• clouds in the circumsolar area of the image are difficult to detect: Forecasts in the 2–5 min horizon are then less reliable, particularly when cloud speed is low;

• low sun elevation and cloud direction along the sun azimuth: When clouds are moving from the horizon towards the sun position forecasts are affected by:

• •

  cloud motion is determined from clouds in the centre of the image and not at the border,

• •

  large viewing angles lead to perspective errors affecting accuracy,

• •

  circumsolar area brightness is often enhanced for low sun elevations.

• clouds move slightly along a different path as predicted by the analysed global cloud motion or clouds form or change size less than 2 min before blocking the sun;

• some very short cloud events were related to single and/or small clouds not detected or not predicted based on a slightly wrong global motion vector.

Based on these findings, improvements for the model can be derived:

• it is important to note, that local single site forecasts with high temporal resolution have been investigated. Spatial averaging, with distributed PV generation, will reduce the influence of cloud events from small, single clouds and by PV power averaging under scattered cloud conditions;

• cloud motion vectors should be derived for single clouds to avoid false cloud tracking due to averaging. Moreover, the uncertainty of the motion vector algorithm should be used to provide confidence intervals rather than one deterministic motion vector;

• at least for local site forecasting (forecasts for the location of camera), camera hardware with a higher dynamic range should be used to reduce misclassification in the circumsolar area;

• problematic conditions (e.g. low solar elevations but also rain drops, birds, dirt) affecting image quality and forecast accuracy should be detected and communicated operationally beforehand so that forecasts can be labelled as unreliable and appropriate measures can be taken;

• general improvements in cloud detection and tracking algorithms will enhance forecast performance.

Fig. 5 Example results for the prediction of cloud events on a day with mixed conditions. The yellow area indicates the period defined as persistent clear sky. Blue colours indicate non-persistent conditions with cloud events in the next 2–5 min. When the periods end, the forecast system should reliably predict the change. The resulting generator state is shown as the result of past irradiance conditions and forecasts (bottom).

4 Conclusions

We evaluated sky imager based very short term forecasts for a single local site in Perth, Western Australia for the horizon of 2–5 min. The ability and reliability to predict clear sky persistence of the forecast model was investigated in view of a diesel generator scheduling application in a remote network. The examined period (November–December 2016) of time can be characterized to at least 60% by clear sky conditions. Half of the days were almost completely clear. The other days were mostly dominated by mixed clouds (scattered or broken) resulting in large short-term irradiance fluctuations.

For persistent clear sky conditions, the sky imager based forecasts achieve 97% with only 3% of false alarms (cloud event falsely predicted). Reliable forecasts for persistent clear sky can be used to switch/keep generators off to save fuel and reduce costs.

Of greater importance are non-persistent conditions, meaning that irradiance conditions change from clear sky to cloudy conditions. In this situation, we assumed the generators have to be switched on at least 2 min before the event. The results show that of the 84 cloud events after a clear sky period only 90% were correctly predicted at least with one forecast 2 min in advance. Nevertheless, the analysis of the falsely forecasted events leads to suggestions to improve the forecasting method. For example, refined cloud motion tracking and irradiance modelling will be implemented in future developments of the forecasting method for a more robust and reliable short-term forecast. This study did not focus on the sensitivity of the chosen thresholds and the generator scheduling. For example, more conservative thresholds and decisions on the minimum runtime of the diesel generators, after being started, will have some effect on the fuel savings potential and grid stability issues.

References

All Tables

All Figures

	Fig. 1 Lens calibration results showing the projection of viewing angles to hemispheric zenith angles and the back projection of the sun position into the image. The grey line indicates the path of the sun on the given day.
In the text

	Fig. 2 Basic steps in sky image processing: (a) raw masked image, (b) binary cloud detection, (c) cloud tracking with final cloud motion path and (d) projected irradiance map.
In the text

	Fig. 3 Diesel generator scheduling used in the study.
In the text

Fig. 4 Illustration of non-persistent clear sky conditions. If the irradiance (black dashed line) drops below the threshold at any time in the forthcoming 2–5 min, actual conditions are flagged as non-persistent. While instances at the green diamonds represent persistent conditions, red diamonds indicate non-persistence. The past 10 min are considered in the definition of a constant clear sky period.

In the text

Fig. 5 Example results for the prediction of cloud events on a day with mixed conditions. The yellow area indicates the period defined as persistent clear sky. Blue colours indicate non-persistent conditions with cloud events in the next 2–5 min. When the periods end, the forecast system should reliably predict the change. The resulting generator state is shown as the result of past irradiance conditions and forecasts (bottom).

In the text