1 Introduction

In recent years, limited fossil fuel reserves and the harmful emissions associated with them have led to an increasing focus on renewable energy sources.

Among renewable energy sources, wind power is the cheapest source of electricity, but it is only viable when weather conditions allow. In order to ensure the economic dispatch of wind power, the ability to make short-term predictions of future wind speeds is very important, as wind speed directly affects the generation capacity. Without this capability, wind farm operators are likely to allocate more generators or replenish energy reserves to ensure that the budgeted power output is achieved, ultimately leading to increased operating costs.

The types of neural networks used in these studies generally consist of feedforward multilayer perceptron (MLP) networks, or recurrent neural networks (RNN), structures. Studies have shown that the recurrent neural network structure is the most effective for predicting time series data, and therefore, it is considered the best architecture for short-term wind speed prediction.

This paper investigates the short-term wind speed prediction capabilities of MLP, RNN, and synchronized recurrent neural networks (SRN). The training algorithm used is the particle swarm optimization algorithm (PSO). The neural networks trained with this training algorithm are compared.

Previous work has shown that temperature is the most important meteorological parameter for predicting short-term future wind speed. Humidity and current wind speed have also been identified as key indicators. Therefore, the data used to train the neural networks in this paper contains the following three inputs (in addition to the bias value) recorded by an instrument on the 82-meter-tall M2 tower located at or near the National Wind Technology Center (NWTC) site of the National Renewable Energy Laboratory (NREL) 5 miles south of Boulder, Colorado:

• Current wind speed at 80 meters in milliliters

• Current temperature at 2 meters in degrees Celsius

• Current relative humidity in percent

The training data includes the values of each minute input, with each data point representing the average of the readings taken every two seconds during that minute. This training data is used to optimize the synaptic weights of each neural network to provide the best prediction of the wind speed at 80 meters 15 minutes later.

The rest of the paper is organized as follows: the detailed principles of the three neural networks used are given in Section II; the training methods are detailed in Section III, and the training results are given in Section IV. The conclusions of this work are given in Section V.

2 Neural Networks

Neural networks are one of the five main types of computational intelligence and are known as universal approximators. The first (and most popular) network is the MLP network; the second is the RNN; and finally, the third neural network studied is the SRN. In each case, the network input is as described in the previous section, and the output is simply the wind speed for the next 15 minutes. Before being presented to the network, all input values are normalized to their maximum value in one week of training data.

a .Multilayer Perceptron Network

The MLP network is a member of the feedforward network architecture and is the simplest of the networks studied. In this network, there are 3 layers, each consisting of neurons. These three layers are the input, hidden, and output layers. The input layer (with a linear activation function) receives the input value, which is then multiplied by the input weight matrix W, passed through the hidden layer (using a sigmoid activation function), multiplied by the output weight matrix V, and finally input to the output layer (using a linear activation function).

Formulas (1) to (4) are used to calculate the output y for a given input x.

B. Recurrent Neural Networks

RNNs are a family of recurrent architectures, and as mentioned earlier, this type of architecture excels at time series forecasting. RNNs are similar in general structure to MLPs, except that they contain a feedback loop (with unit delay) from some later stage of the network back to the input layer. In this study, an Elman network was used, which takes the output from the hidden layer; another type of RNN is the Jordan network, which takes feedback from the output layer. This feedback is stored in another layer called the context layer, allowing the network to retain internal memory.

Since the hidden layer is wider than the output layer, this study chose the Elman architecture over the Jordan network. This wider layer allows more values to be fed back to the input, allowing the network to obtain more information.

To calculate the output of the RNN, the same equation from the MLP is used, replacing (1) with equation (5).

C. Synchronous Recurrent Neural Networks

Like RNNs, SRNs are also members of a feedback architecture. However, SRNs differ from RNNs in that the feedback portion does not contain a delay unit, but instead takes the feedback value directly from a later stage of the network. Like RNNs, SRNs can adopt either an Elman or Jordan configuration.

One major difference between SRNs and RNNs is that since the feedback does not contain a delay, the input values must propagate through the network several times until the output reaches equilibrium. This process of repeatedly applying the input is called internal recurrence, and due to these internal recurrences, the same equations used to obtain the output of an RNN can be used for SRNs, except that they must be repeated several times until the output y in (4) stabilizes. To ensure the stability of the output, the internal recurrence is performed until two consecutive outputs do not differ by more than 0.01 or 20 iterations.

In this study, each network has 4 input neurons (one for each input and also a bias value) and 1 neuron in the output layer because the network will produce a single output value. However, to keep the number of synaptic weights the same (and therefore the amount of information contained in each network), the MLP has 10 neurons in its hidden layer, while the RNN and SRN each have 5 neurons in their hidden layer, which also means that the context layer is 5 neurons wide.

In the case of the MLP, W is of size 4x I0 and V is of size 1x I0, while both the RNN and SRN have a W of size 9 × 5 and a V of size 5 × 1, with the total number of weights being 50 in all cases. The choice of hidden layer size is rather arbitrary, as the optimal number cannot be known without trial and error or analysis of the degrees of freedom of the training data.

3 Training steps

In each of the neural network cases mentioned above, the input and output weights need to be trained to allow the network to provide sufficient output. This process is optimized using a particle swarm algorithm and implemented in Matlab. Previous studies have shown that the PSG algorithm exhibits superior performance compared to the backpropagation algorithm when used to train feedforward neural networks such as MLP.

PSG is a computational optimization technique that utilizes a swarm of particles flying through a multidimensional problem space, where each particle represents a potential solution. The number of dimensions in the problem space is equal to the number of parameters that need to be optimized. Because each of the networks studied has the same number of synaptic weights, the dimension of each particle is 50 in each case.

The standard canonical form used in this study is given by equation (6) for the velocity update and equation (7) for the position update. In each equation, e is the number of particles and d is the particle dimension.

Before the actual training of the neural network, the parameters of the PSG algorithm (w, c, C2) as well as the number of particles used for PSG optimization were optimized. We used 30 particles and found an initial value of w of 0.7492 (which linearly decreased to half of the initial value at the last iteration), while the optimal values of CI and C2 were 0.8107 and 2.5000, respectively. After all PSG parameters were optimized, each of the previously mentioned networks was trained with PSG for 300 iterations, using all the best settings and using 10,080 training data points from the week of November 10, 2008 to November 16, 2008, each corresponding to a one-minute average of readings every two seconds. This training was repeated 20 times in order to average out the performance variations due to the stochastic aspects of the PSG algorithm.

4 Result

We present the worst, average, and best MSE for each network trained using PSG, as well as the time required to run the entire optimization process for each network. The best performance of each network is also compared, as well as the absolute relative error (ARE). A sampling of each over a small selected time frame is shown. The performance of the MLP, RNN, and SRN networks for a sample of the training data is shown separately. From these figures, it can be seen that while the performance of each network is similar, the recurrent architectures (RNN and SRN networks) perform slightly better than the MLP.

Notably, the recurrent architecture outperforms the MLP, especially between minutes 7600 and 7700. The magnitude of the ARE for the recurrent architecture is lower than that of the MLP during this time frame.