Short-term weather forecasting using hybrid data
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating weather forecasting modules from historical weather data, including unstructured and structured data. The weather forecasting modules each comprise at least one model, such as a neural network, that generates a forecast from input weather data. Each module may be focused on a particular region, weather variable, and or forecasting time period. Unstructured data may be analyzed by the system to generate weather-related variables which are combined with structured weather data to generate a combined weather feature set that is input to the model. Alternatively, unstructured data may be input directly in the module, with or without preprocessing. The weather forecasts can then be used to forecast various power-generation related variables or events, including power generation, electrical load, etc.
1. A method for forecasting a power-related variable, the method comprising:
receiving historical unstructured weather data, the unstructured weather data comprising one or more images including weather-related features;
processing the historical unstructured weather data to generate values for at least one weather variable, the processing comprising extracting hue information from the one or more images;
receiving historical structured weather data;
combining the historical structured weather data with the generated weather variable values to create a combined weather feature set;
selecting one or more first algorithms based on one of more characteristics of the combined weather feature set;
training the one or more first algorithms to forecast a weather indicator at least in part using the combined weather feature set, the weather feature set including at least one weather variable based on the extracted hue information;
receiving historical data for the power-related variable;
training a second algorithm to forecast the power-related variable at least in part using the received historical data for the power-related variable and one or more of the historical unstructured weather data and the historical structured weather data;
receiving current structured weather data and current unstructured weather data; and
generating a forecast for the power-related variable using the current structured weather data, the current unstructured weather data, the one or more trained first algorithms, and the trained second algorithm.
2. The method of claim 1 , wherein the one or more first algorithms comprises one or more of regression models, Markov chains, time series models, state space models, Bayesian models, boosted decision trees, neural networks, convolutional neural networks, and recurrent neural networks.
3. The method of claim 1 , wherein the processing of the historical unstructured data comprises segmenting each of the one or more images into a plurality of segments and extracting hue information from each of the plurality of segments.
4. The method of claim 3 , wherein the hue information represents an image overlay.
5. The method of claim 1 , wherein the power-related variable comprises power output.
6. The method of claim 1 , wherein the power-related variable comprises electrical load.
7. The method of claim 1 , wherein the power-related variable relates to the possibility of a power outage.
8. The method of claim 5 , wherein the power-related variable comprises power output of a wind farm, and the historical data includes turbine data.
9. A method for forecasting a power-related variable, the method comprising:
receiving current structured weather data and current unstructured weather data, the unstructured weather data comprising one or more of: images, video, audio, and narrative text; and
generating a forecast for the power-related variable using the current structured weather data, the current unstructured weather data, one or more trained first algorithms, and a trained second algorithm;
wherein the one or more trained first algorithms are created by:
receiving historical unstructured weather data, the unstructured weather data comprising one or more images including weather-related features;
processing the historical unstructured weather data to generate values for at least one weather variable, the processing comprising extracting hue information from the one or more images;
receiving historical structured weather data;
combining the historical structured weather data with the generated weather variable values to create a combined weather feature set;
selecting one or more first algorithms based on one of more characteristics of the combined weather feature set; and
training the one or more first algorithms to forecast a weather indicator at least in part using the combined weather feature set to create one or more trained first algorithms the weather feature set including at least one weather variable based on the extracted hue information; and
wherein the trained second algorithm is created by:
receiving historical data for the power-related variable; and
training a second algorithm to forecast the power-related variable at least in part using the received historical data for the power-related variable and one or more of the historical unstructured weather data and the historical structured weather data.
10. The method of claim 9 , wherein the one or more first algorithms comprises one or more of regression models, Markov chains, time series models, state space models, Bayesian models, boosted decision trees, neural networks, convolutional neural networks, and recurrent neural networks.
11. The method of claim 9 , wherein the processing of the unstructured data comprises segmenting each of the one or more images into a plurality of segments and extracting hue information from each of the plurality of segments.
12. The method of claim 11 , wherein the hue information represents an image overlay.
13. The method of claim 1 , wherein the power-related variable comprises power output.
14. The method of claim 1 , wherein the power-related variable comprises electrical load.
15. The method of claim 1 , wherein the power-related variable relates to the possibility of a power outage.
16. The method of claim 13 , wherein the power-related variable comprises power output of a wind farm, and the structured data includes turbine data.