Weather Prediction with Maching Learning

ClimateWins is a European non-profit exploring how machine learning can help predict weather and the impacts of climate change. ClimateWins wants to determine whether daily weather could be predicted as “pleasant” or “unpleasant” based on historical climate data. A variety of supervised and unsupervised algorithms were tested to identify patterns in temperature, precipitation, wind speed, cloud cover, etc. The goal was to assess both the feasibility of forecasting weather experiences and to determine which machine learning models delivered the most accurate and interpretable results.

Audience

  • ClimateWins shareholders

  • Environmental & Climate Scientists

  • Government Agencies & Policy Makers

Data

  1. Historical climate data from mainland Europe taken from 18 different weather stations daily.

    • Includes variables like temperature (max, min, and average), precipitation, wind speed, snow, cloud cover, etc.

  2. A second data set (‘answers’) contains labeled data—where “pleasant” days were marked as 1 and “unpleasant” days as 0. We used this set to train different supervised and unsupervised learning models

Algorithms used for the project:

  • K-Nearest Neighbor (KNN): A supervised machine learning algorithm that classifies or predicts data points based on the majority label or average value of the k closest data points in the feature space, making it simple and effective for both classification and regression tasks.

  • Decision Tree:

    A supervised machine learning algorithm that models decisions and their outcomes using a tree-like structure of if-then rules to identify patterns in the input features. In this project, a decision tree was trained on weather station data to classify days as either “pleasant” or “unpleasant” based on observed conditions.

  • Artificial Neural Network (ANN):

    A machine learning model inspired by the human brain, consisting of interconnected layers of nodes that apply non-linear transformations to detect complex patterns in the data. In this project, an ANN was used to model relationships in weather data and classify days as “pleasant” or “unpleasant” based on multiple observed conditions.

  • Convolutional Neural Network (CNN):

    A type of artificial neural network (ANN) designed for processing spatial data, using convolutional layers to automatically detect patterns, features, and relationships within the input. In this project, a CNN model was developed using Keras, based on the HAR (Human Activity Recognition) sample script, to explore its effectiveness in classifying weather conditions.

  • Random Forest:
    An ensemble learning algorithm that constructs multiple decision trees and aggregates their predictions to improve accuracy and reduce overfitting. In this project, a Random Forest model was used to classify weather conditions by leveraging the combined output of several trees trained on different subsets of the weather station data.

  • Generative Adversarial Network (GAN):
    A neural network framework consisting of two competing models—a generator and a discriminator—that work together to produce realistic synthetic data. In this project, a GAN was explored for its potential in generating synthetic weather patterns to augment the training dataset and improve model generalization.

Confusion Matrix:

  • Table that evaluates the performance of a classification model by comparing its predictions to the actual results.

Objectives:

  • Find new patterns in weather changes over the last 60 years.

  • Identify weather patterns outside the regional norm in Europe.

  • Determine whether unusual weather patterns are increasing.

  • Generate possibilities for future weather conditions over the next 25 to 50 years based on current trends.

  • Determine the safest places for people to live in Europe within the next 25 to 50 years.

K-Nearest Neighbor (KNN)

  1. Prepare the data

  • Define features (X) as the weather station data

  • Define the target (y) as the labeled "pleasant" or "unpleasant" day data

  • Split the data into training and testing sets using a standard train-test split

2. Run the KNN model on the data

3. Plot the relationship between number of neighbors and accuracy

4. Asses model performance with a confusion matrix

The K-Nearest Neighbors (KNN) model achieved 88% accuracy in classifying days as “pleasant” or “unpleasant,” making it the highest-performing model in this project.

*Sonnblick station shows overfitting. Sonnblick is a cold region in the Austrian Alps and the model correctly predicted every day as ‘unpleasant’.

Decision Tree

  1. Prepare the data

  • Define features (X) as the weather station data

  • Define the target (y) as the labeled "pleasant" or "unpleasant" day data

  • Split the data into training and testing sets using a standard train-test split

2. Run the decision tree

3. Asses the accuracy of the decision tree with a confusion matrix

Input:

Output:

*Again Sonnblick station shows overfitting. The model correctly predicted every day as ‘unpleasant’.

4. Check accuracy and f1 score for decision tree

The Decision Tree model achieved 60% accuracy in classifying days as “pleasant” or “unpleasant.” While this performance is modest, it indicates the model was able to capture some useful patterns in the data, though there is still significant room for improvement.

Artificial Neural Network (ANN)

  1. Prepare the data

  • Define features (X) as the weather station data

  • Define the target (y) as the labeled "pleasant" or "unpleasant" day data

  • Split the data into training and testing sets using a standard train-test split

2. Scale the data

3. Create the ANN and experiment with different hyperparameters to optimize accuracy.

  • The most accurate scenario was #3 pictured below.

4. Asses model performance using confusion matrix

5. Check accuracy of the ANN

The Artificial Neural Network (ANN) model achieved 62% accuracy in classifying days as “pleasant” or “unpleasant.” While further hyperparameter tuning could potentially improve performance, the ANN’s accuracy was comparable to that of the Decision Tree model, suggesting limited added benefit in this case.

Convolutional Neural Network (CNN)

  1. Prepare the data

  • Define features (X) as the weather station data

  • Define the target (y) as the labeled "pleasant" or "unpleasant" day data

  • Split the data into training and testing sets using a standard train-test split

2. Create Keras Layered Model

  • Four tests were run using the Tanh, Sigmoid, Relu, and Softmax activation functions to see which performed best. Model 3 will be showcased using the Softmax activation.

3. Run the model on the data.

  • Ideally, the training should converge, indicating the model has optimized its parameters effectively.

  • Accuracy should increase and loss should decrease.

4. Evaluate model performance using a confusion matrix.

Overall, model accuracy was very low across all trials. Trial 3, which used the softmax activation function, achieved only 13% accuracy with a loss exceeding 7 million. However, it was the only trial that successfully recognized all 15 weather stations. This suggests that, despite poor overall performance, the model architecture in Trial 3 may have potential. Adjusting hyperparameters or refining input features could help improve accuracy.

Random Forest Classification

Due to the size of the datasets, running a Random Forest model on the full data could be time-consuming. To improve efficiency, the datasets were reduced to include only records from the 2010s decade.

1. Prepare the data

  • Define features (X) as the weather station data

  • Define the target (y) as the labeled "pleasant" or "unpleasant" day data

  • Split the data into training and testing sets using a standard train-test split

2. Create Random Forest Model

3. Run the Random Forest

4. Use metrics module to assess accuracy.

The Random Forest model classified days as “pleasant” or “unpleasant” with 59% accuracy. While this indicates the model learned some meaningful patterns, it also suggests there is room for improvement through feature engineering, hyperparameter tuning, or trying alternative algorithms.

Uncovering Feature Importance

One of the key features of the Random Forest model is the ability to determine ‘feature importance’. This basically means that it can tell us which features (in this case weather stations) are most important to training the model.

1. Retrieve feature importance from the trained Random Forest model.

2. Reshape the array and collapse into one observation for each weather station.

3. Reassign the stations to a list and assess the importance of each station.

4. Create a data frame to associate the stations with their ‘importance’.

5. Plot the results.

This means that, during model training, the data from the Maastricht station had the greatest impact on the model’s ability to distinguish between “pleasant” and “unpleasant” days. Followed by data from Heathrow, Stockholm, and so on. These stations provided weather patterns that were most strongly correlated with the target labels, making them the most influential in the classification process.

Generative Adversarial Network (GAN)