As a tech enthusiast and a DIYer, I often find myself looking for ways to integrate various technologies to solve real-world problems. Recently, I embarked on a project to classify clouds using my existing Allsky camera setup. Despite searching extensively, I couldn’t find a ready-to-go implementation for a Home Assistant instance. So, I decided to create my own solution. This blog post details the journey of building a cloud classification system using TensorFlow Lite, running on an ODroid C2, and achieving an impressive 90% accuracy.

Project Overview

The goal of this project was to classify cloud types from images captured by my Allsky camera and publish the results to an MQTT broker, which can then be integrated with Home Assistant. The project leverages a TensorFlow Lite model trained on the CCSN dataset.

Data Augmentation

To improve the robustness of the model, I used data augmentation techniques. Data augmentation helps artificially increase the training dataset’s size by creating modified versions of images in the dataset. Here’s the code snippet for data augmentation:

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Step 1: Data Augmentation
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=40,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    fill_mode='nearest'
)

val_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical'
)

val_generator = val_datagen.flow_from_directory(
    val_dir,
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical'
)

Model Creation Using Transfer Learning

For the model, I used MobileNetV2 as the base model due to its efficiency and accuracy. Transfer learning allows us to leverage pre-trained models on large datasets, which can significantly speed up the training process and improve accuracy. Here’s the code snippet for creating the model:

from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras import layers, models

# Step 2: Create Model using Transfer Learning (MobileNetV2 as base model)
base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(128, activation='relu'),
    layers.BatchNormalization(),
    layers.Dropout(0.5),
    layers.Dense(train_generator.num_classes, activation='softmax')  # number of classes detected automatically
])

Training the Model

The model was trained on the CCSN dataset, which contains various cloud types. The training process involved fine-tuning the pre-trained MobileNetV2 model and adjusting hyperparameters to achieve the best performance.

Deployment on ODroid C2

Once the model was trained, I converted it to TensorFlow Lite format for deployment on the ODroid C2. TensorFlow Lite is optimized for mobile and embedded devices, making it an ideal choice for this project.

Integration with Home Assistant

The final step was to integrate the cloud classification system with Home Assistant. The classified cloud types are published to an MQTT broker, which Home Assistant subscribes to. This allows for real-time updates and automation based on cloud conditions.

Conclusion

This project was a fantastic learning experience, and I’m thrilled with the results. Achieving 90% accuracy in cloud classification and integrating it with Home Assistant opens up numerous possibilities for automation and weather monitoring.

If you’re interested in cloud classification or looking to enhance your Home Assistant setup, check out the project on GitHub: GitHub Repository

Feel free to reach out if you have any questions or suggestions!