BeGees - Docker-Compose


Begees is an internal BeOptimized project focused on the understanding of the energy consumption and production at BeOptimized.
This project uses several open-source packages and covered various aspects of the data lifecycle: the data capture, data quality, data transfert, data storage and data presentation. Energy data are captured thanks a to a Raspberry Pi connected to a smart metter and data are sent to a database via network. To make it working, all the packages & the database have been converted into containers orchestrate via Docker-Compose. The containers and Docker are run on a NAS server and mounted to my Deskop folders. More preciselly:

    • Creation of a web application (Python - Django - Mysql) displaying/ analyzing in near real-time the energy consumption and production of BeOptimized.
    • Energy consumption and production are captured thanks to a Raspberry PI connected to a smart-meter and scheduled using Apache Airflow.
    • Gaz and Water indexes are captured thanks to WebCams.
    • Database and Web application are Docker container hosted on a NAS server. Data are sent from the Raspberry to the NAS thanks to the Django RESTAPI framework
    • Data are analyzed thanks to Pandas, Tensorflow and OpenCV.
    • Environment: Python, Docker, Docker-Compose, Raspberry Pi (Debian), Synology (DSM)

BeGees: Control your energies !


This project will continue to evolve in the futur, we could indeed analyse the time series to detect unusual patterns, analyse devices signatures, make some trends, analyse electric car consumption, generate SMS when there is a gaz leakage or a consumption peak or even when the electritity is down ...
On the Raspberry, we could also replace the SQlite/Airflow by a Docker image in order to package the solution and install it more easily. We could also put the existing containers into the Cloud and replace Docker-Compose by Kubernetes...

We could go even further... imagine you are a manufacturing industry with lot of machines to monitor ... we could work together in order to analyse in real time everything you want !. Feel free to contact us, it will be a pleasure to help you.

Step by step

By Christophe Kabacinski Duration 8 mn

In this video the project and devices are presented

By Christophe Kabacinski Duration 5 mn

In this video I briefly explain which python library was used to make the connection to the smart-meter. I also explain how I did the data extraction and storage.
In a second phase I describe what I did to push the information to Power BI in order to display the information in real-time.

By Christophe Kabacinski Duration 5 mn

Crontab was firstly used in the project but when I saw the GUI and simplicity of Airflow I couldn't resist anymore :-).

By Christophe Kabacinski Duration 10 mn

If, in 2023, you are still installing your developments software on your desktop, dealing with versions, trying software and then removing them... they you should have a look to Docker.
Personnaly, I never used it before and it's like an industrial revolution... In a couple of click you can install an image, run it as a container; don't have to worry anymore if it's compatible with your OS.
if you don't like your image, you simply delete it -)

By Christophe Kabacinski Duration 6 mn

Here I explain how I use Tensorflow to read picture, how I store the pictures successfully processed or not and push them to the Django RestAPI

By Christophe Kabacinski Duration 9 mn

Django is the central piece of the puzzle because it is used to collect the dataflow through the RestAPI framework and it is used to display the reports and graphics in the browser.
In that video I start with PhpMyadmin to display the data tables stored in the MySQL DB, then I explain briefly that Django created these tables thanks to classes in the
Django is also practical because it makes a separation between the Python coders and the HTML designers.

This site uses cookies. .

By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details.