Cam2Sat

From the satellite perspective, a cloud could be both a problem and study phenomena. Land observation is severely limited by cloud presence where approximately 70% of the surface is cloud-covered at any time [King et al 2013]. On the other hand, meteorological satellites aim to specifically monitor cloud dynamics and related climatological parameters. In both cases, accurate cloud segmentation plays a key role in data processing, in the first case to be masked out and in the second one to be clearly identified and tracked. Satellite image cloud masking often overestimates or underestimates cloud presence which leads to error propagation in downstream products. Those consequences are generally due to bright surfaces and the semi-transparent nature of thin clouds, which have mixed pixel values as they combine cloud and ground signals giving rise to tradeoffs between omission and commission errors [Tarrio et al 2020]. Moreover, existing satellite cloud masking references are usually generated manually and consequently prone to mislabels caused by the subjectivity of human photo interpretation during the annotation procedure [Baetens et al 2019].
This project will address these challenges by studying the feasibility of creating a fisheye sky camera network to automatically retrieve high-resolution georeferenced cloud cover information from full-sky images. The derived information will be directly used in combination with satellite observations to either validate and/or improve cloud masks for land observation satellites and complement meteorological satellites and models with one of the most relevant yet challenging variables to be retrieved, namely real-time cloud cover information. Specialized hardware will be deployed along with novel methods for semantic image segmentation. Knowledge-based and deep-learning approaches will be tested and integrated to generate a robust all-scenario solution capable of extracting quality information from any environmental and scene condition.