In-season and dynamic crop mapping using 3D convolution neural networks and sentinel-2 time series

An accurate, frequently updated, automatic and reproducible mapping procedure to identify seasonal cultivated crops is a prerequisite for many crop monitoring activities. Deep learning was demonstrated to be an effective mapping approach already successfully applied to decametric resolution satellite images (like Sentinel-2 data) to produce yearly crop maps. In this framework, algorithm training is performed with ground truth typically consisting of spatially explicit information available after the end of the season (e.g. yearly crop maps and/or farmer declaration for subsidies at parcel level); however, such data (i) does not allow performing in-season prediction, and (ii) does not provide temporal details fundamental to describe a dynamic crop succession and/or to understand crop management (i.e. planting and harvesting). In this paper we present a Deep Neural Network-based approach capable of generating (i) a crop map of the current season at a specific point in time ("In season mapping"conventionally at the end of the current year), along with (ii) all intermediate maps during the season able to describe in near real-time the evolution of crop presence ("Dynamic-mapping"at the temporal granularity of satellite imagery revisiting, e.g., 5 days for Sentinel-2 data). This approach adopts a smart training procedure of a Deep Neural model by exploiting historical satellite data and ground truth. We introduce a method to automatically generate "short-term"ground truth maps (i.e. 5 days reference) starting from the "long-term"ones (i.e. available yearly static reference) and characterizing temporally the different crop presence by performing a phenological analysis of historical time series. The model was trained and validated in Lombardy (North of Italy) exploiting multi-annual authoritative crop maps from 2016 to 2019. Validation was performed both in time (same areas used for training in a different year) and space (different location) for the year 2019. The quantitative error metrics calculation and Spatio-temporal analysis clearly demonstrate that the model can predict in-season crop presence with a generalization capacity over the long-term (yearly maps: OA > 70% and Kappa > 0.64%) and that the short-term predictions (5 days maps) are coherent with the reference information from expert knowledge (local crop calendars). The model can produce dynamically along the season short-term maps with a medium-high crop-specific User Accuracy at the maximum green-up phase (UA > 53% up to 95%). These products are of extreme interest for final users providing information at the peak of plant development that dynamically changes according to the considered crop, the specific location and the investigated season. These results demonstrate that it is possible to produce a crop map early in the season and extract useful additional information such as crop intensity (e.g. double crops presence) and crop dynamics related to different sowing dates.