In 2020, I looked at 3 publicly available rainfall datasets- IMERG, CHIRPS and ARC2 and compared their accuracy against ground station data from several locations across North and Central Mali. I’ve put the writeup and results here for anyone interested in reproducing the analysis.
An archive of CHIRPS, ARC2 and IMERG data was used, having been resampled and reprojected to the same resolution (10km). These were compared against station data downloaded from Tutiempo (https://en.tutiempo.net/climate/mapa/) and the NOAA ground station database . Full-year dekadal station data was acquired for Mopti (2015, 2016,2017), Hombori (2010,2011) and Gao (2009)
Station data for Mopti and Gao was downloaded from the Tutitempo service and used to compare the three satellite datasets. The Hombori data was acquired from the NOAA’s ground station database. The Mopti dataset contained the least number of visible errors. Both the Hombori and Gao station data contained several gaps. Numerous gaps in data are observed for Gao station throughout the season that do not seem to follow regular rainfall patterns. As a result, dekadal observations for Gao station may not be valid. However, the seasonal total of 269mm recorded for Gao is on par with observed averages for the cercle of Gao. As a result, while the study takes into account the results from Gao and Hombori, the Mopti data forms the core.
Method
The coordinates of each station was placed against a raster-stack of each dataset. A dekadal timeseries for each dataset was created for the single pixel containing the station’s coordinates. These were compared by dekadal and seasonal totals.
Results:
ARC2 was the most accurate of the datasets for both seasonal and dekadal totals.
In terms of rainfall quantity, Arc2 seasonal totals had a 99% accuracy compared to the Gao station data and generally high accuracy for the Mopti station, with the exception of 2017, in which CHIRPS data was considerably more accurate. The Hombori station data was found to be closest to CHIRPS’ seasonal estimates. The IMERG dataset was the least accurate in seasonal totals. Rainfall was vastly overestimated for all sites. It is worth noting that only the Mopti station estimates are considered fully reliable.
Error of seasonal totals compared to ground stations:
Seasonal totals by dataset (mm):
Both ARC2 and CHIRPS were each the most accurate dataset for three periods.. However, given that the Mopti station data is more reliable, ARC2’s higher score here indicates a better accuracy. As a result, ARC2 is considered the most accurate dataset in terms of seasonal totals.
For Mopti Station, ARC2 totals were generally on-par with station-level observations for Mopti station, mimicking the dips and rises. CHIRPS’s totals follow a much smoother profile, however they do not reflect the dips and rises of the dekadal totals of mopti station. Even in the 2017 period, where CHIRPS seasonal totals were closest to the ground data, a decline can be observed in CHIRPS totals from dekad 083 until the end of the season which is not reflected in the ground data, or any other dataset. IMERG data significantly overestimated dekadal totals, showing multiple peaks at double or triple the value of any other observed dataset. Likewise multiple peaks were registered (such as 2015083, 2016073 2017072 that were undetected by any other dataset.
For Hombori, both CHIRPS and ARC2 follow similar trends, with ARC2 overestimating and CHIRPS giving generally more conservative dekadal totals. IMERG is very clearly the least accurate of datasets here. However the distinction between ARC2 and CHIRPS is difficult, given that CHIRPS seems to perform more accurately in 2011, whereas ARC2 seems to mirror the seasonal peaks more closely in 2010. CHIRPS does seem to be the more accurate rainfall dataset here, given that the seasonal totals are likewise much closer.
For Gao, the station dekadal totals seem to be unreliable, as the station recorded an abnormally high rainfall of 150mm in the first dekad of the season (Second dekad of May) of 150mm and no 0 rainfall until the second dekad of June. This points to a likely error (no additional sources confirmed such high rainfall).
While ARC2’s seasonal totals were remarkably close to the station’s readings, the dekadal totals do not show a strong match. No dataset seems to mirror the station’s dekadal totals, indicating a likely error.
Correlation Coefficient against ground data by dekad
The chart above shows the correlation coefficient between all three satellite sources and station ground data. Gao was not included due to its unreliability. ARC2 is most strongly correlated with dekadal station values for both sites (though the margin is quite close in Hombori).
Conclusion:
Based on comparisons with station observations, ARC2 is the most accurate of the three datasets. ARC2 performed the most reliably with the most reliable of stations- the Mopti station. Likewise the dekadal totals were the most accurate of the three. Dekadal accuracy should be prioritised over seasonal accuracy, given that pastoralists will most likely need rainfall data early in the season. ARC2 should be recommended for use with the GARBAL interface.