Improved estimates of organic carbon using proximally sensed vis-NIR spectra corrected by piecewise direct standardization

Abstract

We investigated the use of piecewise direct standardization (PDS) to remove the effects of water and other environmental factors from proximally sensed (field) visible-near infrared (vis-NIR) spectra. Our hypothesis was that the PDS-standardized field spectra can be used to predict soil carbon effectively with calibrations derived from existing spectroscopic databases of spectra recorded in the laboratory on dried, ground and sieved samples. In our experiments we used field spectra recorded in situ with a portable spectrometer at 124 sites in 11 paddy fields in Zhejiang Province, China. We sampled the soil at these same sites, recorded their spectra in the laboratory and measured their soil organic carbon (SOC) contents with a conventional laboratory technique. Two-thirds of the samples were used to relate the laboratory spectra to SOC by partial least squares regression (PLSR), and the remaining one-third was used as an independent validation dataset. We selected a representative set of samples from corresponding field and laboratory spectra that we could use as the PDS transfer set. Piecewise direct standardization was used to relate each wavelength in the laboratory spectra to the corresponding wavelength and its neighbours in the field spectra. The field spectra of the validation samples were then corrected with PDS so that they acquired the characteristics of the spectra measured under laboratory conditions. The approach was evaluated by (i) quantifying the similarity between the PDS-standardized spectra and their corresponding laboratory spectra, (ii) measuring the accuracy of their SOC predictions on the independent validation dataset and (iii) comparing these results with those of direct standardization (DS). Both PDS and DS led to considerable improvements in the predictions of SOC (R<sup>2</sup>=0.71, R<sup>2</sup>=0.60, respectively), compared with those with original field spectra (R<sup>2</sup>=0.03). However, fewer transfer samples were needed with PDS to obtain similar results.