AQUA Satellite Data and Imputation of Geopotential Height: A Case Study for Pakistan

Usman Saleem^{1,2},
Mian Sohail Akram^{1},
Muhammad Fahad Ullah^{3},
Faisal Rehman^{3}^{*},
Muhammad Riaz Khan^{4}

Show more

1. Introduction

Missing data is a big problem encountered at a number of times during environmental research [1][2][3][4]. A lot of causes such as routine maintenances, sampling errors in satellite sensor, failures of satellite sensor during observations, meteorological abnormalities and human errors are responsible for the discontinuity of data set [3][4]. Geopotential height is the height of a pressure surface in the atmosphere above mean sea level [MSL]. The geopotential height data gathered from AQUA satellite contains incomplete data matrices in 24 standard pressures levels [5]. A research can become inaccurate if missing data sets are used [4][6]. Geopotential height was the function of air temperature, pressure, winds, and topography of the area, which required a careful method for its imputations. One of the oldest and most suggested methods to fill this missing information was replacing mean values of neighbor samples [1][2][3].

Many different interpolation techniques have been developed [2][6][7][8][9]. The best method depends upon the spatial and temporal variations of geopotential height in the atmosphere. Shen, Reiter [10], applied different interpolations on geopotential height keeping in view its variations in the atmosphere. Knox, Higuchi [11]investigate secular variations, Shabbar, Higuchi [12]did regional analysis, Griesser, Brönnimann [13]reconstructed geopotential height for 850, 700, 500, 300, 200 and 100 hPa. White [14]calculated statistics and climatology for the Northern Hemisphere’s geopotential height over 1000 and 500 hPa. Wallace, Zhang [15]investigated intera-decadal variability and teleconnections in the Northern hemisphere’s geopotential height over 500 and 700 hPa respectively.

Pakistan is the central country of South Asia bordered with India to East, China in North, South to Arabian Sea and Afghanistan to West (Figure 1 and Figure 2). It is arid to semi-arid country except in the north areas which received annual rainfall of 760 mm to 2000 mm annually. Pakistan has four provinces, of which Baluchistan is the driest and desert area facing 210 mm of rain averagely [16]. 3/4^{th} area of the country is getting no more than 250 mm of rain annually. In summer season relative humidity remains between 20% and 50%. In winter average temperature varies from 4˚C to 20˚C in most areas, while an increasing temperature of 0.6˚C to 1.0˚C is found along the coastal areas [17].

The actual thrust of this research work is to devise a workable methodology for carrying out scientific observations of upper atmosphere meteorology in Pakistan in spite of lacking modern equipment and technological resources in relevant departments. The published literature is not available in Pakistan, however, Saleem and Ahmed [18]; Saleem [19]; Saleem [20]are few initiatives on upper-level atmospheric observations.

2. Material and Methods

2.1. Data Used

In this research, the monthly mean of geopotential height [in meters]for the past 13 years, obtained from Atmospheric Infrared Sounder [AIRS]level 3, was used. AIRS was the instrument on AQUA satellite, which launched in May 2002.

Figure 1. Location map of the Pakistan with it host regions (20).

Figure 2. Altitude map of Pakistan showing elevation [in meters]depicted in different color scales (20).

This satellite has very high spectral resolutions: e.g., it captures climate data through nearly 2382 bands in the electromagnetic spectrum and its geopotential height product is very high resolution 0.5˚ × 0.5˚ grid cell. Version 6 of its product contains fewer biases in geopotential height [5]. Besides good quality of climate data, GESDISC^{1} provides geopotential height data for the whole global.

2.2. Spatial Interpolations of Missing Geopotential Height

Randomly 30% of the 324 samples were missing data which were then estimated from the 70% known data using different interpolation techniques like IDW, NN, BI and NI [2][7]. Robeson [21]; Price, McKenney [22]; Perry and Hollis [7]; Yozgatligil, Aslan [3]considered these performance parameters like, Mean Absolute Error [MAE], Root Mean Square Error [RMSE], Coefficient of Determinations [R^{2}]and Correlation Coefficient [Corr], to find out the best interpolation technique for missing climatic data set.

1) INVERSE DISTANCE WEIGHTING

This imputation resembles to Tobler’s first law of geography in which the weight of the known samples will be determined based on the distances from the imputed sample [Robeson, 1994]. More will be the distance of neighbors from a predicted sample less will be their weight in interpolation. Ferrari and Ozaki [9]used Equation (1) which is given below:

${z}_{ij}=\frac{{{\displaystyle \underset{a=1}{\overset{n}{\sum}}z}}_{oa}{d}_{a\text{\hspace{0.17em}}j}^{-r}}{{\displaystyle \underset{a=1}{\overset{n}{\sum}}{d}_{a\text{\hspace{0.17em}}j}^{-r}}}$ (1)

where
${d}_{aj}^{-r}$ is the weighting factor of distance between the a^{th} original neighbor sample
${z}_{oi}$ ,
${z}_{ij}$ is j^{th} the point to be estimated, n is the total number of the sample used, and r weighting factor. Langella [23], formula for IDW was used in the missing data imputations.

2) NEAREST NEIGHBORS INTERPOLATION [NN]

Missing values were directly imputed with a most suitable neighbor around the missing sample [24][25]in this interpolation technique.

3) BILINEAR INTERPOLATION [BI]

Junninen et al. [2004]used Equations (2) and (3) for Bilinear Interpolations

${z}_{i}={z}_{i}{}_{1}\text{\hspace{0.17em}}+m\left({z}_{o}+{z}_{o1}\right)$ (2)

$\begin{array}{l}m=\frac{{z}_{i}{}_{2}-{z}_{i}{}_{1}}{{z}_{o}{}_{2}-{z}_{o1}}\\ {z}_{o1}<{z}_{o}<{z}_{o}{}_{2}\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{and}\text{\hspace{0.17em}}\text{\hspace{0.17em}}{z}_{i}{}_{1}\text{\hspace{0.17em}}<{z}_{i}<{z}_{i}{}_{2}\end{array}$ (3)

It was a linear equation with $\left({z}_{o1},\text{\hspace{0.17em}}{z}_{I}{}_{1}\right)$ and $\left({z}_{o}{}_{2},\text{\hspace{0.17em}}{z}_{i}{}_{2}\right)$ sample values, m being a gradient of this line.

4) NATURAL NEIGHBORS INTERPOLATION [NI]

This spatial interpolation gives the nearest neighbor value of the sample to the missing geopotential height. D. and Boissonnat and Cazals [25]explain the selection of such natural neighbors for randomly missing data being on Delaunay triangulation.

2.3. Performance Indicators for Interpolations

These following performance parameters have been frequently used by Robeson [21]; Price, McKenney [22]; Junninen, Niska [2]; Perry and Hollis [7]; Stahl, Moore [24]; Norazian [4]; Ferrari and Ozaki [9]; Saleem and Ahmed [18]for imputation of missing climate data set.

1) ROOT MEAN SQUARE ERROR [RMSE]

Root Mean Square was calculated by dividing the sum of the square of the difference between imputed geopotential heights and actual value with the total number of samples, and then finally taking the square root of this term [4]. Smaller values indicate a perfect estimation of missing data set. Equation (4) was its mathematical formula used in this research.

$RMSE={\left(\frac{1}{n}{\displaystyle \underset{a=1}{\overset{n}{\sum}}{\left[{z}_{oa}\text{\hspace{0.17em}}-{z}_{ia}\right]}^{2}}\right)}^{\frac{1}{2}}$ (4)

This parameter calculates the total difference [±]between original and interpolated geopotential height.

2) MEAN ABSOLUTE ERROR [MAE]

This provides more information about the residual error as compared with RMSE. Junninen, Niska [2]and Norazian [4]provided Equation (5) for MAE.

$MAE=\frac{1}{n}{\displaystyle \underset{a=1}{\overset{n}{\sum}}\left|{z}_{ai}\text{\hspace{0.17em}}-{z}_{ia}\right|}$ (5)

MAE value range from 0 to $\infty $ . Its value close to 1 indicates more accurate and perfect imputation of missing data set.

3) CORRELATION COEFFICIENT [Corr]

Its value of +1 indicates very strong correlation and near to 0 signifies a bad correlation between actual and predicted geopotential height. Equation (6) was used for the correlation coefficient in this research.

$corr\text{\hspace{0.17em}}=\text{\hspace{0.17em}}\text{\hspace{0.17em}}\frac{\mathrm{cov}\left({z}_{i}\text{\hspace{0.17em}},{z}_{o}\right)}{{\partial}_{zi}{\partial}_{zO}}$ (6)

In Equation (6) nominator represents covariance while denominator represents the product of their standard deviations in the data set.

4) COEFFICIENT OF DETERMINATION [R^{2}]

This parameter provides a degree of correlation between the actual and predicted sample geopotential height [1]which varies between 0 and 1. Noor, Abdullah [4], suggested values closer to 1 indicate a perfect fit for the data set. Rahman and Islam, [2011]used the following formula for R^{2}.

${R}^{2}=\left[\frac{1}{n}\text{\hspace{0.17em}}\frac{{\displaystyle \underset{a=1}{\overset{n}{\sum}}\left({z}_{ia}-{A}_{i}\right)\left({z}_{oa}-{A}_{o}\right)}}{{\partial}_{zi}{\partial}_{zo}}\right]$ (7)

In Equation (7), A_{i} was the average of predicted samples and A_{o} is the average of sample values before prediction.

3. Results

These were the results of the performance parameter for each interpolation technique.

3.1. Performance Parameters from IDW

On all pressure level IDW showed very biased results. IDW produced highest RMSE ± 14.45 m over 1 hPa while lowest value of this error was ±3.66 m at 925 hPa. Actual and predicted values indicating low quality of interpolation for missing values of geopotential height with IDW as correlation coefficient was very low (Table 1).

3.2. Performance Parameters from Nearest Neighbor Interpolation

RMSE value remains between ±4.925 and ±11.369 m with Nearest Neighbor Interpolations. Such a large RMSE, poor correlation, and poor fit to the surface indicated bad refilling of data with this interpolation technique (Table 2).

3.3. Performance Parameters from Bilinear Interpolation

Bilinear Interpolation appeared to be relatively better as compared to the above mentioned two interpolations. RMSE was ±2.461 to ±5.241 m in refilling of gaps in data up to 1000 hPa. MAE remains less than 1 and strong correlation (0.98) was found in the imputation of geopotential height. Coefficient of Determination was close to 0.98 for imputation over 1, 1.5, 2, 3, 5, 7, 10, 15, 70, 100, 150, 200, 250, 300 hPa (Table 3).

3.4. Performance Parameters from Natural Neighbor Interpolation

Reasonable low RMSE come in refilling of geopotential height over 2, 3, 5, 7, 30, 50, 70, 200, 250, 400, 500, 600 hPa. Largest RMSE was ±5.10 m at 10 hPa and lowest RMSE ±2.2 m for refilling of gaps in data at 850, 925, 1000 hPa. A good correlation coefficient [near to 0.99]was come in the refilling of geopotential height. R^{2} was near to 1 concluding a good line of fit between actual and predicted data set (Table 4).

4. Discussions

Refilling of geopotential height over 24 pressure levels was good with Bilinear and Natural Neighbor Imputations (Tables 1-4). In order to nominate optimum interpolation from both of them, scatter plots of original and estimated geopotential heights were investigated. Poor data refilling was come in February and March (Figure 3(a)).

Table 1. Results indicating poor performance parameters with Inverse Distance Weighting Interpolation.

Table 2. Results indicating poor performance indicators from Nearest Neighbor Interpolation.

Table 3. Results indicating good performance parameters for refilling of gaps in data with Bilinear Interpolation.

Table 4. Good results of performance indicators with Natural Neighbor Interpolation.

Figure 3. (a) Results of interpolation of relative humidity [1 hPa to 15 hPa]with Bilinear Interpolation; (b) Bilinear Interpolation for relative humidity imputation from 20 hPa to 250 hPa; (c) Imputation of relative humidity from 300 hPa to 1000 hPa with Bilinear Interpolation.

The Imputations for months of January, February and March were not precise (over 20, 30, 50, 70, 100, 150, 200, 250 hPa) with Bilinear Interpolation. Bilinear Interpolation for remaining pressure levels accurately filled the gaps in the Geopotential height (Figure 3(b)).

The original sample and imputed sample for each months were plotted together to create theses scatter plots. However (over 500, 600, 850, 1000 hPa) Bilinear Interpolation poorly filled months of February, March and April (Figure 3(c)).

The similar technique of plotting original samples with imputed samples was used to create scatter plot of each month. Natural Neighbor Interpolation Imputations were more precise than Bilinear Interpolation. Only month of February was not good by Natural Neighbor Interpolation. Natural Neighbor Interpolation precisely filled the Geopotential height (over 1, 1.5, 2, 3, 5, 7, 10 and 15 hPa) (Figure 4(a)). The imputation with NNI for 20 hPa to 250 hPa and 300 hPa to 1000 hPa are illustrated in Figure 4(b) and Figure 4(c) respectively.

5. Conclusion

AQUA Satellite data was interpolated for Missing Data of Geopotential height. Based on critical checks and evaluation of interpolations regarding their product, it concluded that the NN and IDW interpolations for filling of missing

Figure 4. (a) Imputation of relative humidity (1 hPa to 15 hPa) with Natural Neighbor Interpolation; (b) Imputation of relative humidity (20 hPa to 250 hPa) with Natural Neighbor Interpolation; (c) Imputation of relative humidity (300 hPa to 1000 hPa) with Natural Neighbors Interpolation.

geopotential height data were proved not to be best and perfect (Table 1 and Table 2). Good results were found between BI and NI. However, after examining scatter plots of each month, it was found that NI was more accurate and reliable for missing data of geopotential height over 24 hPa levels.

Acknowledgements

The authors wish to acknowledge valuable guidance provided by Mr. Thomas Hearty and Mr. Edward T Olsen to refill gaps in AIRS relative humidity data set. The valuable suggestions are appreciated by Mr. Alessio Martion, University of the Rome, La Sapienza Italy which helped to improve this research.

NOTES

^{1}GESDISC stands for Goddard Earth Sciences Data Information Services Center.

References

[1] Schneider, T. (2001) Analysis of Incomplete Climate Data: Estimation of Mean Values and Covariance Matrices and Imputation of Missing Values. Journal of Climate, 14, 853-871.

https://doi.org/10.1175/1520-0442(2001)014<0853:AOICDE>2.0.CO;2

[2] Junninen, H., Niska, H., Tuppurainen, K., Ruuskanen, J. and Kolehmainen, M. (2004) Methods for Imputation of Missing Values in Air Quality Data Sets. Atmospheric Environment, 38, 2895-2907.

https://doi.org/10.1016/j.atmosenv.2004.02.026

[3] Yozgatligil, C., Aslan, S., Iyigun, C. and Batmaz, I. (2013) Comparison of Missing Value Imputation Methods in Time Series: The Case of Turkish Meteorological Data. Theoretical and Applied Climatology, 112, 143-167.

https://doi.org/10.1007/s00704-012-0723-x

[4] Noor, N.M., Abdullah, M.M.A.B., Yahaya, A.S. and Ramli, N.A. (2015) Comparison of Linear Interpolation Method and Mean Method to Replace the Missing Values in Environmental Data Set. Materials Science Forum, 803, 278-281.

[5] Tian, B., Manning, E., Fetzer, E., Olsen, E., Wong, S., Susskind, J., et al. (2013) AIRS/AMSU/HSB Version 6 Level 3 Product User Guide. Jet Propulsion Laboratory, Pasadena, CA.

[6] Rahman, G. and Islam, Z. (2011) A Decision Tree-Based Missing Value Imputation Technique for Data Pre-Processing. Proceedings of the Ninth Australasian Data Mining Conference, Ballarat, Australia, 1-2 December 2011, 41-50.

[7] Perry, M. and Hollis, D. (2005) The Generation of Monthly Gridded Datasets for a Range of Climatic Variables over the UK. International Journal of Climatology: A Journal of the Royal Meteorological Society, 25, 1041-1054.

https://doi.org/10.1002/joc.1161

[8] Hofstra, N., Haylock, M., New, M., Jones, P. and Frei, C. (2008) Comparison of Six Methods for the Interpolation of Daily, European Climate Data. Journal of Geophysical Research: Atmospheres, 113, 1-19.

https://doi.org/10.1029/2008JD010100

[9] Ferrari, G.T. and Ozaki, V. (2014) Missing Data imputation of Climate Datasets: Implications to Modeling Extreme Drought Events. Revista Brasileira de Meteorologia, 29, 21-28.

https://doi.org/10.1590/S0102-77862014000100003

[10] Shen, R., Reiter, E.R. and Bresch, J.F. (1986) Vertical Interpolation of Meterological Variables. Monthly Weather Review, 114, 123-134.

https://doi.org/10.1175/1520-0493(1986)114<0123:VIOMV>2.0.CO;2

[11] Knox, J.L., Higuchi, K., Shabbar, A. and Sargent, N.E. (1988) Secular Variation of Northern Hemisphere 50 kPa Geopotential Height. Journal of Climate, 1, 500-511.

https://doi.org/10.1175/1520-0442(1988)001<0500:SVONHK>2.0.CO;2

[12] Shabbar, A., Higuchi, K. and Knox, J.L. (1990) Regional Analysis of Northern Hemisphere 50 kPa Geopotential Heights from 1946 to 1985. Journal of Climate, 3, 543-557.

https://doi.org/10.1175/1520-0442(1990)003<0543:RAONHK>2.0.CO;2

[13] Griesser, T., Bronnimann, S., Grant, A., Ewen, T., Stickler, A. and Comeaux, J. (2010) Reconstruction of Global Monthly Upper-Level Temperature and Geopotential Height Fields Back to 1880. Journal of Climate, 23, 5590-5609.

https://doi.org/10.1175/2010JCLI3056.1

[14] White, G.H. (1980) Skewness, Kurtosis and Extreme Values of Northern Hemisphere Geopotential Heights. Monthly Weather Review, 108, 1446-1455.

https://doi.org/10.1175/1520-0493(1980)108<1446:SKAEVO>2.0.CO;2

[15] Wallace, J.M., Zhang, Y. and Lau, K.-H. (1993) Structure and Seasonality of Interannual and Interdecadal Variability of the Geopotential Height and Temperature Fields in the Northern Hemisphere Troposphere. Journal of Climate, 6, 2063-2082.

https://doi.org/10.1175/1520-0442(1993)006<2063:SASOIA>2.0.CO;2

[16] Farooqi, A.B., Khan, A.H. and Mir, H. (2005) Climate Change Perspective in Pakistan. Pakistan Journal of Meteorology, 2, 11-21.

[17] Iqbal, M.J. and Quamar, J. (2011) Measuring Temperature Variability of Five Major Cities of Pakistan. Arabian Journal of Geosciences, 4, 595-606.

https://doi.org/10.1007/s12517-010-0224-0

[18] Saleem, M.U. and Ahmed, S.R. (2016) Missing Data Imputations for Upper Air Temperature at 24 Standard Pressure Levels over Pakistan Collected from Aqua Satellite. Journal of Data Analysis and Information Processing, 4, 132-146.

https://doi.org/10.4236/jdaip.2016.43012

[19] Saleem, M.U. (2016) Statistical Investigation and Mapping of Monthly Modified Refractivity Gradient over Pakistan at the 700 Hectopascal Level. Open Journal of Antennas and Propagation, 4, 46-63.

https://doi.org/10.4236/ojapr.2016.42005

[20] Saleem, M.U. (2015) Atmospheric Ducts: Their Applications in Radio Frequency Propagation Using Satellite Remote Sensing Techniques. LAP Lambert Academic Publishing, Germany, 56 p.

[21] Robeson, S.M. (1994) Influence of Spatial Sampling and Interpolation on Estimates of Air Temperature Change. Climate Research, 4, 119-126.

https://doi.org/10.3354/cr004119

[22] Price, D.T., McKenney, D.W., Nalder, I.A., Hutchinson, M.F. and Kesteven, J.L. (2000) A Comparison of Two Statistical Methods for Spatial Interpolation of Canadian Monthly Mean Climate Data. Agricultural and Forest Meteorology, 101, 81-94.

https://doi.org/10.1016/S0168-1923(99)00169-0

[23] Langella, G. (2010) Inverse Distance Weighted (IDW) or Simple Moving Average (SMA) INTERPOLATION. File Exchange Version 1.0.0.0.

[24] Stahl, K., Moore, R., Floyer, J., Asplin, M. and McKendry, I. (2006) Comparison of Approaches for Spatial Interpolation of Daily Air Temperature in a Large Region with Complex Topography and Highly Variable Station Density. Agricultural and Forest Meteorology, 139, 224-236.

https://doi.org/10.1016/j.agrformet.2006.07.004

[25] Boissonnat, J.-D. and Cazals, F. (2002) Smooth Surface Reconstruction via Natural Neighbour Interpolation of Distance Functions. Computational Geometry, 22, 185-203.

https://doi.org/10.1016/S0925-7721(01)00048-7