Processing Data from Satellite Remote Sensing

Data formats

Most NASA EOS data distributed as HDF files, which are very similar to netCDF.

alt text

NetCDF / HDF

alt text

Reading EO Data

terra package functions can read NetCDF/HDF files if ncdf4 package is installed.

ncfile="data/MOD11A2.006_aid0001.nc"
nc=ncdf4::nc_open(ncfile,verbose = F)
str(nc$var)

## List of 3
##  $ crs        :List of 20
##   ..$ id                :List of 5
##   .. ..$ id         : num 0
##   .. ..$ group_index: num -1
##   .. ..$ group_id   : int 65536
##   .. ..$ list_index : num 1
##   .. ..$ isdimvar   : logi FALSE
##   .. ..- attr(*, "class")= chr "ncid4"
##   ..$ name              : chr "crs"
##   ..$ ndims             : int 0
##   ..$ natts             : int 10
##   ..$ size              : num 1
##   ..$ dimids            : int NA
##   ..$ prec              : chr "byte"
##   ..$ units             : chr ""
##   ..$ longname          : chr "crs"
##   ..$ group_index       : int 1
##   ..$ chunksizes        : int NA
##   ..$ storage           : int 1
##   ..$ shuffle           : int 0
##   ..$ compression       : logi NA
##   ..$ dims              : list()
##   ..$ unlim             : logi FALSE
##   ..$ make_missing_value: logi FALSE
##   ..$ missval           : logi NA
##   ..$ hasAddOffset      : logi FALSE
##   ..$ hasScaleFact      : logi FALSE
##   ..- attr(*, "class")= chr "ncvar4"
##  $ LST_Day_1km:List of 24
##   ..$ id                :List of 5
##   .. ..$ id         : num 4
##   .. ..$ group_index: num -1
##   .. ..$ group_id   : int 65536
##   .. ..$ list_index : num 2
##   .. ..$ isdimvar   : logi FALSE
##   .. ..- attr(*, "class")= chr "ncid4"
##   ..$ name              : chr "LST_Day_1km"
##   ..$ ndims             : int 3
##   ..$ natts             : int 14
##   ..$ size              : int [1:3] 42 47 811
##   ..$ dimids            : int [1:3] 2 1 0
##   ..$ prec              : chr "int"
##   ..$ units             : chr "K"
##   ..$ longname          : chr "8-day daytime 1km grid Land-surface Temperature"
##   ..$ group_index       : int 1
##   ..$ chunksizes        : int [1:3] 21 24 406
##   ..$ storage           : int 2
##   ..$ shuffle           : int 1
##   ..$ compression       : int 4
##   ..$ dims              : list()
##   ..$ dim               :List of 3
##   .. ..$ :List of 10
##   .. .. ..$ name         : chr "xdim"
##   .. .. ..$ len          : int 42
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 2
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 3
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "m"
##   .. .. ..$ vals         : num [1:42(1d)] 1373722 1374649 1375575 1376502 1377429 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   .. ..$ :List of 10
##   .. .. ..$ name         : chr "ydim"
##   .. .. ..$ len          : int 47
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 1
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 2
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "m"
##   .. .. ..$ vals         : num [1:47(1d)] 2362432 2361505 2360578 2359652 2358725 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   .. ..$ :List of 11
##   .. .. ..$ name         : chr "time"
##   .. .. ..$ len          : int 811
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 0
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 1
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "days since 2000-01-01 00:00:00"
##   .. .. ..$ calendar     : chr "julian"
##   .. .. ..$ vals         : int [1:811(1d)] 48 56 64 72 80 88 96 104 112 120 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   ..$ varsize           : int [1:3] 42 47 811
##   ..$ unlim             : logi FALSE
##   ..$ make_missing_value: logi TRUE
##   ..$ missval           : int 0
##   ..$ hasAddOffset      : logi TRUE
##   ..$ addOffset         : num 0
##   ..$ hasScaleFact      : logi TRUE
##   ..$ scaleFact         : num 0.02
##   ..- attr(*, "class")= chr "ncvar4"
##  $ QC_Day     :List of 22
##   ..$ id                :List of 5
##   .. ..$ id         : num 5
##   .. ..$ group_index: num -1
##   .. ..$ group_id   : int 65536
##   .. ..$ list_index : num 3
##   .. ..$ isdimvar   : logi FALSE
##   .. ..- attr(*, "class")= chr "ncid4"
##   ..$ name              : chr "QC_Day"
##   ..$ ndims             : int 3
##   ..$ natts             : int 7
##   ..$ size              : int [1:3] 42 47 811
##   ..$ dimids            : int [1:3] 2 1 0
##   ..$ prec              : chr "short"
##   ..$ units             : chr ""
##   ..$ longname          : chr "Quality control for daytime LST and emissivity"
##   ..$ group_index       : int 1
##   ..$ chunksizes        : int [1:3] 42 47 811
##   ..$ storage           : int 2
##   ..$ shuffle           : int 1
##   ..$ compression       : int 4
##   ..$ dims              : list()
##   ..$ dim               :List of 3
##   .. ..$ :List of 10
##   .. .. ..$ name         : chr "xdim"
##   .. .. ..$ len          : int 42
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 2
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 3
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "m"
##   .. .. ..$ vals         : num [1:42(1d)] 1373722 1374649 1375575 1376502 1377429 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   .. ..$ :List of 10
##   .. .. ..$ name         : chr "ydim"
##   .. .. ..$ len          : int 47
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 1
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 2
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "m"
##   .. .. ..$ vals         : num [1:47(1d)] 2362432 2361505 2360578 2359652 2358725 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   .. ..$ :List of 11
##   .. .. ..$ name         : chr "time"
##   .. .. ..$ len          : int 811
##   .. .. ..$ unlim        : logi FALSE
##   .. .. ..$ group_index  : int 1
##   .. .. ..$ group_id     : int 65536
##   .. .. ..$ id           : int 0
##   .. .. ..$ dimvarid     :List of 5
##   .. .. .. ..$ id         : int 1
##   .. .. .. ..$ group_index: int 1
##   .. .. .. ..$ group_id   : int 65536
##   .. .. .. ..$ list_index : num -1
##   .. .. .. ..$ isdimvar   : logi TRUE
##   .. .. .. ..- attr(*, "class")= chr "ncid4"
##   .. .. ..$ units        : chr "days since 2000-01-01 00:00:00"
##   .. .. ..$ calendar     : chr "julian"
##   .. .. ..$ vals         : int [1:811(1d)] 48 56 64 72 80 88 96 104 112 120 ...
##   .. .. ..$ create_dimvar: logi TRUE
##   .. .. ..- attr(*, "class")= chr "ncdim4"
##   ..$ varsize           : int [1:3] 42 47 811
##   ..$ unlim             : logi FALSE
##   ..$ make_missing_value: logi TRUE
##   ..$ missval           : int 0
##   ..$ hasAddOffset      : logi FALSE
##   ..$ hasScaleFact      : logi FALSE
##   ..- attr(*, "class")= chr "ncvar4"

Read NetCDF with rast()

lst=rast(ncfile,subds="LST_Day_1km")

lst

## class       : SpatRaster 
## dimensions  : 47, 42, 811  (nrow, ncol, nlyr)
## resolution  : 926.6254, 926.6254  (x, y)
## extent      : 1373259, 1412177, 2319343, 2362895  (xmin, xmax, ymin, ymax)
## coord. ref. : +proj=aea +lat_0=23 +lon_0=-96 +lat_1=29.5 +lat_2=45.5 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs 
## source      : MOD11A2.006_aid0001.nc:LST_Day_1km 
## varname     : LST_Day_1km (8-day daytime 1km grid Land-surface Temperature) 
## names       : LST_D~1km_1, LST_D~1km_2, LST_D~1km_3, LST_D~1km_4, LST_D~1km_5, LST_D~1km_6, ... 
## unit        :           K,           K,           K,           K,           K,           K, ... 
## time (days) : 2000-02-18 to 2017-10-16

Plot Raster Stack

lst[[1:10]] %>% gplot()+
    geom_raster(aes(fill=value))+
    facet_wrap(~variable,nrow=1)+
    coord_equal()+
    theme(axis.text=element_blank())

Collection-Level Naming Conventsions

MODIS/Terra Surface Reflectance 8-Day L3 Global 500m SIN Grid V005

MODIS/Terra - Instrument/Sensor
Surface Reflectance - Geophysical Parameter
8-Day - Temporal Resolution
L3 - Processing Level
Global - Global or Swath
500m - Spatial Resolution
SIN Grid - Gridded or Not
V005 - Collection Version

MODIS Gridding system

Filename Conventions

MOD09A1.A2006001.h08v05.005.2006012234657.hdf

MOD09A1 - Product Short Name
.A2006001 - Julian Date of Acquisition (A-YYYYDDD)
.h08v05 - Tile Identifier (horizontalXXverticalYY)
.005 - Collection Version
.2006012234567 - Julian Date of Production (YYYYDDDHHMMSS)
.hdf - Data Format (HDF-EOS)

MODIS Temporal Resolution

Daily
8-Day
16-Day
Monthly
Quarterly
Yearly

MODIS Spatial Resolution

Bands 1–2 250-meter
Bands 3–7 500-meter
Bands 8–36 1000-meter

MODIS Data

Distributed Active Archive Centers (DAACs)

Level 1 data: geolocation, cloud mask, and atmosphere products ladsweb.nascom.nasa.gov
Land products: lpdaac.usgs.gov
Cryosphere (snow/ice) products: nsidc.org/daac/modis
Ocean color and sea surface temperature: oceancolor.gsfc.nasa.gov

Accessing data

The Land Processes Distributed Active Archive Center has a nice “Data Discovery” Tool:

MODIS Products Table

Lists all available MODIS land products

Annual Land Cover Type Description

Detailed layer information

Data access links

Downloading: http/ftp access

or the LP DAAC2Disk Download Manager which builds a download script.

MODIS Reprojection Tool

Available at lpdaac.usgs.gov/tools/modis_reprojection_tool.

MODIS Reprojection Tool: Subset & Reproject

MCD12Q1.A2012001.h12v04.051.2014288200441.hdf

Application for Extracting and Exploring Analysis Ready Samples (AppEEARS)

https://lpdaacsvc.cr.usgs.gov/appeears/

Provide access to regional subsets
Merge across tiles
Download full time series

Data Subset

Submit and wait for results…

Case Study

Analysis of Land Use Land Cover and Land Surface Temperature across Buffalo using MODIS timeseries.

Presentation Credits

Images: NASA
Some contents from Steve Ackerman stevea@ssec.wisc.edu, Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin-Madison