Google Earth Engine Python API – Resolving Function Not Mapping Over a Collection

google-earth-enginepython

I have made a function to add an isNull:False and the mean values of each band as properties to an image, or isNull: True if the image contains no data. I want to use the function to map over an image collection.

A reproducible example:

import ee
ee.Initialize()

def cloudmask_LS(image):
    cloudAdjacentBitMask = (1 << 3)
    cloudShadowBitMask = (1 << 2)
    cloudsBitMask = (1 << 1)
    ## get the pixel QA band.
    qa = image.select('QA_PIXEL') 
    mask = qa.bitwiseAnd(cloudShadowBitMask).eq(0).And(qa.bitwiseAnd(cloudsBitMask).eq(0)).And(qa.bitwiseAnd(cloudAdjacentBitMask).eq(0))
    return image.updateMask(mask)

aoi = ee.Geometry.Polygon([10.02244,2.27462, 10.06668,2.26841, 10.06869,2.23237, 10.01983,2.23518])

start_date =  "2018-01-01"
end_date =  "2021-12-31"
l7 = (ee.ImageCollection("LANDSAT/LE07/C02/T1_L2")
        .filterDate(ee.Date(start_date), ee.Date(end_date))
        .filterBounds(aoi)
        .map(lambda image: image.clip(aoi))
    )

## band names
band_names = ['SR_B1', 'SR_B2', 'SR_B3', 'SR_B4', 'SR_B5', 'SR_B7', 'QA_PIXEL']

## cloud mask collection
l7 = l7.select(band_names).map(cloudmask_LS)

## drop QA_PIXEL band
band_names = [b for b in band_names if not b == 'QA_PIXEL']
l7 = l7.select(band_names)

def get_band_means(image):
    ## get means of bands in a dictionary
    meanDict = image.reduceRegion(reducer=ee.Reducer.mean(), geometry=aoi, scale=30)
    
    ## convert to Python dict
    means = meanDict.getInfo()
    
    ## if mean values missing, set isNull to True
    if None in means.values():
        isNull = True
    else:
        isNull = False
    
    ## add a key, value pair to the means dict indicating whether image is null
    means['isNull'] = isNull
    
    ## add the means dictionary to the image as properties
    return image.set(means)
    
l7 = l7.map(get_band_means)

Throws the error:

ee.ee_exception.EEException: ValueNode is empty

and a little further up the error message

with_cast = lambda e: algorithm(element_type(e))
File "< string >", line 39, in get_band_means

which is the line means = meanDict.getInfo()

However, when I choose a null image and manually perform the operations, it works as expected.

l7_list = l7.toList(len(l7.getInfo()['features']))

## a null image was found by visual inspection
null_image = ee.Image(l7_list.get(4))

## get means of bands in a dictionary
meanDict = null_image.reduceRegion(reducer=ee.Reducer.mean(), geometry=aoi, scale=30)

## convert to Python dict
means = meanDict.getInfo()

## if mean values missing, set isNull to True
if None in means.values():
    isNull = True
else:
    isNull = False

## add a key, value pair to the means dict indicating whether image is null
means['isNull'] = isNull

## add the means dictionary to the image as properties
null_image = null_image.set(means)

null_image.getInfo()['properties']

Note the isNull property is added as desired.

{'system:footprint': {'type': 'Polygon', 'coordinates': […]}, 'isNull': True, … }

To verify the mean values were added as properties, I chose a non-null image and repeated the operations manually. The properties were successfully added.


nonNull_image = ee.Image(l7_list.get(70))

{'system:footprint': {'type': 'Polygon', 'coordinates': […]}, 'SR_B1': 8933.524251313218, 'isNull': False, 'SR_B2': 9380.560890927749, 'SR_B3': 8928.16497726643, 'SR_B4': 17325.19809467158, 'SR_B5': 11964.02872713739, 'SR_B7': 9085.548239710133,

Best Answer

You can't use .getInfo() in the function you pass to ee.ImageCollection.map(). The result of .getInfo() is a client-side object, which can't get back to EE servers in a .map expression. You can do ee.ImageCollection.filter(ee.Filter.notNull()) on the reduction properties to get rid of images that had a null reduction.

Basically your get_band_means function would be this:

def get_band_means(image):
    ## get means of bands in a dictionary
    meanDict = image.reduceRegion(reducer=ee.Reducer.mean(), geometry=aoi, scale=30)
    
    ## add the means dictionary to the image as properties
    return image.set(meanDict)

...and then to remove any images that had null reductions, do this:

l7_with_data = l7.filter(ee.Filter.notNull(band_names))

When you compare the collection size, original has 121 images and after filtering out the null reduction images there are 80.

If you need to know what images were filtered out, you can probably get that info with a join on the pre- and post-null-filtered collections (submit a separate question).

Related Question