Ingest satellite imagery in Azure Data Manager for Agriculture

Satellite imagery is a foundational pillar of agriculture data. To support scalable ingestion of geometry-clipped imagery, Microsoft partnered with Sentinel Hub by Sinergise to provide a seamless bring your own license (BYOL) experience for Azure Data Manager for Agriculture. You can use this BYOL experience to manage your own costs. This capability helps you with storing your field-clipped historical and up-to-date imagery in the linked context of the relevant fields.

Prerequisites

• To search for and ingest imagery, you need a user account that has suitable subscription entitlement with Sentinel Hub.
• Read the Sinergise Sentinel Hub terms of service and privacy policy.
• Have your providerClientId and providerClientSecret values ready.

Ingesting geometry-clipped imagery

Using satellite data in Azure Data Manager for Agriculture involves the following steps:

Note

Microsoft Azure Data Manager for Agriculture is currently in preview. For legal terms that apply to features that are in beta, in preview, or otherwise not yet released into general availability, see Supplemental Terms of Use for Microsoft Azure Previews.

Microsoft Azure Data Manager for Agriculture requires registration and is available to only approved customers and partners during the preview period. To request access to Microsoft Data Manager for Agriculture during the preview period, use this form.

Consumption visibility and logging

Because all ingested data is under a BYOL model, the cost of a job is transparent. Azure Data Manager for Agriculture offers built-in logging to provide transparency on processing unit (PU) consumption for calls to upstream partner Sentinel Hub. The information appears under the SatelliteLogs category of the standard Azure Data Manager logging.

STAC search for available imagery

Azure Data Manager for Agriculture supports the industry-standard SpatioTemporal Asset Catalogs (STAC) search interface to find metadata on imagery in the Sentinel Hub collection before committing to downloading pixels. To do so, the search endpoint accepts a location in the form of a point, polygon, or multipolygon, plus a start and end date/time. Alternatively, if you already have the unique item ID, you can provide it as an array of up to five to retrieve those specific items directly.

Important

To be consistent with STAC syntax, feature ID is renamed to item ID from the 2023-11-01-preview API version.

If you provide an item ID, any location and time parameters in the request are ignored.

Single-tile source control

Published tiles overlap space on the earth to ensure full spatial coverage. If the queried geometry lies in a space where more than one tile matches for a reasonable time frame, the provider automatically mosaics the returned image with selected pixels from the range of candidate tiles. The provider produces the best resulting image.

In some cases, using more than one tile isn't desirable and traceability to a single tile source is preferred. To support this strict source control, Azure Data Manager for Agriculture supports specifying a single item ID in the ingest job.

Note

This functionality is available only from the 2023-11-01-preview API version.

If the geometry for a provided item ID has partial coverage (for example, the geometry spans more than one tile), the returned images reflect only the pixels that are present in the specified item's tile and result in a partial image.

Reprojection

Important

Reprojection functionality has changed from the 2023-11-01-preview API version, but it's immediately applicable to all versions. Older versions used a static conversion of 10 m * 10 m set at the equator. Imagery ingested before this release might have a difference in size from imagery ingested after this release.

Azure Data Manager for Agriculture uses WGS84 (EPSG: 4326), a flat coordinate system. Sentinel-2 imagery is presented in UTM, a ground projection system that approximates the round earth.

Translating between a flat image and a round earth involves an approximation translation. The accuracy of this translation is set to equal at the equator (10 m^2) and increases in error margin as the point in question moves away from the equator to the poles.

For consistency, Azure Data Manager for Agriculture uses the following formula at 10 m^2 base for all Sentinel-2 calls:

$$ Latitude = \frac{10 m}{111320} $$

$$ Longitude = \frac{10 m}{\frac{111320}{cos(lat)}} $$

$$ \ Where\ lat = The\ centroid's\ latitude\ from\ the\ provided\ geometry $$

Caching

Important

Caching functionality is available only from the 2023-11-01-preview API version. Item caching is applicable only for retrieval that's based on item ID. For a typical geometry and time search, the returned items aren't cached.

Azure Data Manager for Agriculture optimizes performance and costing of highly repeated calls to the same item. It caches recent STAC items retrieved by item ID for five days in the customer's instance and enables local retrieval.

For the first call to the search endpoint, Azure Data Manager for Agriculture brokers the request and triggers a request to the upstream provider to retrieve the matching or intersecting data items. The request incurs any provider fees.

Any subsequent search first directs to the cache for a match. If there's a match, data is served from the cache directly. This process doesn't result in a call to the upstream provider, so it doesn't incur more provider fees. If there's no match, or if the five-day retention period elapses, a subsequent call for the data is passed to the upstream provider. That call is treated as another first call, so the results are cached.

If an ingestion job is for an identical geometry, referenced by the same resource ID, and with overlapping time to an already retrieved scene, Azure Data Manager for Agriculture uses the locally stored image. The image isn't downloaded again from the upstream provider. There's no expiration for this pixel-level caching.

Satellite sources that Azure Data Manager for Agriculture supports

While Azure Data Manager for Agriculture is in preview, it supports ingesting data from the Sentinel-2 constellation.

Sentinel-2

Sentinel-2 is a satellite constellation that the European Space Agency (ESA) launched under the Copernicus mission. This constellation has a pair of satellites and carries a multispectral instrument (MSI) payload that samples 13 spectral bands: four bands at 10 m, six bands at 20 m, and three bands at 60-m spatial resolution.

Sentinel-2 has two products:

• Level 1 data for the top of the atmosphere.
• Level 2 data for the bottom of the atmosphere. This variant is atmospherically corrected.

Azure Data Manager for Agriculture supports ingesting and retrieving Sentinel_2_L2A and Sentinel_2_L1C data from Sentinel 2.

Image names and resolutions

APIs that you use to ingest and read satellite data (for Sentinel-2) in Azure Data Manager for Agriculture support the following image names and resolutions:

Category	Image name	Description	Native resolution
Raw bands	B01	Coastal aerosol	60 m
Raw bands	B02	Blue	10 m
Raw bands	B03	Green	10 m
Raw bands	B04	Red	10 m
Raw bands	B05	Vegetation red edge	20 m
Raw bands	B06	Vegetation red edge	20 m
Raw bands	B07	Vegetation red edge	20 m
Raw bands	B08	Near infrared (NIR)	10 m
Raw bands	B8A	Narrow NIR	20 m
Raw bands	B09	Water vapor	60 m
Raw bands	B11	Short-wave infrared (SWIR)	20 m
Raw bands	B12	SWIR	20 m
Sen2Cor processor output	AOT	Aerosol optical thickness map	10 m
Sen2Cor processor output	SCL	Scene classification data	20 m
Sen2Cor processor output	SNW	Snow probability	20 m
Sen2Cor processor output	CLD	Cloud probability	20 m
Derived indices	NDVI	Normalized difference vegetation index	10 m/20 m/60 m (user defined)
Derived indices	NDWI	Normalized difference water index	10 m/20 m/60 m (user defined)
Derived indices	EVI	Enhanced vegetation index	10 m/20 m/60 m (user defined)
Derived indices	LAI	Leaf area index	10 m/20 m/60 m (user defined)
Derived indices	LAIMask	Leaf area index mask	10 m/20 m/60 m (user defined)
CLP	Cloud probability based on s2cloudless	Values range from 0 (no clouds) to 255 (clouds).	10 m/20 m/60 m (user defined)
CLM	Cloud masks based on s2cloudless	Value of 1 represents clouds, 0 represents no clouds, and 255 represents no data.	10 m/20 m/60 m (user defined)
dataMask	Binary mask to denote availability of data	Value of 0 represents unavailability of data or pixels lying outside the area of interest.	Not applicable, per pixel value

Points to note

Azure Data Manager for Agriculture uses CRS EPSG: 4326 for Sentinel-2 data. The resolutions quoted in the APIs are at the equator.

For the preview:

• A maximum of five satellite jobs can run concurrently, per tenant.
• A satellite job can ingest data for a maximum of one year in a single API call.
• Only TIFs are supported.
• Only 10-m, 20-m, and 60-m images are supported.

Next steps

• Test the Azure Data Manager for Agriculture REST APIs