How to write a weather extension

מאמר
03/14/2023

In this section, you see a step-by-step guide to write your own weather extension.

What is a weather extension

Weather extension in Data Manager for Agriculture is a manifest (JSON) file providing the complete details on the APIs and a template for each APIs response (output). Therefore, extension file is essentially an API template structure as defined by Data Manager for Agriculture, for it to understand the API input (request) and output (response) characteristics.

Weather extension structure

At a high-level the extension file is a JSON consisting of two things:

Provider metadata (JSON Object)
API Information (JSON Array)

Provider metadata

It's a json object providing the details on the below fields that are necessary to uniquely identify an extension and its versioning information. The details provided in this section of the extension are shown to external customers in Data Manager for Agriculture marketplace. Therefore extensionId & extensionName (for easy identification) and description (for value-proposition) needs to be customer focused.

Sample provider metadata

"provider": {
    "extensionId": "abc.weather", 
    "extensionName": "ABC weather",
    "description": "Get Weather data from ABC weather into Azure Data Manager for Agriculture platform using this extension",
    "dataCategory": "Weather",
    "farmBeatsSchemaVersion": "1.0",
    "extensionVersion": "1.0",
    "supportUrl": "www.abc.com/support",
    "supportEmail": "support@abc.com"
  }

Provider metadata details

Name	Type	Description
extensionId	string	The ID provided in the fashion of organization name (Contoso) and service (weather) Ex: `org.service`. extensionId is the unique identifier of the extension and the one which users are using on the Data Manager for Agriculture platform to interact with the extension APIs
extensionName	string	Name of the extension as to be used in Data Manager for Agriculture extension marketplace.
description	string	Description stating the capabilities and services offered by the extension.
dataCategory	string	For weather extensions, use `weather`.
farmBeatsSchemaVersion	string	The version of the manifest file on the Data Manager for Agriculture side. Any updates to the existing manifest file lead to a new version update to this field.
extensionVersion	string	The version of extension file. Starting with `1.0`. Updates to your extension file increment this version number according to the convention of major & minor updates.
supportUrl	string	Website link to raise support queries & FAQs
supportEmail	string	Email to send in the support queries.

API information

The API Information JSON array (apiInfos) can be further broken into the following structural elements.

API metadata
Authentication parameters
API input parameters
Extracted API input parameters
Functional parameters
Units system
Platform & Custom parameters
Platform & Custom template

API metadata

This section consists of basic information regarding the API used by Data Manager for Agriculture to identify the apiName (called by users explicitly) and redirect the API request to the right endpoint based on the appropriate requestType.

Sample API metadata

"apiInfos": [
    {
          "apiName": "dailyforecast",
          "description": "The Daily Forecast API",
          "endpoint": "https://ag.us.clearapis.com/v1.1/forecast/daily",
          "requestType": "GET",
          "isLoadAPI": "false",
          "typeOfData": "forecast",
          "granularity": "daily",
          "defaultUnitSystem": "us"
    }
]

API metadata details

Name	Type	Description
apiInfos	array	The JSON array of objects, where each API is an object within `apiInfos` array.
apiName	string	The API Name as supported by the extension, it's the exact name using which the users would be calling into the extension APIs. Kindly follow the same naming convention as mentioned in your API documentation.
description	string	API description
endpoint	string	API endpoint for Data Manager for Agriculture to call into the `apiName`.
requestType	string	`GET` or `POST` or `PUT` request type as supported by the `apiName`.
isLoadAPI	boolean	If the `apiName` is a pass-through API like current weather data, make this key as `false`. For all the load APIs (historical & forecast), keep this field as `true`. When the `isLoadAPI` key is `false`, the API response would be directly sent to the user and wouldn't be stored in the Data Manager for Agriculture storage service.
typeOfData	string	Currently supported values are `Historical` and `Forecast`.
granularity	string	Currently supported values are `Daily` and `Hourly`.
defaultUnitSystem	string	Provide the name of the default units system supported by the `apiName`.

Authentication parameters

This section takes in the authentication related parameters as supported by the apiName. As Data Manager for Agriculture supports two types of auth-related keys (x-ms-farmBeats-data-provider-id & x-ms-farmBeats-data-provider-key) in the API header section, the extension file needs to explicitly provide the key name of its respective authentication keys as required by the apiName.

As Data Manager for Agriculture collects the authentication information via the API header (in the Create Weather Job API). Authentication parameter mapping is done to ensure that Data Manager for Agriculture can pass the key accordingly to the extension as required.

Sample authentication parameters

"apiInfos": [
    "authInputParameters": [
        {
          "farmBeatsAuthMapping": "x-ms-farmBeats-data-provider-id",
          "name": "app_id",
          "isRequired": "true",
          "providerDataType": "string",
          "description": "Provide the APP ID, username etc. that your API supports",
          "location": "apiQuery"
        },
        {
          "farmBeatsAuthMapping": "x-ms-farmBeats-data-provider-key",
          "name": "app_key",
          "isRequired": "true",
          "providerDataType": "string",
          "description": "Provide the API Key or password etc. that your API supports",
          "location": "apiQuery"
        }
      ]
]

Authentication parameters details

Name	Type	Description
authInputParameters	array	JSON array of authentication parameters, where each object signifies a type of authentication supported. Use the key based on the authentication type supported by your extension.
farmBeatsAuthMapping	string	Currently two types of authentication related keys are supported. For API Key based authentication, use only `x-ms-farmBeats-data-provider-key` object, whereas for APP ID and APP Key based authentication use both `x-ms-farmBeats-data-provider-id` &`x-ms-farmBeats-data-provider-key` objects.
name	string	Name of the authentication key as supported by the `apiName`.
isRequired	boolean	Is this name a required parameter to the apiName? Provide true or false values.
providerDataType	string	Provide the datatype of the `name` parameter.
description	string	Data Manager for Agriculture description of what each of the `farmBeatsAuthMapping` means within each object.
location	string	Where in the API should the `name` parameter be sent. Currently supported values are `apiQuery` & `apiHeader`.

API input parameters

This section provides the details about the API signature (input parameters) to successfully call into the apiName.

Sample API input parameters

"apiInfos": [
    "apiInputParameters": [
        {
          "name": "start",
          "isRequired": "true",
          "providerDataType": "double",
          "description": "Start of time range. Valid start values range from zero to nine. Day zero represents the current day.",
          "location": "apiQuery"
        },
        {
          "name": "end",
          "isRequired": "true",
          "providerDataType": "double",
          "description": "End of time range. Valid end values range from zero to nine.",
          "location": "apiQuery"
        },
        {
          "name": "location",
          "isRequired": "true",
          "providerDataType": "string",
          "description": "User-provided latitude and longitude coordinates. Formatted as location=[(<lat_1>,<lon_1>)].",
          "location": "apiQuery"
        },
        {
          "name": "unitcode",
          "isRequired": "false",
          "providerDataType": "string",
          "description": "Unit conversion set to be used. Default is us-std. Valid values are us-std, si-std.",
          "location": "apiQuery"
        },
      ]
]

API input parameters details

Name	Type	Description
apiInputParameters	array	JSON array of API input parameters, where each object signifies an input parameter supported by the `apiName`.
name	string	Name of the input parameter as supported by the `apiName`.
isRequired	boolean	Is this name a required parameter to the apiName? Provide true or false values.
providerDataType	string	Provide the datatype of the `name` parameter.
description	string	Provide a description of what `name` parameter means.
location	string	Where in the API should the `name` parameter be sent. Currently supported values are `apiQuery` & `apiHeader`.

Extracted API input parameters

This section is for Data Manager to extract input parameters passed in the API request for computation and storage. In this example Data Manager for Agriculture would be extracting the location information (latitude and longitude) from the API Input request and store them as part of each weather output in Data Manager for Agriculture.

Hence the extension needs to provide a HandleBars template on how to extract location information. The below example suggests that the extension API collects location information as "lat" for "latitude" and "lon" for "longitude".

Sample extracted API input parameters

"extractedApiInputParameters": [
        {
          "name": "location",
          "template": "{ \"latitude\": \"{{lat}}\", \"longitude\": \"{{lon}}\"  } "
        }
      ]

Extracted API input parameters details

Name	Type	Description
extractedApiInputParameters	array	JSON array of extraction functionalities, where each object signifies what information needs to be extracted. Currently `location` is one such extraction.
name	string	Name of the extraction, currently the supported value is `location` .
template	string	HandleBars template depicting how is latitude and longitude information collected in the API input parameters.

Functional parameters

This section is dedicated for the functionalities/capabilities built by Data Manager for Agriculture. For weather extension, centroid calculation is one such functionality.

When users don't provide the latitude/longitude coordinates, Data Manager for Agriculture uses the primary geometry of the field (ID passed by user) to compute the centroid. The computed centroid coordinates are passed as the latitude and longitude to the extension (data provider). Hence for Data Manager for Agriculture to be able to understand the usage of location coordinates the functional parameters section is used.

For Data Manager for Agriculture to understand the usage of latitude and longitude in the apiName input parameters, the extension is expected to provide the name of key used for collecting location information followed by a handlebar template to imply how the latitude and longitude values need to be passed.

Sample functional parameters

"apiInfos": [
    "functionalParameters": [
        {
          "name": "CentroidCalculation",
          "description": "Provide names of the parameters used to collect latitude and longitude information",
          "functionalParameterEntities": [
            {
              "name": "lat",
              "template": "{ \"lat\": \"{{latitude}}\" } "
            },
            {
              "name": "lon",
              "template": "{ \"lon\": \"{{longitude}}\" } "
            }
          ]
        }
      ],
]

Functional parameters details

Name	Type	Description
functionalParameters	array	JSON array of functionalities, where each object signifies a functionality supported by Data Manager for Agriculture. Currently `CentroidCalculation` is one such functionality.
name	string	Name of the functionality, currently the supported value is `CentroidCalculation` .
description	string	Data Manager for Agriculture description of functionality.
functionalParameterEntities	array	JSON array of objects, where each object is specific to the latitude & longitude.

Units system

This section is used by Data Manager for Agriculture to understand the various types of unit systems supported by the extension. Hence the extension needs to provide the key name used for collecting units information in the API inputs and followed by the various units system names (Ex: us-std) as supported by the apiName.

Sample units system

"unitSystems": 
    {
        "key": "unitcode",
        "values": [
            "us-std",
            "si-std",
            "us-std-precise",
            "si-std-precise"
            ]
    }

Units system details

Name	Type	Description
unitSystems	object	JSON object to collect the unit system information.
key	string	Name of the parameter used to collect the units information in the API input.
values	string	List of units system names as supported by the extension.

Platform & custom parameters

In each weather API response, the weather measures, which are sent as part of the output (ex: temperature, dewpoint etc.) are called as parameters.

Hence, when it comes to parameters, Data Manager for Agriculture internally supports the following set of parameters and treats them as Platform parameters.

cloudCover
dateTime
dewPoint
growingDegreeDay
precipitation
pressure
relativeHumidity
soilMoisture
soilTemperature
temperature
visibility
wetBulbTemperature
windChill
windDirection
windGust
windSpeed

Therefore, any extension sending weather parameters, which don't fall under the platform parameters, is sending them as part of Custom parameters. The key difference between platform & customer parameters is that, the users using Data Manager for Agriculture weather APIs are able to query and filter on the platform parameters (Ex: temperature > 30) and not on custom parameters. However, custom parameters are sent as part of the weather query output.

In this section, the extension provides the units information for each of the parameters for every units system that is supported. Using this Data Manager for Agriculture knows what is the underlying measurement unit for each weather parameter based on the information provided in this section of the extension.

Note

For a particular parameter if units are not applicable then do not mention the units for those alone (Ex: weatherDescriptor)
For a particular parameter if the units are same for all the units system then mention the same in all units system. (Ex: cloudCover)

Sample platform & custom parameters

"apiInfos": [
     "platformParameters": [
        {
          "farmBeatsName": "cloudCover",
          "farmBeatsDataType": "double",
          "description": "The average percentage of sky covered by clouds.",
          "measurementUnits": [
            {
              "unitSystem": "us-std",
              "unit": "%"
            },
            {
              "unitSystem": "us-std-precise",
              "unit": "%"
            },
            {
              "unitSystem": "si-std",
              "unit": "%"
            },
            {
              "unitSystem": "si-std-precise",
              "unit": "%"
            }
          ]
        },
        {
          "farmBeatsName": "dewPoint",
          "farmBeatsDataType": "double",
          "description": "The air temperature at which the air will become saturated, and dew moisture will condense into fog (or dew).",
          "measurementUnits": [
            {
              "unitSystem": "us-std",
              "unit": "F"
            },
            {
              "unitSystem": "us-std-precise",
              "unit": "F"
            },
            {
              "unitSystem": "si-std",
              "unit": "C"
            },
            {
              "unitSystem": "si-std-precise",
              "unit": "C"
            }
          ]
        },
    "customParameters": [
        {
          "providerName": "weatherDescriptor",
          "providerDataType": "string",
          "description": "General weather descriptor data"
        },
        {
          "providerName": "airTempMax",
          "providerDataType": "double",
          "description": "Maximum daily air temperature at two meters above ground level.",
          "measurementUnits": [
            {
              "unitSystem": "us-std",
              "unit": "F"
            },
            {
              "unitSystem": "us-std-precise",
              "unit": "F"
            },
            {
              "unitSystem": "si-std",
              "unit": "C"
            },
            {
              "unitSystem": "si-std-precise",
              "unit": "C"
            }
          ]
        },
]

Platform parameters details

Name	Type	Description
platformParameters	array	JSON array of platform parameters where each object is one platform parameter.
farmBeatsName	string	Name of the parameter as provided by Data Manager for Agriculture.
farmBeatsDataType	string	Data type of the parameter as provided by Data Manager for Agriculture.
description	string	Description of the parameter as provided by Data Manager for Agriculture.
measurementUnits	string	JSON array of units for each of the unit system `values` supported by the extension.
unitSystem	string	Unit system `value` as supported by the extension.
unit	string	Unit of measurement for the specific weather parameter Ex: `F` for dewPoint.

Custom parameters details

Name	Type	Description
customParameters	object	JSON array of custom parameters where each object is one custom parameter.
providerName	string	Name of the parameter as provided by the extension.
providerDataType	string	Data type of the parameter as provided by extension.
description	string	Description of the parameter as provided by extension.
measurementUnits	string	JSON array of units for each of the unit system `values` supported by the extension.
unitSystem	string	Unit system `value` as supported by the extension.
unit	string	Unit of measurement for the specific weather parameter Ex: `F` for airTempMax.

Platform & custom template

Template is the mapping information provided by the extension to convert the extension API output (JSON response) to the format which Data Manager for Agriculture expects. Using this different API output formats can now be uniformly mapped/converted to one single format.

Template solution is found to be one of the most effective ways to parse the JSON output provided by the extension. In the case of weather extension, Data Manager for Agriculture expects the extension to be written using HandleBars template. HandleBars is an open source templating language with simple to use expressions.

Note

It is highly recommended to try-out the HandleBars template with the examples provided and learn how to make use of the helper functions to build your own parsing logic if it is not already provided by Data Manager for Agriculture.

On a high-level this is how templates work, by taking the API response as the input and generating the output in the format expected by Data Manager for Agriculture.

As shown in the above figure, validate your template against the respective API response and use the validated template in the extension. Below is an example of an API response and its respective platform and custom template.

Sample API response

{
    "47,-97": {
        "2016-12-15": {
            "overall": {
                "air_temp_avg": {
                    "unit": "F",
                    "value": -3.0
                },
                "air_temp_max": {
                    "unit": "F",
                    "value": 5.0
                },
                "air_temp_min": {
                    "unit": "F",
                    "value": -11.0
                },
                "cloud_cover_avg": {
                    "unit": "%",
                    "value": 26.4
                },
                "descriptors": {
                    "cloud_cover_descriptor": {
                        "code": 21107,
                        "icon": "https://.../cover_partlycloudyday.png",
                        "text": "Increasing Clouds"
                    },
                    "precipitation_descriptor": {
                        "code": 61113,
                        "icon": "https://.../precip_flurries.png",
                        "text": "Slight Chance of Flurries"
                    },
                    "weather_descriptor": {
                        "code": 61113,
                        "icon": "https://.../precip_flurries.png",
                        "text": "Slight Chance of Flurries"
                    },
                    "wind_direction_descriptor": {
                        "code": 51600,
                        "icon": "https://.../direction_sw.png",
                        "text": "Southwest"
                    },
                    "wind_trend_descriptor": {
                        "code": 10500,
                        "icon": "https://.../error_none.png",
                        "text": "None"
                    }
                },
                "dew_point_avg": {
                    "unit": "F",
                    "value": -11.0
                },
                "dew_point_max": {
                    "unit": "F",
                    "value": -4.0
                },
                "dew_point_min": {
                    "unit": "F",
                    "value": -19.0
                },
                "ice_acc_period": {
                    "unit": "in",
                    "value": 0.0
                },
                "liquid_acc_period": {
                    "unit": "in",
                    "value": 0.0
                },
                "long_wave_radiation_avg": {
                    "unit": "W/m^2",
                    "value": 170.0
                },
                "pet_period": {
                    "unit": "in",
                    "value": 0.009
                },
                "precip_acc_period": {
                    "unit": "in",
                    "value": 0.001
                },
                "precip_prob": {
                    "unit": "%",
                    "value": 10.0
                },
                "relative_humidity_avg": {
                    "unit": "%",
                    "value": 68.0
                },
                "relative_humidity_max": {
                    "unit": "%",
                    "value": 77.0
                },
                "relative_humidity_min": {
                    "unit": "%",
                    "value": 61.0
                },
                "short_wave_radiation_avg": {
                    "unit": "W/m^2",
                    "value": 70.0
                },
                "snow_acc_period": {
                    "unit": "in",
                    "value": 0.02
                },
                "sunshine_duration": {
                    "unit": "hours",
                    "value": 6
                },
                "wind_gust_max": {
                    "unit": "n/a",
                    "value": "n/a"
                },
                "wind_speed_2m_avg": {
                    "unit": "mph",
                    "value": 6.0
                },
                "wind_speed_2m_max": {
                    "unit": "mph",
                    "value": 8.0
                },
                "wind_speed_2m_min": {
                    "unit": "mph",
                    "value": 2.0
                },
                "wind_speed_avg": {
                    "unit": "mph",
                    "value": 8.0
                },
                "wind_speed_max": {
                    "unit": "mph",
                    "value": 11.0
                },
                "wind_speed_min": {
                    "unit": "mph",
                    "value": 3.0
                }
            }
        }
    }
}

Sample Platform & Custom template for the above API response

{
"apiInfos": {
    "platformTemplate": "{ {{#each .}}\"value\": [{{#each .}} {\"cloudCover\": \"{{overall.cloud_cover_avg.value}}\" ,\"dewPoint\": \"{{overall.dew_point_avg.value}}\" ,\"precipitation\": \"{{overall.precip_acc_period.value}}\" ,\"relativeHumidity\": \"{{overall.relative_humidity_avg.value}}\" ,\"dateTime\": \"{{convertDateInYYYYMMDDToDateTime @key}}\",      \"temperature\": \"{{overall.air_temp_avg.value}}\" ,\"windSpeed\": \"{{overall.wind_speed_avg.value}}\" ,   },{{/each}}]{{/each}} }",
    "customTemplate": "{ {{#each .}}\"value\": [{{#each .}} {\"air_temp_max\": \"{{overall.air_temp_max.value}}\",\"air_temp_min\": \"{{overall.air_temp_min.value}}\",\"cloudCoverDescriptor\": \"{{overall.descriptors.cloud_cover_descriptor.text}}\",\"precipitationDescriptor\": \"{{overall.descriptors.precipitation_descriptor.text}}\",\"weatherDescriptor\": \"{{overall.descriptors.weather_descriptor.text}}\",\"windDirectionDescriptor\": \"{{overall.descriptors.wind_direction_descriptor.text}}\",\"windTrendDescriptor\": \"{{overall.descriptors.wind_trend_descriptor.text}}\",\"dewPointMax\": \"{{overall.dew_point_max.value}}\",\"dewPointMin\": \"{{overall.dew_point_min.value}}\",\"iceAccPeriod\": \"{{overall.ice_acc_period.value}}\",\"liquidAccPeriod\": \"{{overall.liquid_acc_period.value}}\",\"longWaveRadiationAvg\": \"{{overall.long_wave_radiation_avg.value}}\",\"petPeriod\": \"{{overall.pet_period.value}}\",\"precipProb\": \"{{overall.precip_prob.value}}\",\"relativeHumidityMax\": \"{{overall.relative_humidity_max.value}}\",\"relativeHumidityMin\": \"{{overall.relative_humidity_min.value}}\",\"shortWaveRadiationAvg\": \"{{overall.short_wave_radiation_avg.value}}\",\"snowAccPeriod\": \"{{overall.snow_acc_period.value}}\",\"sunshineDuration\": \"{{overall.sunshine_duration.value}}\",\"windSpeed2mAvg\": \"{{overall.wind_speed_2m_avg.value}}\",\"windSpeed2mMax\": \"{{overall.wind_speed_2m_max.value}}\",\"windSpeed2mMin\": \"{{overall.wind_speed_2m_min.value}}\",\"windSpeedMax\": \"{{overall.wind_speed_max.value}}\",\"windSpeedMin\": \"{{overall.wind_speed_min.value}}\",},{{/each}}]{{/each}} }"
}
}

Note

The template generated from the HandleBars is stringified by adding \"<text>\" to make it compatible with the JSON format.

Helper functions

Helper functions are used by the templates to perform specific transformation on the data, which isn't supported natively. Here are helper functions supported by Data Manager for Agriculture.

Sample helper functions

Handlebars.registerHelper('splitAndTake', function(title, char, index) {
  var t = title.split(char);
  return t[index];
});
Handlebars.registerHelper('converttime', function(timestamp) {
    return new Date(timestamp);
});
Handlebars.registerHelper('convertunixtime', function(unix_timestamp) {
    return new Date(unix_timestamp * 1000);
});

What action these helper functions do?

SplitAndTake(object valueObject, string separator, int index) - This function is used to split a string (Ex: "47,-97") based on a separator (Ex: ",") and takes the element at given index (Ex: index=0 gives "47")
ConvertDateInFormatToDateTime(object dateObject, string format) - This function is used to parse a date in given format (Ex: 2016-12-15) to DateTime string.
ConvertUnixTimeToDateTime(object unixTimeStamp) - This function is used to convert unix timestamp (Ex: 1392267600) to datetime string.
GetObjectFromListWithKeyValuePair(Array listOfObjects, string key, string value) - Given a list of objects it fetches the object based on key (type) value (RAIN) pair. In the below example, to pick the precipitation of "type": "RAIN" this function is used.

{
"precipitation": [
      {
        "type": "SNOW",
        "amount": 0
      },
      {
        "type": "RAIN",
        "amount": 0.01
      }
    ]
}

GetValueFromObject(string jsonString, string key) - Given a json object as string, it gets the value based on key.

Note

If the extension you are writing requires additional helper functions to parse the API response then reach out to us by creating a support ticket.

שתף באמצעות

How to write a weather extension

What is a weather extension

Weather extension structure

Provider metadata

Sample provider metadata

Provider metadata details

API information

API metadata

Sample API metadata

API metadata details

Authentication parameters

Sample authentication parameters

Authentication parameters details

API input parameters

Sample API input parameters

API input parameters details

Extracted API input parameters

Sample extracted API input parameters

Extracted API input parameters details

Functional parameters

Sample functional parameters

Functional parameters details

Units system

Sample units system

Units system details

Platform & custom parameters

Sample platform & custom parameters

Platform parameters details

Custom parameters details

Platform & custom template

Sample API response

Sample Platform & Custom template for the above API response

Helper functions

Sample helper functions

משוב

משאבים נוספים