REST api Microsoft.Consumption/usageDetails gives nextLink with 39 responses and cost totals to not match portal

Question

When I execute REST call

https://management.azure.com/${scope}/providers/Microsoft.Consumption/usageDetails?api-version=${api_version}&startDate=${startDate}&endDate=${endDate}

it responds with json which has tag nextLink
and when my code uses link in nextLink
to make the next REST call and repeat it will tend to stop after 39 such REST calls Why ?
I find it very troubling that even the very last of these 39 results from above call always contains exactly 1000 messages !!!!!!
If this api worked properly the last json result message would most likely contain less that that hard limit of 1000 messages.

and when I programmatically add together all "properties" tags of paygCostInUSD ( also tried payGPrice )
the total cost does not match portal

here is an example json properties message

{
  "id": "/subscriptions/8bogus1/providers/Microsoft.Consumption/usageDetails/fbogus369",
  "kind": "modern",
  "name": "8bogus1",
  "properties": {
    "additionalInfo": "{  \"Provider\": \"3\",  \"ConsumptionBeginTime\": \"2022-04-04T01:00:00\",  \"ConsumptionEndTime\": \"2022-04-04T02:00:00\"}",
    "benefitId": "",
    "benefitName": "",
    "billingAccountId": "fbogusb0",
    "billingAccountName": "",
    "billingCurrencyCode": "USD",
    "billingPeriodEndDate": "0001-01-01T00:00:00Z",
    "billingPeriodStartDate": "0001-01-01T00:00:00Z",
    "billingProfileId": "",
    "billingProfileName": "",
    "chargeType": "Usage",
    "consumedService": "Microsoft.Network",
    "costAllocationRuleName": "",
    "costCenter": "",
    "costInBillingCurrency": 2.04e-05,
    "costInPricingCurrency": 0.0,
    "costInUSD": 0.0,
    "customerName": "",
    "customerTenantId": "",
    "date": "2022-04-04T00:00:00Z",
    "effectivePrice": 0.17,
    "exchangeRate": "1",
    "exchangeRateDate": "2022-04-01T00:00:00Z",
    "exchangeRatePricingToBilling": 0,
    "frequency": "",
    "instanceName": "/subscriptions/86bogus1011/resourceGroups/someAppServices/providers/Microsoft.Network/frontdoors/foo",
    "invoiceId": "",
    "invoiceSectionId": "",
    "invoiceSectionName": "",
    "isAzureCreditEligible": true,
    "meterCategory": "Azure Front Door Service",
    "meterId": "037somebigstring3",
    "meterName": "Data Transfer Out",
    "meterRegion": "Zone 1",
    "meterSubCategory": "Azure Front Door Service",
    "partnerEarnedCreditApplied": "",
    "partnerEarnedCreditRate": 0.0,
    "partnerName": "",
    "partnerTenantId": "",
    "payGPrice": 0.17,
    "paygCostInBillingCurrency": 2.04e-05,
    "paygCostInUSD": 2.04e-05,
    "previousInvoiceId": "",
    "pricingCurrencyCode": "USD",
    "pricingModel": "OnDemand",
    "product": "Azure Front Door Service - Standard - Zone 1",
    "productIdentifier": "Dwhoknowns06",
    "productOrderId": "4aontherbigstringee",
    "productOrderName": "Azure plan",
    "provider": "",
    "publisherId": "",
    "publisherName": "Microsoft",
    "publisherType": "Microsoft",
    "quantity": 0.00012,
    "resellerMpnId": "",
    "resellerName": "",
    "reservationId": "",
    "reservationName": "",
    "resourceGroup": "someAppServices",
    "resourceLocation": "GLOBAL",
    "resourceLocationNormalized": "Unknown",
    "serviceFamily": "Networking",
    "serviceInfo1": "",
    "serviceInfo2": "",
    "servicePeriodEndDate": "2022-05-01T00:00:00Z",
    "servicePeriodStartDate": "2022-04-01T00:00:00Z",
    "subscriptionGuid": "86bogus75c21011",
    "subscriptionName": "Microsoft Azure",
    "term": "",
    "unitOfMeasure": "1 GB",
    "unitPrice": 0.17
  },
  "tags": null,
  "type": "Microsoft.Consumption/usageDetails"
}

so this api has two issues

(1) it mysteriously gives 39 results which indicates something is wrong

(2) repeatedly when using different month periods for start and end dates the total cost when adding up tag paygCostInUSD ( also tried payGPrice ) is way different than cost as show in portal

My goal is to retrieve all consumption items so I can determine at a granular level what resources contribute to my cost per month. Should I be using a different better api ?