[Azure-Table-Storage-Access-RestAPI-via-PowerShell] - Invoke-RestMethod : Bytes to be written to the stream exceed the Content-Length bytes size specified

Question

Target: Fetch a list of installed applications from multiple computers to Azure Table Storage using Invoke-RestMethod.

I am trying to collect information from the computer and put it in a batch lets say 100 rows and then insert it into the table.

Prepared the script based on the https://learn.microsoft.com/en-us/rest/api/storageservices/performing-entity-group-transactions this article.

Error: "Invoke-RestMethod: Bytes to be written to the stream exceed the Content-Length bytes size specified"

Here is the code below

$tableEndpoint = 'https://****.table.core.windows.net/$batch'  
  
  
$tableName = 'InstalledApplications'  
$StorageAccount = "********"  
  
$URI = $tableEndpoint + $tableName + $SAS  
  
$cot =@()  
  
  
$array1 = Get-ItemProperty HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\* | Select-Object DisplayName, DisplayVersion, Publisher, InstallDate -unique  
  
foreach($arr in $array1){  
  
  
$RequestBody = ConvertTo-Json -InputObject @{  
        "TagetName"= $arr.DisplayName;  
        "Message"= $arr.DisplayVersion;  
        "ComputerName"= $ENV:ComputerName;  
        "Username"= $ENV:Username;  
        "BatchName" = "_a1e9d677-b28b-435e-a89e-87e6a768a431";  
        "Content-Type" = "multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977"  
        "PartitionKey"= "$ENV:ComputerName";  
        "RowKey"= $arr.DisplayName  
        "TableName" = "InstalledApplication"  
    }  
    $cot += $RequestBody  
}  
  
$batching = $cot[1..10]  
  
$RequestHeaders = @{  
    "x-ms-date"=(Get-Date -Format r);  
    "x-ms-version"="2009-09-19";  
    "Accept-Charset"="UTF-8";  
    "DataServiceVersion"="3.0;NetFx";  
    "MaxDataServiceVersion"="3.0;NetFx";  
    "Content-Type" = "multipart/mixed; boundary=batch_a1e9d677-b28b-435e-a89e-87e6a768a431"  
    "Content-Length" =  "1323"  
    "Authorization"  = "SharedKeyLite " + $StorageAccount + ":" + $SAS  
}  
  
Invoke-RestMethod -Method POST -Uri $tableEndpoint -Headers $RequestHeaders -Body $cot -ContentType "multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977"  

Please let me know why content length exceeds although the returned rows are no more than 60.

Answer 1

Hello @SujithkumarGSuriyamurthy-7096 ,
Thanks for your query ! I just tried your previous version of the script and was able to insert the required data to the azure table storage.
I was able to repro your issue on your previous version by explicitly mentioning the "Content-Length" to some x value in the headers.

• What is the reason you want to explicitly mention the content length ?
• Did you try removing that property ?
• If the Content-Length is not hardcoded , how are you calculating the content Length dynamically?

Basically , content length implicitly also includes the type of encoding which we use . For example , if the actual data is 5 bytes - then over all content-length can be more than 5 bytes depending upon the encoding used. So while the data is being pushed using REST API when it goes through transit - it implicitly uses some sort of encoding - you might want to think over about that.

ASCIIEncoding encoding = new ASCIIEncoding ();
byte[] samplebyte= encoding.GetBytes (sampledata);

For example: For the below string , if you try to get a normal length - it will just output 1 . But if you try to get the length through some encoding in terms of bytes those will return different outputs (2 & 1)

var s = "é"
var s = "e";

Kindly let us know if you need additional help.

Answer 2

Thanks for your response to my query!

the previous version works well for me still the problem is when you run the PS command to get a list of applications it returns 90 rows, what happens is, it calls the API 90 times since the invoke-rest method resides inside the for loop. let's suppose if I use the same script on 8000 computers then you can imagine what will be the transactional cost at the end. correct me if I am wrong here, as per my understanding 1 transaction = $0.0004.

This is the reason why I am trying to use batch insert operation.

Initially, I was encoding the data and copy it into a variable, and pass the same variable in the header where it says "content-length = $variable. length". this way was also giving me the same error.

As per your suggestion, today, I tried removing the "content-length" still no luck.

therefore, please assist me if there any misconstruction in the above script.

Answer 3

As per this article https://learn.microsoft.com/en-us/rest/api/storageservices/performing-entity-group-transactions#table-service-support-for-odata-batch-requests,

I tried to insert a batch with two entities using postman but I get an error "one of the request inputs is not valid". please check out the body below I used for testing.

--batch_f351702c-c8c8-48c6-af2c-91b809c651c4    
Content-Type: multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977    
    
--changeset_8a28b620-b4bb-458c-a177-0959fb14c977    
Content-Type: application/http    
Content-Transfer-Encoding: binary    
    
POST https://*****.table.core.windows.net/TableName HTTP/1.1  
Content-Type: application/json    
Accept: application/json;odata=minimalmetadata    
Prefer: return-no-content    
DataServiceVersion: 3.0;    
    
{"PartitionKey":"Channel_19", "RowKey":"1", "Rating":9, "Text":".NET..."}    
--changeset_8a28b620-b4bb-458c-a177-0959fb14c977    
Content-Type: application/http    
Content-Transfer-Encoding: binary    
    
POST https://*****.table.core.windows.net/TableName HTTP/1.1  
Content-Type: application/json    
Accept: application/json;odata=minimalmetadata    
Prefer: return-no-content    
DataServiceVersion: 3.0;    
    
{"PartitionKey":"Channel_19", "RowKey":"2", "Rating":9, "Text":"Azure..."}    
--changeset_8a28b620-b4bb-458c-a177-0959fb14c977    
Content-Type: application/http    
Content-Transfer-Encoding: binary    
  
--changeset_8a28b620-b4bb-458c-a177-0959fb14c977--    
--batch_f351702c-c8c8-48c6-af2c-91b809c651c4    

please assist me with the above JSON body construct.