Azure Blob Storage on IoT Edge does not synchronize with the cloud Blob Storage
We have a prototype where a local blob container synchronizes with a cloud blob storage.
We now have upload failures between the local blob container and the cloud blob storage.
I tried to reboot in case it was a connectivity issue that led to that behavior, but to no avail.
The network between IoT Edge and cloud goes through cellular, with ~300ms latency (guessed from a ping to google.com).
I remember reading the following question and it hints of non obvious properties to modify the behavior of the local blob:
https://learn.microsoft.com/en-us/answers/questions/183321/azure-blob-storage-on-iot-edge-module-stops-upload.html
in particular:
• Set deviceToCloudUploadProperties:retryIntervalSeconds to a higher number, eg. 20
• Set deviceToCloudUploadProperties:consecutiveErrorsLimit to a higher number, eg. 200
I see the following logs. Any insights ? Should I adapt the tolerance for latency to match the cellular connection?
Is there possibly an error on server side (cont 1 has >4000 files in it, all files are ~40KB) ?
Thanks,
-Sylvain
root@agora-442899012:/# clear; docker logs blobStorage --tail 3000 |grep tarting -A3000
Starting Azure Blob Storage on IoT Edge, version 1.4.1.0.
[2022-09-09 14:34:36.399] [info ] [tid 1] [MetaStore.cc:581] [Initialize] Existing DB version:1, current version:1
[2022-09-09 14:34:36.410] [info ] [tid 31] [main.cc:76] [GCUncommittedBlocks] GCUncommittedBlocks thread starts. UncommittedBlocksExpiryTimeInMinutes:1440
[2022-09-09 14:34:36.410] [info ] [tid 31] [main.cc:88] [GCUncommittedBlocks] GCUncommittedBlocks thread sleeps. UncommittedBlocksExpiryTimeInMinutes:1440
[2022-09-09 14:34:36.669] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Common.EventGridConfiguration: Event Grid endpoint not specified, events turned off.
[2022-09-09 14:34:36.693] [info ] [tid 1] Microsoft.AzureStack.Services.Storage.EntryPoint.BlobService: BlobService - In StartAsync
[2022-09-09 14:34:36.796] [info ] [tid 1] Info: Loading config Param {0} ({1}) read: {2}, p0="NephosAllowPathStyleUris", p1="Allow path-style URIs", p2="true"
[2022-09-09 14:34:36.796] [info ] [tid 1] Info: Successfully loaded {0}: {1}, p0="NephosAllowPathStyleUris", p1="True"
[2022-09-09 14:34:36.797] [info ] [tid 1] Info: Loading config Param {0} ({1}) read: {2}, p0="Nephos.MaskClientIPAddressesInLogs", p1="Indicates whether client IP addresses should be masked in logs.", p2="false"
[2022-09-09 14:34:36.797] [info ] [tid 1] Info: Successfully loaded {0}: {1}, p0="Nephos.MaskClientIPAddressesInLogs", p1="False"
[2022-09-09 14:34:36.797] [info ] [tid 1] Info: Loading config Param {0} ({1}) read: {2}, p0="NephosIncludeInternalDetailsInErrorResponses", p1="Include internal details in error responses", p2="true"
[2022-09-09 14:34:36.797] [info ] [tid 1] Info: Successfully loaded {0}: {1}, p0="NephosIncludeInternalDetailsInErrorResponses", p1="True"
[2022-09-09 14:34:36.799] [info ] [tid 1] Info: Loading config Param {0} ({1}) read: {2}, p0="StampName", p1="Stamp Name", p2="Default Stamp"
[2022-09-09 14:34:37.053] [info ] [tid 1] Microsoft.AzureStack.Services.Storage.EntryPoint.BlobService: BlobService - StartAsync completed
[2022-09-09 14:34:37.060] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobTieringService: Starting service...
[2022-09-09 14:34:37.109] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.186] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.189] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz.2022-09-08T02:18:16.1671702Z
[2022-09-09 14:34:37.261] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz.2022-09-08T02:18:16.1671702Z
[2022-09-09 14:34:37.278] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.278] [info ] [tid 25] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.278] [info ] [tid 25] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz BlobType:block_blob ContentLength:36441 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.291] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.291] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.291] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.291] [info ] [tid 13] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz BlockListType:1
[2022-09-09 14:34:37.292] [info ] [tid 13] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_18_15__DATA_300_sec_id3.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:36441 ModificationTime:133070770961671702
[2022-09-09 14:34:37.292] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz.2022-09-08T02:18:16.4118465Z
[2022-09-09 14:34:37.292] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz.2022-09-08T02:18:16.4118465Z
[2022-09-09 14:34:37.292] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.293] [info ] [tid 25] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.293] [info ] [tid 25] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz BlobType:block_blob ContentLength:35286 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.296] [info ] [tid 15] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz BlockListType:1
[2022-09-09 14:34:37.297] [info ] [tid 15] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:35286 ModificationTime:133070770964118465
[2022-09-09 14:34:37.306] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.306] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.306] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.306] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.307] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.307] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz.2022-09-08T02:23:16.7857164Z
[2022-09-09 14:34:37.307] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz.2022-09-08T02:23:16.7857164Z
[2022-09-09 14:34:37.307] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.308] [info ] [tid 41] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.308] [info ] [tid 41] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz BlobType:block_blob ContentLength:37006 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.308] [info ] [tid 40] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz BlockListType:1
[2022-09-09 14:34:37.308] [info ] [tid 40] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:37006 ModificationTime:133070773967857164
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.321] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.321] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz.2022-09-08T02:23:17.0283781Z
[2022-09-09 14:34:37.322] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz.2022-09-08T02:23:17.0283781Z
[2022-09-09 14:34:37.322] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.322] [info ] [tid 25] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.322] [info ] [tid 25] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz BlobType:block_blob ContentLength:35349 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.326] [info ] [tid 41] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz BlockListType:1
[2022-09-09 14:34:37.326] [info ] [tid 41] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:35349 ModificationTime:133070773970283781
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.335] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.335] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz.2022-09-08T02:28:17.4077731Z
[2022-09-09 14:34:37.336] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz.2022-09-08T02:28:17.4077731Z
[2022-09-09 14:34:37.336] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.338] [info ] [tid 42] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.338] [info ] [tid 42] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz BlobType:block_blob ContentLength:36594 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.343] [info ] [tid 25] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz BlockListType:1
[2022-09-09 14:34:37.343] [info ] [tid 25] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:36594 ModificationTime:133070776974077731
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.349] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.350] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070779980222485.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.350] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz.2022-09-08T02:28:17.6504745Z
[2022-09-09 14:34:37.350] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz.2022-09-08T02:28:17.6504745Z
[2022-09-09 14:34:37.350] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.350] [info ] [tid 43] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.351] [info ] [tid 43] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz BlobType:block_blob ContentLength:35299 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.352] [info ] [tid 42] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz BlockListType:1
[2022-09-09 14:34:37.352] [info ] [tid 42] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:35299 ModificationTime:133070776976504745
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.363] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070779980222485.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070779980222485.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070779982708927.402-442899012/2/raw/2022_09_08__02_33_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.364] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz.2022-09-08T02:33:18.0222485Z
[2022-09-09 14:34:37.364] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploading cont1.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz.2022-09-08T02:33:18.0222485Z
[2022-09-09 14:34:37.364] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:null MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.364] [info ] [tid 45] [BlobInterface.cc:255] [GetBlobProperties] GetBlobProperties received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.365] [info ] [tid 45] [BlobInterface.cc:283] [GetBlobProperties] GetBlobProperties completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz BlobType:block_blob ContentLength:36748 SvcMetadataSize:36 AppMetadataSize:0 NumCommittedBlocks:1
[2022-09-09 14:34:37.365] [info ] [tid 44] [BlobInterface.cc:1029] [GetBlockList] GetBlockList received. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz BlockListType:1
[2022-09-09 14:34:37.365] [info ] [tid 44] [BlobInterface.cc:1119] [GetBlockList] GetBlockList completed. Container:cont1 Blob:402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz BlockIdSize:0 UncommitedBlocksCount:0 CommitedBlocksCount:0 ContentLength:36748 ModificationTime:133070779980222485
[2022-09-09 14:34:37.379] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.379] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070770964118465.402-442899012/2/raw/2022_09_08__02_18_15__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773967857164.402-442899012/3/raw/2022_09_08__02_23_16__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070773970283781.402-442899012/2/raw/2022_09_08__02_23_16__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776974077731.402-442899012/3/raw/2022_09_08__02_28_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070776976504745.402-442899012/2/raw/2022_09_08__02_28_17__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070779980222485.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070779980222485.402-442899012/3/raw/2022_09_08__02_33_17__DATA_300_sec_id3.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070779982708927.402-442899012/2/raw/2022_09_08__02_33_17__DATA_300_sec_id2.csv.gz
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1494] [ListBlobsInOrder] ListBlobsInOrder received. Container:cont1 BlobNameStart:cont1.00133070779982708927.402-442899012/2/raw/2022_09_08__02_33_17__DATA_300_sec_id2.csv.gz MaxBlobNames:1 OrderType:0 Flags:1
[2022-09-09 14:34:37.380] [info ] [tid 1] [BlobInterface.cc:1528] [ListBlobsInOrder] ListBlobsInOrder completed. Container:cont1 BlobEntryCount:1 NextBlobName:cont1.00133070782986295469.402-442899012/3/raw/2022_09_08__02_38_18__DATA_300_sec_id3.csv.gz
[2022-09-09 14:34:37.381] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.TieringOrchestrator: Found cont1.402-442899012/2/raw/2022_09_08__02_33_17__DATA_300_sec_id2.csv.gz.2022-09-08T02:33:18.2708927Z
[2022-09-09 14:34:37.381] [info ] [tid 1] Microsoft.Azure.Devices.BlobStorage.Tiering.BlobUpload: Uploadi