Defender: Example APP

Question

Hi,

I'm reading 'Windows Defender for Azure Storage' documentation and the application (Employee TAX Receipt Submission Portal) seems to be very useful. Here is the URL used in that documentation: https://prodenvstoracc01.z28.web.core.windows.net/

Just two questions:

• Does anybody know if this app is available on github (or any repo)? - I've searched and I didn't find it.
• What are the SLA for Defender for blob storage scanning a file? Example: a .zip with 1,5GB and 400.000 compressed files should be scanned in XX minutes.

Thanks in advance and best regards,

Renan Nascimento

Answer 1

Hello Renan Nascimento,

Greetings! Welcome to Microsoft Q&A Platform.

As of today, Defender for Storage’s malware scanner indeed only scans files upon upload in Azure Blob Storage. It ensures that any content uploaded to your storage accounts is thoroughly inspected for malicious elements before being stored. This proactive approach helps maintain the security and integrity of your data. However, there are some limitations which are not supported by Malware Scanning mentioned https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-storage-malware-scan#limitations where blobs that are larger than the 2GB limit will not be scanned. In the file meta data, you will find the scan has been aborted with this message “SAM259206: Scan aborted - the requested blob exceeded the maximum allowed size of 2 GB. Correlation Id: xxx”

If you would like to scan large files that are around 100GB and hence below are some considerations that you can try to do the same,

1.Consider splitting them into smaller chunks before uploading. This way, each chunk remains within the size limit and can be scanned individually using the the Azure Blob Storage REST API's "Put Block" and "Put Block List" operations to upload large files in chunks. Here are the high-level steps you can follow:

1. Divide the large file into smaller chunks.
2. Create a new blob in the Azure Blob Storage account using the "Put Blob" operation. This operation creates a new blob or updates an existing blob with the specified content.
3. Upload each chunk of the file using the "Put Block" operation. This operation uploads a block of data to the specified block blob.
4. Commit the blocks to the blob using the "Put Block List" operation. This operation commits the list of blocks that have been uploaded to the specified block blob.

There are few similar SO threads discussion here: https://stackoverflow.com/questions/61481720/upload-video-in-chunks-azure-blob-storage

https://stackoverflow.com/questions/61857337/how-to-upload-a-large-file-in-chunks-with-parallelism-in-azure-sdk-v12

• Scan throughput rate limit: Malware Scanning can process up to 2 GB per minute for each storage account. If the rate of file upload momentarily exceeds this threshold for a storage account, the system attempts to scan the files in excess of the rate limit. If the rate of file upload consistently exceeds this threshold, some blobs won't be scanned.
• Blob scan limit: Malware Scanning can process up to 2,000 files per minute for each storage account. If the rate of file upload momentarily exceeds this threshold for a storage account, the system attempts to scan the files in excess of the rate limit. If the rate of file upload consistently exceeds this threshold, some blobs won't be scanned.
• Blob size limit: The maximum size limit for a single blob to be scanned is 2 GB. Blobs that are larger than the limit won't be scanned.

reference docs- https://github.com/Azure/Microsoft-Defender-for-Cloud,

https://github.com/Azure/Microsoft-Defender-for-Cloud/blob/main/Workbooks/README.md

Alternatively, files that are greater than 2GB, try using Azure Logic Apps for handling malware scan results and copying the blob to another storage account. Logic Apps provide a simple, no-code approach to setting up response, although the response time might be slower than the event-driven code-based approach. Please see Option 1: Logic App based on Microsoft Defender for Cloud security alerts for steps on configuring this, the default is deleted but you can modify to move it.

Reference - https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-storage-malware-scan, https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/defender-for-cloud/defender-for-storage-malware-scan.md

Similar thread for reference - https://learn.microsoft.com/en-us/answers/questions/1457488/how-to-get-round-the-microsoft-defender-2gb-file-l

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

---

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Hi @Nehruji R

Wow, thank you so much for the outstanding detailed response, the depth of your insights and the time you took to break everything down. I've forwarded your suggestions internally, so they can evaluate/test the chunk approach.

Just one more question: I've mentioned about a portal that Idk if it is managed by Microsoft/community (and the source code is open for forking). Do you know anything about it? here is the app: https://prodenvstoracc01.z28.web.core.windows.net/

Thank you again and feel free to close this thread and mark/accept your response.

Renan Nascimento