Interative Write Operation vs Write Operation

Question

All,

Why is "Iterative Write" operations which is based on meta-data more expensive than Write operations for Hierarchical Namespace?

Thanks,
Gopi

Answer 1

Hello @grajee ,

Thanks for the question and using MS Q&A platform.

My understanding is that you would want to know the difference between iterative write operations compared to that of write operations and how are they billed. If so, please refer to below info which would give you more insights of each functionality and their billing.

1. Write operations: The following are considered write operations: AppendFile, CreateFilesystem, CreatePath, CreatePathFile, FlushFile, SetFileProperties, SetFilesystemProperties, RenameFile, RenamePathFile, CopyFile

**How are they billed: **
When uploading or appending data to existing files, or when reading data from a file, the operation gets chunked into 4MB pieces. You will be billed for each 4MB chunk.

The Copy File, Rename, Set Properties, etc. would not be charged using the “per 4 MB” rule. They would be charged as a single transaction.

For files < 4MB, a full transaction will be charged for each file.

2. Iterative write operations : Iterative write operations are operations performed on a folder that requires the system to iterate through all the subfolders and files in that folder to complete. Examples: RenameDirectory, RenamePath, RenamePathDir
   A hierarchical namespace allows operations like folder renames and deletes to be performed in a single atomic operation, which with a flat namespace requires a number of operations proportionate to the number of objects in the structure.

**How are they billed: **
For example, if you would want to rename a directory containing 5k files. The rename would be charged as a single metadata operation (iterative writes).

For more info, please refer to below articles:

• Azure Data Lake Storage Gen2 Billing FAQs
• Azure Data Lake Storage pricing

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators