How to convert the content md5 value (base64) from getmetadata activity to hex to match locally generated MD5 value?

Question

I developed a code to get the content MD5 value from Getmetadata activity of a file from my azure data lake storage account. Content MD5 value thus generated is in base64 format. I am receiving the MD5 value from a linux system which is in hexadecimal (hex) format and is not matching with my Content MD5 value. I am not having any batch service so cannot use azure functions. I tried to convert my content MD5 value using hex(),base64toString() functions available in adf but not giving correct output. Please suggest me here on how to match these MD5 values from local to adf generated valuee.

Answer 1

Hello D Sai Sowmya,

Thank you for posting your query here!

ContentMD5 is a base64 representation of the binary hash value and not the resulting hex string.

Please try to convert MD5 byte array into Base64 string and see if that matches. https://powers-hell.com/2021/12/31/calculate-validate-md5-hashes-on-azure-blob-storage-files-with-powershell/

Here we convert base64 -> binary -> hexadecimal:

$md5sum = [convert]::FromBase64String('Z78raj5mVwVLS4bhN6Ejgg==')
$hdhash = [BitConverter]::ToString($md5sum).Replace('-','')

If you need to do it the other way around, you'll first need to split the hexadecimal string into byte-size chunks, then convert the resulting byte array to base64:

$hdhash = '67BF2B6A3E6657054B4B86E137A12382'
$bytes  = [byte[]]::new($hdhash.Length / 2)
for($i = 0; $i -lt $bytes.Length; $i++){
  $offset = $i * 2
  $bytes[$i] = [convert]::ToByte($hdhash.Substring($offset,2), 16)
}
$md5sum = [convert]::ToBase64String($bytes)

Do let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.