You're going to want to iterate over each alert and convert them each to JSON individually, otherwise you're converting an array of alerts into one json string (which when compressed will be on one continuous line).
Note that if you do convert each alert individually, the resulting file won't contain valid json (each line will be valid, but the collection of lines will be invalid)
Based on the snippet above, this should end up with one line of valid json per alert
$Alerts = Invoke-RestMethod -Method Get -Uri $Url -Headers $Headers -ErrorAction Stop
$OutputJsonPath = "$Path\test.json"
foreach ($Alert in $Alerts.value) {
$Alert | ConvertTo-Json -Compress -Depth 100 | Out-File -FilePath $outputJsonPath -Encoding ascii -Append
}
(I don't know what format the result comes back in, $Alert in $Alerts.value
may be able to just be $Alert in $Alerts
)