Share via

Powershell query to download file from web

Manny 151 Reputation points
2022-03-06T07:05:22.767+00:00

Hi

My powershell query downloads a file with 0 kb. That isn't right. Copying and pasting the $Source URL in a browser will let me download the file. That file is correct. However for some reason when I use the Start-BitsTransfer cmdlet the rates.csv file gets created but it has no data in it. What am I doing wrong? 

$Source = "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/daily-treasury-rates.csv/all/all?type=daily_treasury_yield_curve&field_tdr_date_value=all&page&_format=csv"  
$Destination = "C:\Users\Me\Documents\Discount_Rates\rates.csv"  
Start-BitsTransfer $Source $Destination

@Rich Matheisen , are you able to help?

Thank you

Windows Server PowerShell
Windows Server PowerShell
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.PowerShell: A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
5,462 questions
0 comments
Accepted answer
  1. Andreas Baumgarten 104K Reputation points MVP
    2022-03-06T08:36:11.187+00:00

    Hi @Manny ,

    why not using Invoke-WebRequest instead of Start-BitsTransfer ?

    This works here: 

    $Source = "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/daily-treasury-rates.csv/all/all?type=daily_treasury_yield_curve&field_tdr_date_value=all&page&_format=csv"  
$Destination = ".\Junk\rates.csv"  
Invoke-WebRequest -Uri $source -OutFile $Destination

    ----------

    (If the reply was helpful please don't forget to upvote and/or accept as answer, thank you)

    Regards
    Andreas Baumgarten

2 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Rich Matheisen 45,906 Reputation points
    2022-03-06T20:56:40.417+00:00

    I had a look at the Start-BitsTransfer (using your source) with both Fiddler and WireShark. I can see the TLS handshake and then . . . nothing. :-/

    I used this code to download the file, though: 

    $WebClient = New-Object System.Net.WebClient
$WebClient.DownloadFile($source,$destination)

    It's faster than Invoke-WebRequest and, because it doesn't try to parse the HTML in the response, less prone to hanging.

    0 comments
  2. Rich Matheisen 45,906 Reputation points
    2022-03-07T00:01:34.757+00:00

    It's always best to stick with the datetime object to handle the conversion of dates and times. The results are consistent and the datetime object store the information that makes it possible to do comparisons and date arithmetic. 

    # BEST - parses date according to string and produces a datetime object
[datetime]::ParseExact("3/05/2022","M/d/yyyy",$null)

# works, but you're still left with a string instead of DateTime object and doesn't always produce 2-character month and day
"03/05/2020" -replace "(\d\d?)/(\d\d?)/(\d\d\d\d)",'$3/$1/$2'

# works but uses indexing and type conversion but output format is consistent
$x = "3/5/2020" -split "/"
"{0:d4}/{1:d2}/{2:d2}" -f [int]$x[2],[int]$x[0],[int]$x[1]

# example of use (pick your poison from the three methods above)
Import-CSV c:\junk\$Destination |
    ForEach-Object{
        $_.Date = [datetime]::ParseExact($_.Date, "M/d/yyyy", $null)    # preferred method
        $_
    } | Export-Csv c:\junk\newfile.csv -NoTypeInformation