I am using powershell to read the contents of a file and write to a new file (line by line), but the new file is always bigger than the old

Greg Booth 1,371 Reputation points
2022-12-12T10:56:51.45+00:00

I am using powershell to read the contents of a file line by line and write to a new file (line by line), but the new file is always bigger than the old.

The script looks like below.
for example with an input file of size 1627KB the newfile is 1635KB - 8 KB bigger. The original file had 7983 lines.
Might I be adding an extra control character to the end of each line by writing to the new file line by line.

Thanks

Script:
$filename = "c:\dev\20221209.csv"
$newfile = "c:\dev\new.txt"
$halffile = "c:\dev\half.txt"
New-Item $newfile
New-Item $halffile
$file = New-Object System.IO.StreamReader -Arg $filename
$tot = (Get-Content $filename).Length
$half = $tot/2
$half = $half+1;
$lnum =0
while (($line = $file.ReadLine()) -ne $null) {
$lnum = $lnum + 1;
Add-Content -Path $newfile -Value $line
If ($lnum -lt $half)
{
Add-Content -Path $halffile -Value $line
}
}
echo "done"

Windows for business | Windows Server | User experience | PowerShell
0 comments No comments
{count} votes

Accepted answer
  1. Rich Matheisen 47,901 Reputation points
    2022-12-12T15:46:48.47+00:00

    Are you doing anything with each line, or are you just copying the file?

    Get-Content <file> -Raw | Out-File <newfile> -NoNewLine  
    

    Or just use the Copy-Item cmdlet.


1 additional answer

Sort by: Most helpful
  1. Greg Booth 1,371 Reputation points
    2022-12-12T14:35:26.817+00:00

    The issue appears to be that the file has come from linux that only has linefeed (LF) and not carriage return (CR)
    When i use Add-Content it adds LF and CR to the end of each line - so an extra character on each line
    To get round this i have used Add-Content with -NoNewLine (so no LF or CR on each line) and manually added a LF to each line.
    So in the above script
    $lf = "`n"; # Line feed
    $linelf = $line + $lf; # Add line feed to the end of the line
    Add-Content -NonewLine -Path $newfile -Value $linelf # adds the line with only the line feed to the file

    This way the file sizes are the same.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.