ms sql 2019 community polybase log file filling up disk drive

Flash Gordon 176 Reputation points
2021-03-23T20:11:06.16+00:00

Hi, I have ms sql community 2019. I noticed an update for ms sql recently. Today my hard drive filled up, normally I have about 1/2 of it or 250 gig free. In Treesize below you see that in folder ~MSSQL -> Log -> Polybase it says that folder is 345 gig, so something has gone nuts. This is a nothing, client based, database I use to remember sql, ssis, and vs 2019 techniques, it is less than a megabyte in size of data. I never used polybase. How do you clear this Polybase log? Would it be OK to just delete the Polybase directory? Or is my only option to uninstall MS Sql? This problem is being fairly well behaved. There are a few gigs of free disk space left and nothing is failing because of no disk space. I think this is the latest ms sql server update is below: KB4583458 - Description of the security update for SQL Server 2019 GDR: January 12, 2021 80862-polybasetreesize.jpg 1: /api/attachments/80862-polybasetreesize.jpg?platform=QnA 80855-polybasealone.png80872-dumpdirectory.jpg

SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
12,601 questions
0 comments
Accepted answer
  1. Erland Sommarskog 100.8K Reputation points MVP
    2021-03-23T22:29:28.917+00:00

    This is a known issue that has been fixed. Get the most recent https://support.microsoft.com/en-us/topic/kb5000642-cumulative-update-9-for-sql-server-2019-97ad5c3e-e002-4b6d-b566-698bf70ca44a
    here.

    Although, if you don't plan to use the Polybase feature, I would consider to turn off the Polybase services (in SQL Server Configuration Manager), or even uninstalling the component.

    1 person found this answer helpful.

5 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. CathyJi-MSFT 21,066 Reputation points Microsoft Vendor
    2021-03-24T03:29:35.92+00:00

    Hi @Flash Gordon ,

    > I never used polybase. How do you clear this Polybase log? Would it be OK to just delete the Polybase directory? Or is my only option to uninstall MS Sql?

    You do not need to uninstall SQL server instance, suggest you uninstall Polybase feature only , since you do not need to use this feature.

    Control Panel\Programs\Programs and Features . Right click MS SQL 2019 setup> Uninstall > only choose the Polybase feature on select feature page.

    81001-screenshot-2021-03-24-111421.jpg

    > I think this is the latest ms sql server update is below: KB4583458

    CU 9 (KB5000642) is the latest Cumulative Update for SQL 2019, KB4583458 is the latest GDR for SQL 2019. Suggest you applying CU9 for SQL 2019. Refer to MS SQL server versions list.

    > ms sql 2019 community polybase log file filling up disk drive

    Since these dumps are being created specifically in the polybase subdirectory, you may want to focus specifically on polybase dumps. You need to analyze the dump file to fix the issue about multitudinous polybase dumps being created and this fill up disk space.

    But since you do not use Polybase, uninstalling Polybase feature is the best choice.

    If the response is helpful, please click "Accept Answer" and upvote it, thank you.

    1 person found this answer helpful.
    0 comments
  2. Flash Gordon 176 Reputation points
    2021-03-24T15:32:12.133+00:00

    Erland/Cathy, 

       Thanks for the prompt reply.   When I turned on the machine this morning this c: disk space looked normal in File Explorer (see screen print below), even though I didn't do anything.  The Polybase and Dump directory still showed 374 gig in File Explorer.  Is it possible that the Polybase disk usage reporting is just spurious?   I actually work integrating different database environments, so Polybase might be interesting, although getting access to db's like Teradata is difficult outside of a big company environment.  But if I have to, I'll uninstall it.

    81147-afterdiskspace.jpg

  3. Flash Gordon 176 Reputation points
    2021-03-25T15:38:08.523+00:00

    Ok, this is now understood. Polybase did not just fill my disk drive in one day. What it does do is add a unique1/2 gig Polybase time stamped log file every time you start Management Studio against the database?. Polybase does not reuse this log file, it creates a new one for every Management Studio session. I saved the March polybase log files so everyone can see them below. The breadcrumb on my machine to get to the polybase directory is this:

    C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\MSSQL\Log\Polybase\dump

    I had ~500 of these files hence the 250 gig on my hard drive. It's easy to fix once you know what's happening. You can delete these files with File Explorer on an admin id. If maintaining it like that bothers you, you could write a script. Kudos to TreeSize for presenting a graphical image of disk usage that allowed me to find this problem. Thanks, Erland and Cathy for the help.

    81597-finalpolybasedir.jpg81598-finaldiskspace.jpg

    0 comments
  4. Flash Gordon 176 Reputation points
    2021-03-25T21:22:47.523+00:00

    Ok, I'm a liar. When I deleted all prior Polybase log files I could see how many log files got created in a day. I signed on to Management Studio maybe 5 hours ago, I did nothing in Mgmt Studio, and it has created 7 of these 1/2 gig log files just today (see below). So you may have to manage these log files pretty actively, depending on how much free space you have. The ms sql database engine starts if you boot your machine and creates these polybase log files. This problem is only for the release of ms sql community that I am on, 15.0.2080.9. Erland says CU9 fixes this big polybase log file problem.