Not surprised to see that you are relatively new to the platform, based on how you wrote the questions and what kind of answers you accepted. Every time you learn a thing, please try to build up more contexts around them.
PDF Files
While you claimed that you know what a PDB file is, what it actually does and what it is for, then by asking "can I like delete the PDB" indicates that you didn't really know much. You might be astonished to learn that many .NET based products (like JetBrains ReSharper) ship PDB files even when their assemblies are release builds. The most common reason behind is that they are collecting diagnostics data and with PDB files the data can be more accurate (exception call stacks can contain even line numbers).
Like @Castorix31 explained, SLN files are another kind.
Ambiguous Statement
"People tell me that I need to do more other than just selecting Build option to compile a C# App with GUI into an executable" then did you ask under what context you should follow that and why? If they cannot explain fully the reasons behind a recommendation, don't follow it blindly and sometimes don't even waste your time to remember that.
.NET has been there for two decades, and changes rapidly in the past few years due to .NET Core evolution. So, old school knowledge can be deprecated.