Here is some code that does it. This example uses 'hard coded' file paths (from a CopyAsPath), but you could very easily incorporate a folder browser to allow selection to be made.
This code can delete files so TEST it with some test files (I just copied some random .txt files and renamed to .mp3 and .txt in appropriate folders (see code at top for my test paths)
The actual DELETE function is commented out in the code, and only when you have set up a test environment, uncomment the line. I have also put a STOP in the code. When testing for first time, the STOP will allow you to check the values being used, in particular, the variable 'Name' which would be a file about to be deleted. Once happy with all of that, then stop the execution and remove the STOP command and allow the deletion on next run.
As you said, I have not bothered to add any Lists or anything in the UI, but that too is easily added if needed.
Option Strict On
Option Explicit On
Public Class Form1
Dim mp3Path As String = "C:\Users\lesha\Desktop\MP3 Folder"
Dim txtPath As String = "C:\Users\lesha\Desktop\TXT Folder"
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
' create a collection of mp3 files
Dim mp3s() As String = IO.Directory.GetFiles(mp3Path, "*.mp3")
' create a collection of txt files
Dim txts() As String = IO.Directory.GetFiles(txtPath, "*.txt")
' loop through txt file names and
' check if in mp3 collection
For Each txtFile As String In txts
' create a mp3 file name to search
' with (needs to be mp3 filename with
' a .mp3 extendion
Dim name As String = IO.Path.Combine(mp3Path, IO.Path.GetFileNameWithoutExtension(txtFile) & ".mp3")
If Not mp3s.Contains(name) Then
' no file match in mp3 folder so
' can delete this file (the .txt one)
' this next line would do the delete
' but first, try out the code on
' some test files to ensure it does
' what it is supposed to do! Be careful
' to not mix up the paths.