How to Remove Entries from the Remote Desktop Connection

Delete the entries within the registry (run regedit from the command line)

In the following path:
HKEY_CURRENT_USERSoftwareMicrosoftTerminal Server ClientDefault

Right click each remote desktop log profile you want to remove and click delete.