To do this, create a new GPO and drill down to: Computer configuration->Policies->Windows settings->Security Settings->File System
Add each file below, removing Users from the permissions so it’s only Administrators, Creator Owner and System:

• %AllUsersProfile%\Microsoft\Windows\Start Menu\Programs\Accessories\Windows PowerShell\Windows PowerShell.lnk
• %AllUsersProfile%\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk
• %AllUsersProfile%\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Windows PowerShell Modules.lnk

Keep in mind the path for %AllUsersProfile% is typically C:\ProgramData.  When you add the files using Group Policy Management you will select the files by drilling down to C:\ProgramData, Microsoft, Windows, Start Menu, etc, but you will notice when you’ve added them it auto-magically changes the path %AllUsersProfile%.
Use the “Configure this file or folder and propagate inheritable permissions to all subfolders and files” option on all three files.  Make sure to attach the new OU to your RDS OU or modify your existing RDS GPO. Run gpupdate /force from the command prompt on your RDS servers after adding/modifying this GPO and you’re ready to go!

Off course you can also configure the security settings manually or script.

icacls "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Accessories\Windows PowerShell" /T /inheritance:d
icacls "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Accessories\Windows PowerShell" /T /remove:g "BUILTIN\Users" Everyone
icacls "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk" /inheritance:d
icacls "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk" /remove:g "BUILTIN\Users" Everyone

** Update 25-02-2013 **

A colleague of mine has also written a blog about the same problem. He solved the problem with a group policy. For his solution see: http://www.emware.nl/articles/remove-server-manager-and-powershell-icons-from-taskbar.html

Disclaimer.
The information in this article is provided “AS IS” with no warranties, and confers no rights. This article does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my opinion.