Two options that may work depending on the specifics of your deployment are using license reservations in the server settings where you can specify a number of agents will be reserved for either specific agent groups or entire system profiles. The other option would be actually modifying the agent mapping to exclude the agents you no longer want connected. They would still connect but if they are not tied to a system profile then they won't be using a license.
Both options may require either an application/agent, collector, or server restart so that it will evaluate the agent mappings/license rules again (may not be necessary if you use reservations I'm not sure).
Of course the best option is to actually remove the agent from the application. Also keep in mind I believe some contracts may prohibit you from moving agents around between environments or apps and such so keep that in mind if it applies.