While installation of dynatrace manage you can force which IP should be used.
--cluster-ip — If your machine has more than one network interface you need to decide which network interface will be used for Dynatrace Cluster traffic and put its IP4 address here.
I think now you will have to reinstall cluster because if you will kill the one IP which is used by cassandra elastic etc, dynatrace can stop working.
Maybe somebody has done something like that or someone from Dynatrace will confirm that. Now Dynatrace has picked one of your NIC’s and this one will be used in communication and as entry point in web console. I don’t see any other option in documentation actually. You can create new cluster and migrate data from existing one without loosing configurations, data and uptime.
You can also modify the nginx configuration file, but it will get overwritten during upgrades, so you will have to perform the changes after each upgrade.
Also if the nginx is listening on all interfaces, you can also modify the firewall rules to disallow connections on unwanted interfaces.