This is more of a "should I?" type of question and not a "can I?".
We currently have a 2-node SC3.2u3 cluster on Solaris 10 where each node is in a separate building of the campus. The interconnect is a switched ethernet dedicated to the cluster and uses the default IP addresses.
We would like to add a second 2-node cluster between the same buildings. The networking group says it will not run another dedicated ethernet across our campus. We would like to use the same ethernet as the first cluster for our second cluster's interconnect. VLAN tagging is not an option, so our plan is to install the second cluster and configure the interconnect with a different subnet to avoid any IP address conflict.
The question is, should we do this? I know we can do it by using clsetup, but is it a bad idea? Are there any technical reasons this would not be advised? Will this be a configuration that Oracle will not support?
Somewhere on the Oracle webpages there is a set of examples of separate Solaris Cluster configurations using LDOMs as cluster nodes on T-series systems sharing the same physical interconnect. This only works, if the interconnect address spaces are distinct. And that is one reason why scinstall allows you to do this: chose a different base address and a different netmask.
I do not think that this is spelled out explicitly in the docs, but from this example I assume that this is fully supported.
But I do not work in support.... But I have this running in the labs for ~2 years now - without problems.