Performance of the storage will be similar and also of the infiniband.. However number of storage cells will vary. An Exadata quarter rack has 3 storage cells whereas a supercluster t5-8 half rack has 4. Similarly the computer node power of a t5-8 half populated server is likely to be higher than two pizza box sunfire servers.
You can put non oracle data on the ZFS storage and access it over 10 gig or 1 gig ethernet. Whether this is a good idea or not you will have to judge for yourself.