See this StackOverflow question: How to use OpenGL in JavaFX?
It sounds like the best answer for you is the addition of an OpenGL node to JavaFX.
An OpenGL node has not yet been added to the JavaFX platform.
As you have a team of people experienced in 3D development and Java and all of the relevant JavaFX code is open source, I'd encourage you to consider creating an OpenGL node or working with Oracle to create one and (if you are inclined) contributing the development back to the JavaFX code base. If you are interested in this, contact the openjfx-dev mailing list.
> Do we have to develop our new User Interface in Swing because of this?
I don't think so.
> Or can the announced JavaFX SwingNode be used to wrap such a complex Swing component?
I don't know, but it doesn't sound quite the right approach to me.
It seems a dedicated OpenGL rendering node would be a better fit as long as you don't need other Swing functions in your SwingNode.
> If so, how would the performance suffer from the JavaFX wrapping?
I think if you had a JavaFX Node which was a Swing Node which handled OpenGL then performance would likely be worse than a JavaFX Node that handled OpenGL directly.
> If the SwingNode doesn't solve the issue and we therefore need to implement the main application in Swing, should we still consider implementing non-3D sub-views with Java FX 2 or should we go for Swing all the way?
Either way would work, but I do not recommend mixing the technologies for your application unless you need to.
There are some considerations with mixing JavaFX and Swing:
1. Swing widgets look different from JavaFX widgets (and it's not trivial to make them look the same).
2. You have to learn two toolkits then mentally switch between them when developing (this is just annoying).
3. You have to be (very) careful of threading issues as each toolkit has it's own primary thread.
4. There are some bugs in mixing JavaFX and Swing (search JFXPanel in the issue tracker) that simply wouldn't occur if you weren't mixing libraries (most of these bugs have been addressed but some are outstanding).
5. The Swing functionality isn't going evolve, it's good at what it does, but it is not going to change and get better.
There is (very experimental) work in merging the JavaFX and Swing application threads, which makes a combined programming model a bit nicer to deal with, but it remains to be seen if that experimental feature becomes a default for both platforms.
Unless you want to reuse existing extensive Swing libraries (like NetBeans RCP), a pure JavaFX application seems preferred (as long as the OpenGL node can be worked out).
> Our developer group has previous experience from Swing, but none from Java FX.
There are similarities, previous knowledge of Swing is a benefit, but there are a lot new things to learn in JavaFX (and a few things to unlearn from Swing likely).
My 2c . . . best of luck with your project.
Thank you very much for your extensive reply jsmith! All you say makes sense.
A dedicated OpenGL rendering node would most likely be the best solution for us, but I am afraid we will have to settle with Swing for now. Our project is already behind schedule and we won't be able to get resources for framework development at this time, even though it sounds both interesting and fun. I will forward the suggestion to our companies central department though, who supports us with framework and strongly suggests us to use Java FX 2 instead of Swing.