SmartScreen is part of Microsoft's anti-malware initiative. SmartScreen runs on many millions of PCs which provide feedback telemetry to the SmartScreen database at Microsoft. If SmartScreen detects an executable it checks the executable hash against the known executables in the database. Often used programs (by everybody providing telemetry feedback) and known good programs will have a matching hash in the database and SmartScreen will pass them. Programs which are not well used or are known to be bad will trigger SmartScreen alerts.
Essentially, the warning occurs because your software hasn't been used by many people and SmartScreen is not sure whether it is good or bad. This is unsurprising as you just created the software. As more people use the software and nobody reports it bad, this will get fed back to SmartScreen and at some stage it will accept that the software is most likely OK and stop warning. If you release a new version of the software, the process will start again.
So, some possible ways to stop SmartScreen alerting (I haven't tried these):
1. Have enough people download and use your program that SmartScreen treats it as commonly downloaded.
2. Sign your app using Microsoft's Authenticode technology.
3. On a forum, a user said: "The best workaround I've found is to zip up my installer. SmartScreen doesn't seem to complain in that case."
4. Submit your app to Microsoft to certify it and put it on a SmartScreen whitelist (e.g. by publishing it to the Windows Store).
Note that SmartScreen type technology is not specific to Microsoft and Windows, it is used in other browsers, operating systems and internet safety software distributed by security companies.
Anyway, it's not really a JavaFX specific issue, you can google SmartScreen to get further help from other developers and view other related forum posts and Microsoft documentation on it.