Select 'Force HTTP Recording' in OpenScript Preference, ie open OpenScript Preference, the expand Record node, then select Oracle EBS/Forms Load, Then check checkbox 'Force HTTP Recording' true.
If it doesnt work try following:
Dont install a new java version. Openscript comes along with java.
Go to OracleATS intalled folder, open openscript, you can see some batch files there. Open 'UninstallBrowserHelpers.bat', after that open 'InstallBrowserHelpers.bat'. Then open 'HelperService' folder in openScript. Then inside 'bin' open 'UninstallHelperService.bat' and 'InstallHelperService.bat' one after another.
Then try to record forms.
See console whether jar files are been downloaded during recording of forms.
Another solution is there, but not a correct procedure.
If you have another machine with OATS working fine with forms, then go to run, type %temp%. Copy 3 folders - OpenScript_FormsFT, OpenScript_Applet, OatsTempRTCache and paste it to %temp% in your machine(machine in which forms are not get recorded)