I used workaround for this. I just added correct NLS_LANG setting in definition of shortcut used for launching the R client:
and it works fine now.
I know something is messed up in Windows registry and R client cannot correctly read that NLS_LANG from.
I don't want to deal with the registry to fix root cause as there is plenty other Oracle software installed and working proper on that host.
Hopefully it will help others.