Would anyone confirm that there is no way to create a unique robots.txt file within NetSuite?
I assume you cannot since when you create robots.txt in the web hosting files area, you get a funny icon next to it. In addition, when you enter www.website.com/robots.txt you get a global:
# Allow all robots to spider everything by disallowing nothing
User-agent: *
Disallow:
If you crate robots.csv it works fine, you can access the file.