As far as I know - there is not limit to the number og rows that can be produced by the apex_item API. Since there is nothing in the documentation about this, probably only Oracle can answer this.
However you might run into server specific limitations when posting that many data (items) in a single POST call - perhaps you server log files can show you more details when you get "Internal Server Error".
I would advice you to re-consider the tabular form/post method and look for another way to handle your requirements. Ex.:
- Using the Editable Grid which will perform JSON based AJAX calls when updating data, instead of the POST, sending only the changed records.