Not really sure since it's a third-party API. One might guess XLSX format adds functionality that requires more memory than XLS.
Any idea why the XLS seems to work fine for me but the XLSX export on the same data bogs down? Is it just the difference between the how the API works for the different formats? It would make sense.