pivot table and chart slow performance in interactive format

I support our bi publisher system.

We have a report that contains row subtotal Pivot table and bar chart. Even using cache, this report still takes about 30 seconds to open every time in the interactive format.  If I set the default format to PDF, with cache, the report opens almost instantly every time.

The data size of the report is 2M. I do not think that is too big for a bi publish report.

But for the bar chart, the  x-axis  has about 40,000 data. The pivot table is a row subtotal Pivot table. When I open it in excel, it has about 19650 rows.