Our company is using a data extraction tool/provider called FiveTran and it appears they way FiveTran bills is based off of "MAR's" (Monthly Active Records) and have consistently seen 33% of our usage occur during the last day of the month. Per FiveTran, this is due to the following two NetSuite tables do not have a date column. This data column is needed for a delta load, as without it, the full table is load every month to ensure completeness.
bomrevisioncomponentmember
itemgroupmember
Here is the feedback from FiveTran (the data extraction tool/companay)
We have investigated the issue and found that these tables from NetSuite do not include any date column. Consequently, it is not possible to load these tables incrementally. We classify such tables as "reimport tables," meaning that every time(In every sync job), we need to reimport all the records in the table to ensure data consistency, unlike other tables. Without a date column, there is no way to extract data from the point where the previous import ended. Therefore, we charge for the full set of records in the table once a month, irrespective of importing these tables in every sync job multiple times in a billing month, in accordance with the MAR policy outlined in the document provided by
bomrevisioncomponentmember
itemgroupmember
We have also observed that a few tables do not have a primary key. As a result, such tables are also imported during every sync job, as outlined in the connector documentation link. Below is an example of one such table.
BinInventoryBalance
Thank you for any input you can provide, Matthew Pitkin