Is there a way to design a file adapter to throttle its processing bandwidth.
A simple use case scenario is describes as follows;
A File Adapter can only consumes a max of 5 files per minute. The producer average throughput is 3 files per minute but during peak times it can send 100 files per min. The peak times occur during end of year or quarterly accounting periods. If the consumer consumes more than 5 files per min the integrity of the environment and data is compromised.
The SLA for the adapter is to :
- Each file will be processed within 2 seconds.
- Maximum File Transactions Per minute is 5
An example is as follows.
The producer sends 20 files to its staging directory within a minute. The consumer only processes 5 of these files in the first minute, sleeps then wakes up to consume the next 5 files in the second minute. This process is repeated until all files are processed.
The producer can send another batch of files when ever it likes. So in the second minute the producer can send another 70 files. The consumer will throttle the files so it only processes 5 of these files every minute.