I can provide some idea that I have. The loader queue is actually one of the repository-item defined in the DataWarehouseLoaderRepository managed on the data loader server instance. The log file data is populated into this queue through JMS message processing. The components which are responsible for writing the data files periodically rotate log files to indicate that when they are no longer writing any new information in a log file so that the log file is ready for being processed by the data warehouse loader server. There is a LogRotationSink component (/atg/reporting/datawarehouse/loaders/LogRotationSink) sink which is added to ATG's dynamo messaging (patchbay) through ARF\base\config\atg\dynamo\messaging\dynamoMessagingSystem.xml. Now whenever a data file writing component rotates a log file, LogRotationSink receives a JMS message and adds the closed log file and its type to the queue in the DataWarehouseLoaderRepository.
The data loader instance (having the DataWarehouseLoaderRepository component) is actually a separate ATG server instance which doesn't handle customer requests. It just processes data from the log files and loads it into the Data Warehouse DB. For each type of log file, there is a corresponding data loader component that needs to be configured on this instance as per your environment. These loader component configuration include properties like directory path to the log files location and a scheduler specifying the schedule for processing the corresponding queue.
Now while configuring your ATG environment with CIM when you select the "Reporting" add-on during the product selection step then CIM will perform the steps required for configuring the data warehouse loader server instance. Also note that this data loading environment includes several repositories like the user profile repository, product catalog, and order repository whose tables are on the production environment like so the datasources for those repositories have to be configured to point to the production database.
Refer the documentation as well:
First of all a big thanks for you reply.
Actually I have configured the DataWarehouse Loader and its components.Even I have enabled the logging mechanism.
I can manually pass the log files into queue and then populate the data into Data Warehouse database.
You have said "The log file data is populated into this queue through JMS message processing".
This is where I am facing problem.
I am not able dont know the process to configure the JMS message processing.There are not much guides available as well.
Could you please help me to configure the JMS message processing so that the log files get populated into the Loader queue automatically and periodically.