6 Replies Latest reply on Sep 20, 2013 6:13 AM by Gautam Kamath

    Indexing custom item descriptors

    Gautam Kamath

      Hi all,

      In product catalog i've made a new item type. This new item is in no way connected to product or sku. How do i go about indexing this new item type?


      For starters, i've made a component similar to ProductCatalogOutputConfig and have lined it up with others in ProductCatalogSimpleIndexingAdmin's list of components. Ive made a new definition file and for now i'm trying to indexing only its name.

      But when i run baseline indexing, i get a SOAPFaultException and the indexing fails.


      Looking at the log and the exception type, I'm guessing it is because somewhere ATG is failing to submit the records to CAS. I confirmed this by replacing data document submitter of my new component with console document submitter.

      The process works fine but obviously the new item doesn't get submitted to CAS and hence doesn't get indexed.


      Following is the log:-

      12:50:25,986 ERROR [ProductCatalogSimpleIndexingAdmin]

      atg.repository.search.indexing.IndexingException: javax.xml.ws.soap.SOAPFaultException: Fault occurred while processing.

              at atg.endeca.index.admin.IndexingPhase.doPhaseCleanup(IndexingPhase.java:395)

              at atg.endeca.index.admin.IndexingPhase.performPhaseTasks(IndexingPhase.java:324)

              at atg.endeca.index.admin.IndexingPhase.performPhaseTasksInParallel(IndexingPhase.java:248)

              at atg.endeca.index.admin.IndexingJob.performJobInParallel(IndexingJob.java:297)

              at atg.endeca.index.admin.SimpleIndexingAdmin.indexBaseline(SimpleIndexingAdmin.java:655)

              at atg.endeca.index.admin.SimpleIndexingAdmin.indexBaseline(SimpleIndexingAdmin.java:637)

              at atg.endeca.index.admin.SimpleIndexingAdmin$2.run(SimpleIndexingAdmin.java:1017)

              at java.lang.Thread.run(Thread.java:662)

      Caused by: javax.xml.ws.soap.SOAPFaultException: Fault occurred while processing.

              at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:156)

              at $Proxy167.writeRecords(Unknown Source)

              at com.endeca.itl.recordstore.RecordStoreWriter.doFlush(RecordStoreWriter.java:207)

              at com.endeca.itl.recordstore.RecordStoreWriter.flush(RecordStoreWriter.java:180)

              at com.endeca.itl.recordstore.RecordStoreWriter.close(RecordStoreWriter.java:200)

              at atg.endeca.index.RecordStoreDocumentSubmitterSessionImpl.endSession(RecordStoreDocumentSubmitterSessionImpl.java:585)

              at atg.endeca.index.AbstractRecordStoreAggregateSession.commitPending(AbstractRecordStoreAggregateSession.java:489)

              at atg.endeca.index.AbstractRecordStoreAggregateSession.endSession(AbstractRecordStoreAggregateSession.java:427)

              at atg.endeca.index.RecordSubmitterImpl.endSession(RecordSubmitterImpl.java:231)

              at atg.endeca.index.admin.IndexingPhase.doPhaseCleanup(IndexingPhase.java:376)

              ... 7 more

      Caused by: org.apache.cxf.binding.soap.SoapFault: Fault occurred while processing.

              at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.unmarshalFault(Soap11FaultInInterceptor.java:75)

              at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.handleMessage(Soap11FaultInInterceptor.java:46)

              at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.handleMessage(Soap11FaultInInterceptor.java:35)

              at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)

              at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:113)

              at org.apache.cxf.binding.soap.interceptor.CheckFaultInterceptor.handleMessage(CheckFaultInterceptor.java:69)

              at org.apache.cxf.binding.soap.interceptor.CheckFaultInterceptor.handleMessage(CheckFaultInterceptor.java:34)

              at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)

              at org.apache.cxf.endpoint.ClientImpl.onMessage(ClientImpl.java:798)

              at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1667)

              at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1520)

              at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1428)

              at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:143)

              at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:72)

              at org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:102)

              at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)

              at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:658)

              at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)

              at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)

              at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:532)

              at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:464)

              at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:367)

              at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:320)

              at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:89)

              at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)

              ... 16 more



      So my Qs are as follows:-

      1. Am i missing something? Is my approach correct in creating a PCOC-like component?

      2. What could be the reason of the above exception? Any pointers on solving this?


      Any help appreciated.


      Thanks & Regards,


        • 1. Re: Indexing custom item descriptors

          You need to customize the pipeline as well.



          • 2. Re: Indexing custom item descriptors
            Gautam Kamath

            Thanks for the reply!


            Are you sure about customizing the pipeline? Though it did come to my mind, but I thought i shouldn't have to go that deep into endeca territory now because the error is on the ATG side of things.

            As i said in the first post, i think the document submitter isn't able to submit/persist the records to CAS's records stores.


            Whereas, the pipeline has more to do with how endeca processes the records that are present in a valid format in CAS's record stores. Moreoever, the pipeline doesn't kick in until indexing starts. But in my case, the indexing (on ProductCatalogSimpleIndexingAdmin's dyn/admin page) gets cancelled due to errors. So unless the records safely reach CAS, i don't see how modifying pipeline would help here. I maybe wrong in my theory. Please correct me if i am.


            Nonetheless, i will try modifying the pipeline and see if it works.



            • 3. Re: Indexing custom item descriptors

              Hi ,


              Is that new item descriptor now way related to Product or SKU?  if its of a relationship item then you might need to develop a property accessor component in ATG and use it in product-sku-output-config.xml. as show below.


              either as product property or sku property


              <property name="<somename>" data-type="set or <whatever>" property-accessor="/componentpath/NameofPropertyAccessor" is-non-repository-property="true" is-dimension="true/false"     output-name="myoutputname" />




              • 4. Re: Indexing custom item descriptors
                Gautam Kamath

                Thanks for the reply DevReddy.


                Just to clear the confusion(if any), the new item descriptor is NOT related to product or sku. It IS a part of the ProductCatalog repository though.


                In my opinion, the property accessor method will not work in my case because:-

                a. Accessors are for indexing primitive data type properties like string, int of other items and not for indexing new item types as in my case.


                b. The indexable data generated by accessors get attached to each individual product/sku endeca record (based on product indexing/sku indexing). Whereas in my case, i want to create new set of records (in addition to product records) for the new item.


                If I use accessors, i MAY be able to attach a few properties of the new item to the product/sku record. But on searching for those properties i would be returned results of a product. My intention is to be able to get records that are based solely on the new item's property values and these records should not be contaminated by product/sku's data.

                • 5. Re: Indexing custom item descriptors

                  There should be some relation ship between product and custom type...


                  say for example Order repository... will give best seller and sold count for items if you try to index order


                  or index the store locations or inventory...


                  for custom item descriptor  :: create separate record store..in CAS  Edit the pipeline to add your custom data source..


                  Please refer Endeca documentation for more details..


                  Hope it helps...

                  • 6. Re: Indexing custom item descriptors
                    Gautam Kamath

                    ok. creating the custom item descriptor with the separate record store looks like a plan. That will require some in depth reading of Endeca documentation.


                    Nevertheless, thanks for the suggestion!