12 Replies Latest reply on Jul 3, 2013 2:45 PM by 1014064

    DPS 11g and Multiple DataViews

    881344
      DPS 11g Proxy question

      Is it possible to have DPS to just ignore/skip a DataView that is not currently available? I have proxy set up to look at multiple data views. Say ou=aa,dc=common,dc=com and ou=bb,dc=common,dc=com. If someone does a search on dc=common,dc=com and ou=bb backend and DataView is unavailable the originator will get a error:

      Server is unavailable (52)
      Additional information: Unable to process the search request. Reason: [Original
      error=52] Unable to retrieve a backend SEARCH connection.

      I would like to configure it so it would automatically just report what it found or didn't find in ou=aa and not pass an error onto the originator. Seems like there should be some sort of configuration controlling what errors get passed.
        • 1. Re: DPS 11g and Multiple DataViews
          Sylvain Duloutre-Oracle
          When a search request targets multiple data views, every data view must be "available", that is, at least one directory server instance associated with it must be up and running.
          • 2. Re: DPS 11g and Multiple DataViews
            881344
            Bummer. Really a poor design decision, again.

            No reason not to have an option to not pass on errors to the client. In trying to create a HA (High Availability) envrionment you don't want your entire infrastructure as unavailabe if just one of your data views isn't available.

            I would submit an RFE, but can't even get them to add in some other the other missing things from DPS 5.x that were submitted as bugs and then later changed to RFE's.

            /rant
            • 3. Re: DPS 11g and Multiple DataViews
              Sylvain Duloutre-Oracle
              A data view corresponds to a logical part of the directory information tree.
              Entries associated with a data view must always be stored in more than 1 directory server instance for H/A. So there should always be 1 ds up and running for each data view.

              Multiple data views are configured when there is a need to aggregate/connect several entry trees from several ds holding distinct set of entries so that it looks as a single entry tree from the client point of view.

              In your use case, a search request is routed to several data views, so the result set has to be built from several (non-data equivalent) directory server instances. Would all
              ds corresponding to a data view be unavailable, the result would be partial (some entries may be missing). Now the question is : is it valid to return partial result to a client application knowing the fact that there is no standard LDAP error indicating a partial result?

              Could you elaborate on your use case and the reason why you need several data views ?

              Thanks
              • 4. Re: DPS 11g and Multiple DataViews
                881344
                You want the proxy itself to be HA but not every configured data view behind it should have to be!
                is it valid to return partial result to a client application knowing the fact that there is no standard LDAP error indicating a partial result?
                Absolutely! Returning a partial result would always be more beneficial than returning NO RESULT or erroring completely. Especially when the Data view that is unavailable most likely doesn't have the result it wants anyway!

                Also there IS an LDAP standard for returning partial results. See error code 9. ;)
                http://wikis.sun.com/display/SunJavaSystem/LDAP+Error+Codes
                Could you elaborate on your use case and the reason why you need several data views ?
                I could give you several examples and there are several in the Sun/Oracle docs themselves that fit. But I'll just stick with the one above. ou=aa is part of a data view that has serveral global directories on the backend. ou=bb is a single directory at a single location lets say for both logistic and/or legal reasons. However we would want a search to dc=common,dc=com to look at both dataviews if available. Say a network connection goes down to the ou=bb datacenter or for some other reason the server that hosts ou=bb becomes unavailable, currently the proxy then would not return any results and return an error 52.
                • 5. Re: DPS 11g and Multiple DataViews
                  Sylvain Duloutre-Oracle
                  Thanks for the details.

                  The LDAP error 9 you've mentioned is not applicable here: It is obsolete (LDAPv2) and a referral is returned to an alternate ds containing the missing results. Every ds the proxy knows about is down so it cannot return a valid ds from which the client can get the extra data.

                  You may want to consider a new option in DPS 11gR1PS1 that let the proxy process a search request even when some data views are not available.
                  See http://download.oracle.com/docs/cd/E20295_01/html/821-1224/require-data-view-availability-5dpconf.html#scrolltoc

                  DSEE 11gR1PS1 aka 11.1.1.5.0 is available at http://www.oracle.com/technetwork/middleware/downloads/oid-11g-161194.html

                  Edited by: Sylvain Duloutre on Aug 11, 2011 6:01 PM
                  1 person found this helpful
                  • 6. Re: DPS 11g and Multiple DataViews
                    802907
                    Returning a partial result would always be more beneficial than returning NO RESULT or erroring completely. Especially when the Data view that is unavailable most likely doesn't have the result it wants anyway!
                    I can't agree with this in general. You may know all the ways your deployment would handle partial results, but in general the client behavior could easily break. I can see this as a desirable (non-default) configuration option, though.
                    • 7. Re: DPS 11g and Multiple DataViews
                      881344
                      I'm trying out that new setting, but so far that looks to do what I wanted it to do.

                      Thanks
                      • 8. Re: DPS 11g and Multiple DataViews
                        keesor
                        Do the two different dataviews contain the same data, but are different base suffixes? I would think for what I understand you are wanting, you would have both instances of DS with the same base suffix, and then use the Data Source Pools to do HA / Load balancing between them? But I may just not completely understand your situation.
                        • 9. Re: DPS 11g and Multiple DataViews
                          881344
                          chris_c wrote:
                          Returning a partial result would always be more beneficial than returning NO RESULT or erroring completely. Especially when the Data view that is unavailable most likely doesn't have the result it wants anyway!
                          I can't agree with this in general. You may know all the ways your deployment would handle partial results, but in general the client behavior could easily break. I can see this as a desirable (non-default) configuration option, though.
                          Depends on your task. In my example it is mainly for authentication. So if a datasource/view is down I would still rather receive a partial results of the users it can find rather to be able to authenticate them rather than just fail everyone.

                          Edited by: azentropy on Aug 22, 2011 3:15 AM
                          • 10. Re: DPS 11g and Multiple DataViews
                            881344
                            keesor wrote:
                            Do the two different dataviews contain the same data, but are different base suffixes? I would think for what I understand you are wanting, you would have both instances of DS with the same base suffix, and then use the Data Source Pools to do HA / Load balancing between them? But I may just not completely understand your situation.
                            Different data and different suffixes.
                            • 11. Re: DPS 11g and Multiple DataViews
                              881344
                              Sylvain Duloutre wrote:
                              Thanks for the details.

                              The LDAP error 9 you've mentioned is not applicable here: It is obsolete (LDAPv2) and a referral is returned to an alternate ds containing the missing results. Every ds the proxy knows about is down so it cannot return a valid ds from which the client can get the extra data.

                              You may want to consider a new option in DPS 11gR1PS1 that let the proxy process a search request even when some data views are not available.
                              See http://download.oracle.com/docs/cd/E20295_01/html/821-1224/require-data-view-availability-5dpconf.html#scrolltoc

                              DSEE 11gR1PS1 aka 11.1.1.5.0 is available at http://www.oracle.com/technetwork/middleware/downloads/oid-11g-161194.html

                              Edited by: Sylvain Duloutre on Aug 11, 2011 6:01 PM
                              Yes, in my testing setting seems to do what I want.

                              However, been seeing some very bad behavior from 11.1.1.5.0 proxy in general (nothing to do with this setting) that I'm still trying to track down. Seems to get confused and loses the ability to see all the servers in it's pool (everything is marked "Socket is closed") and then just keeps looping and eventually shuts itself down. Haven't been able to find the cause yet as when it happens it loops so fast it generates through 10 gb of access/error logs in a few minutes! I'll start a new thread on that as that is off this threads topic.
                              • 12. Re: DPS 11g and Multiple DataViews
                                1014064

                                Did you ever find a solution for the "( Socket is closed)" issue?  I am experiencing the same issue, it fills my logs very quickly until it fills the filesystem (80GB).