1 2 Previous Next 17 Replies Latest reply on May 27, 2009 6:19 PM by 800477

    JNDI, Active Directory, Paging and Range Retrieval

    800477
      A few other problems that developers come across are queries that return either a large number of results, or query that returns a multi-valued attribute that contains a large number of values.

      Active Directory incorporates a number of controls, that are designed to ensure optimim performance of the server and to mitigate denial of service attacks.

      First of all paging. By default, Active Directory restricts the total number of results that are returned from a LDAP Search to 1000. While this limit can be changed by modifying the LDAP Query policy, the recomended approach is to use paged results.

      Note that this ample, which queries for all users that have a value for the mail attribute. uses a page size of 10, not really an optimal use of either the server or of the network, but merely just to demonstrate paging.

      Also, the usual security comments apply, you shouldn't hardcode credentials in an application, authentication should either use Kerberos (JAAS & GSSAPI) or if using simple authentication, secured using SSL or TLS, and and any sensitive information communicated between the client and the server should also take place over SSL or TLS.
      .
      /**
       * paged.java
       * 5 July 2001
       * Sample JNDI application that performs a paged search.
       * 
       */
      
      import java.util.Hashtable;
      import java.util.Enumeration;
      
      import javax.naming.*;
      import javax.naming.directory.*;
      import javax.naming.ldap.*;
      import com.sun.jndi.ldap.ctl.*;
      
      class paged {
      
      public static void main(String[] args) {
      
      
                Hashtable env = new Hashtable();
                String adminName = "CN=Administrator,CN=Users,DC=antipodes,DC=com";
                String adminPassword = "XXXXXXXX";
                String searchBase = "DC=antipodes,DC=com";
                String searchFilter = "(&(objectClass=user)(mail=*))";
                
                env.put(Context.INITIAL_CONTEXT_FACTORY,"com.sun.jndi.ldap.LdapCtxFactory");
      
                //set security credentials, note using simple cleartext authentication
                env.put(Context.SECURITY_AUTHENTICATION,"simple");
                env.put(Context.SECURITY_PRINCIPAL,adminName);
                env.put(Context.SECURITY_CREDENTIALS,adminPassword);
                          
                //connect to my domain controller
                env.put(Context.PROVIDER_URL, "ldap://mydc.antipodes.com:389");
                try {
      
                     // Create the initial directory context
                     LdapContext ctx = new InitialLdapContext(env,null);
               
                     // Create the search controls           
                     SearchControls searchCtls = new SearchControls();
                
                     //Specify the attributes to return
                     String returnedAtts[]={"sn","givenName","mail"};
                     searchCtls.setReturningAttributes(returnedAtts);
                
                     //Specify the search scope
                     searchCtls.setSearchScope(SearchControls.SUBTREE_SCOPE);
      
                     //Set the page size and initialize the cookie that we pass back in subsequent pages
                     int pageSize = 10;
                     byte[] cookie = null;
      
                     //Request the paged results control
                     Control[] ctls = new Control[]{new PagedResultsControl(pageSize)};
                     ctx.setRequestControls(ctls);
      
                     //initialize counter to total the results
                     int totalResults = 0;
      
                     // Search for objects using the filter
      
                     do {
                          NamingEnumeration results = ctx.search(searchBase, searchFilter, searchCtls);
      
                            // loop through the results in each page
      
                            while (results != null && results.hasMoreElements()) {
                          SearchResult sr = (SearchResult)results.next();
      
                          //print out the name 
                          System.out.println("name: " + sr.getName());
      
                          //increment the counter
                          totalResults++;     
      
                            }
           
           
                     // examine the response controls
                     cookie = parseControls(ctx.getResponseControls());
      
                            // pass the cookie back to the server for the next page
                     ctx.setRequestControls(new Control[]{new PagedResultsControl(pageSize, cookie, Control.CRITICAL) });
      
                     } while ((cookie != null) && (cookie.length != 0));
      
           
                     ctx.close();
      
                     System.out.println("Total entries: " + totalResults);
      
      
                    } 
                catch (NamingException e) {
                       System.err.println("Paged Search failed." + e);
                   }     
                catch (java.io.IOException e) {
                     System.err.println("Paged Search failed." + e);
                      } 
           }
      
           static byte[] parseControls(Control[] controls) throws NamingException {
      
                byte[] cookie = null;
      
                if (controls != null) {
      
                         for (int i = 0; i < controls.length; i++) {
                          if (controls[i] instanceof PagedResultsResponseControl) {
                               PagedResultsResponseControl prrc = (PagedResultsResponseControl)controls;
                               cookie = prrc.getCookie();
                               System.out.println(">>Next Page \n");
                          }
                     }
                }

                return (cookie == null) ? new byte[0] : cookie;
           }
      }




      Note that if we had requested a multi-valued attribute instead of the surname (sn), firstname (givenName) and e-mail (mail) attributes, and if that multi-valued had more than 1000 values, then only the first 1000 values would have been returned, irrespective of whether paged results had ben used.

      Range retrieval is more evident when retrieving the list of members from a group. The list of members in a group is contained in the member attribute. If there are more than 1000 values in the member attribute, then you must use range retrieval to return all the members.

      Information on range retrieval can be found at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adsi/adsi/attribute_range_retrieval.asp

      The following JNDI sample illustrates the retrieval of members from a group, using range retrieval. In this case querying for the list of members of a group called "All Research" in the Active Directory.
      /**
       * range.java
       * December 2004
       * Sample JNDI application to demonstrate range retrieval 
       * 
       */
      
      import java.util.Hashtable;
      import javax.naming.*;
      import javax.naming.ldap.*;
      import javax.naming.directory.*;
      import com.sun.jndi.ldap.ctl.*;
      
      //if we were going to do this properly, should check that the server supports
      //the range retrieval control 1.2.840.113556.1.4.802 !
      
      public class range     {
           public static void main (String[] args)     {
           
                Hashtable env = new Hashtable();
                String adminName = "CN=Administrator,CN=Users,DC=ANTIPODES,DC=COM";
                String adminPassword = "XXXXXX";
                
                env.put(Context.INITIAL_CONTEXT_FACTORY,"com.sun.jndi.ldap.LdapCtxFactory");
                //set security credentials, note using simple cleartext authentication
                env.put(Context.SECURITY_AUTHENTICATION,"simple");
                env.put(Context.SECURITY_PRINCIPAL,adminName);
                env.put(Context.SECURITY_CREDENTIALS,adminPassword);
                          
                //connect to my domain controller
                env.put(Context.PROVIDER_URL,"ldap://mydc.antipodes.com:389");
                
                try {
      
                     // Create the initial directory context
                     LdapContext ctx = new InitialLdapContext(env,null);
                
                     // Create the search controls           
                     SearchControls searchCtls = new SearchControls();
                
                     //Specify the search scope
                     searchCtls.setSearchScope(SearchControls.SUBTREE_SCOPE);
      
                     //specify the LDAP search filter
                     String searchFilter = "(&(objectClass=group)(CN=All Research))";
      
                     //Specify the Base for the search
                     String searchBase = "DC=antipodes,DC=com";
      
                     //initialize counter to total the group members and range values
                     int totalResults = 0;
                     int Start = 0;
                     int Step = 10;
                     int Finish = 9;
                     boolean Finished = false;
                     String Range;
      
                     //loop through the query until we have all the results
                     while (!Finished) {          
                     
                          //Specify the attributes to return
                          Range = Start + "-" + Finish;
                          String returnedAtts[]={"member;Range=" + Range};
                          searchCtls.setReturningAttributes(returnedAtts);
                
                          //Search for objects using the filter
                          NamingEnumeration answer = ctx.search(searchBase, searchFilter, searchCtls);
      
                          //Loop through the search results
                          while (answer.hasMoreElements()) {
                               SearchResult sr = (SearchResult)answer.next();
      
                               System.out.println(">>>" + sr.getName());
      
                               //Print out the members
       
                               Attributes attrs = sr.getAttributes();
                               if (attrs != null) {
      
                                    try {
                                         for (NamingEnumeration ae = attrs.getAll();ae.hasMore();) {
                                              Attribute attr = (Attribute)ae.next();
      
                                              //check if we are finished
                                              if (attr.getID().endsWith("*")) {
                                                   Finished=true;
                                              }
      
                                              System.out.println("Attribute: " + attr.getID());
                                              for (NamingEnumeration e = attr.getAll();e.hasMore();totalResults++) {
      
                                                   System.out.println("   " + totalResults + ". " + e.next());
                                              }
                                         }
      
                                    }      
                                    catch (NamingException e)     {
                                         System.err.println("Problem printing attributes: " + e);
                                    }
                          
                                    Start = Start + Step;
                                    Finish = Finish + Step;
      
                               }
                          }
                     }
      
                      System.out.println("Total members: " + totalResults);
                     ctx.close();
      
                } 
                catch (NamingException e) {
                     System.err.println("Problem searching directory: " + e);
                }
           }
      }
        • 1. Re: JNDI, Active Directory, Paging and Range Retrieval
          843793
          Thanks for the code samples. When I use your code (class paged) then I receive the following error:
          Paged Search failed.javax.naming.OperationNotSupportedException: [LDAP: error code 12 - critical extension is not recognized]; remaining name 'ou=People,dc=umich,dc=edu'

          What can I do if the server dosn't support the operation? Is there an other possibility?

          Thanks,
          Peter
          • 2. Re: JNDI, Active Directory, Paging and Range Retrieval
            843793
            In the above code sample, what ldap server is being used. We tried this on Sun One Directory 5.2 and it did not work.
            • 3. Re: JNDI, Active Directory, Paging and Range Retrieval
              800477
              Wow, It seems like a software company regularly criticised as a marketing company has failed in creating brand recognition.

              I would have thought that the title of this post would give a subtle hint as to what directory was being used: Active Directory which is the directory included in Windows 2000 Server, Windows Server 2003.
              • 4. Re: JNDI, Active Directory, Paging and Range Retrieval
                843793
                I need to get the first 10 sorted users of a group. I can get the first 10 using RANGE option. Is there a way to sort users and then return the first 10?

                Thanks
                • 5. Re: JNDI, Active Directory, Paging and Range Retrieval
                  800477
                  To the best of my knowledge, unfortunately not.
                  The member attribute is a multi-valued attribute and I don't belive that Active Directory stores these values in any particular sort order and hence can't guarantee the order in which they are returned to a user application.

                  A potential work around, which may also have its own pitfalls is to search for
                  users, who are members of a particular group. Your ldap query would be something like:
                  (&(objectClass=user)(memberOf=cn=Scientists,ou=Research,dc=antipodes,dc=com))
                  In this case, you should be able to specify the sort order on the user names.
                  Note that the syntax of the group name (memberOf attribute) is the full distinguished name.

                  One of the pitfalls would be if the groups contains members of other domains. The results of an ldap query may appear incomplete. You may want to refer to a previous posting describing the Global Catalog and LDAP referrals.
                  • 6. Re: JNDI, Active Directory, Paging and Range Retrieval
                    843793
                    Does anyone know if their is another ldap server besides active directory that fully supports paging range?

                    We're probably going to use the Active Directory Application Mode. The export/import is problematic, adding to the schema is confusing and it's tightly coupled to the silly operating system. I definitely feels like i'm tying myself adam. Openldap is way easier to use it just doesn't support the sort,paging, and range stuff.

                    I think it's sad that Sun One directory doesn't support it and you have to pay for that server.
                    • 7. Re: JNDI, Active Directory, Paging and Range Retrieval
                      800477
                      I would be interested in understanding on why you find it difficult to extend the schema and why it is difficult to import and export.

                      For import/export you could quite easily use LDAP Data Interchange Format (LDIF) to perform bulk imports and exports to and from ADAM.

                      Similary to extend the schema, you could use LDIF import files, the Windows Server 2003 tools (ADSIEdit) or programmatically (for example a Java/JNDI app).

                      You can find further information at:
                      http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ad/ad/extending_the_schema.asp

                      Or if I had some time, I could demonstrate how the extend the schema using Java/JNDI. It ain't rocket science.
                      • 8. Re: JNDI, Active Directory, Paging and Range Retrieval
                        843793
                        Hi and thanks to all your precious information about JNDI uses.
                        I only want to know if I can acceed to Active Directory objects (only 'users' and 'groups') through JNDI without having an administrative account. I only want to read these objects, not to modify them.
                        Thanks again, your tips have already helped me
                        • 9. Re: JNDI, Active Directory, Paging and Range Retrieval
                          843793
                          I have found your example code to be very useful. However, I am puzzled
                          by some aspects of the member range example, particularly the loop
                          control.

                          There are two things that seem odd.

                          First the setting of "Finished" to true. If the "while (answer.hasmore)"
                          does not get entered or the "if (attrs != null)" is never true how will the
                          nested loops get to complete?
                          Perhaps there needs to be some additional termination conditions?

                          Second the increment for "Start" and "Finish" seem to be at the wrong level.
                          As with "Finished", if either of the above conditions are not true "Start" and
                          "Finish" will not get incremented. Alternatively, if "while (answer.hasmore)"
                          is entered more than once they will be incremented too frequently.

                          Perhaps I am missing something here!

                          Many thanks
                          • 10. Re: JNDI, Active Directory, Paging and Range Retrieval
                            800477
                            The only thing you're missing is that I've never admitted to being a Java programmer !

                            The reason for selecting the sub-optimal range (in this case 10) is just to simply show how range retrieval works.

                            Believe it or not, the nested loops do actually get to complete. However in response to your question and deciding to check my sanity I discovered two serious flaws with my sample:
                            1. What happens if the group is not found ?
                            2. What happens if the group does not contain any members ?

                            I guess you could say it was a design rather than coding flaw, because I wasn't really interested in those two boundary cases when I wrote the sample (That's my excuse and I'm sticking to it !).

                            To solve those two problems replace
                            while (answer.hasMoreElements())
                            with
                            while (answer.hasMore())
                            and catch the PartialResultException and handle appropriately.
                            • 11. Re: JNDI, Active Directory, Paging and Range Retrieval
                              843793
                              Thanks for the code samples. When I use your code (class paged) then I receive the following error:
                              Paged Search failed.javax.naming.OperationNotSupportedException: [LDAP: error code 12 - critical extension is not recognized]; remaining name 'ou=People,dc=umich,dc=edu'

                              What can I do if the server dosn't support the operation? Is there an other possibility?

                              Thanks,
                              Peter
                              • 12. Re: JNDI, Active Directory, Paging and Range Retrieval
                                800477
                                Are you using Active Directory ?

                                Both Win2K & Win2K3 support LDAP page results.
                                • 13. Re: JNDI, Active Directory, Paging and Range Retrieval
                                  843793
                                  Hi,
                                  I want to use this code. I am using JDK 1.4.
                                  How can I use PagedResultsControl. This class does not exist in javax.naming.ldap package. How can I access com.sun.jndi.ldap.ctl.* package. Do I need to include any Jar file?

                                  Thank you very much for any help in this regards.
                                  mqassim
                                  • 14. Re: JNDI, Active Directory, Paging and Range Retrieval
                                    800477
                                    Hmmm...
                                    Maybe I'm getting a bit ahead of myself these days (playing with Windows Server "Longhorn", .NET Framework 3.0, Windows Communications Foundation and trying to get Java 6 working against them).

                                    IIRC, a number of controls such as the Paged Results control were included in Sun's ldap booster pack (ldapbp.jar).

                                    Good luck.
                                    1 2 Previous Next