7 Replies Latest reply: May 1, 2012 3:50 PM by cjoy RSS

    Lightning request to OPTIONS /dav/home/ 404 causes excessive logging

    803774
      Calendar 7u2-6.11

      We encountered a scenario where a Lightning user causes excessive logging (filled up 10GB logs in a day). These logs over and over...

      1.2.3.4 - - [18/Apr/2012:13:58:42 -0500] "PROPFIND /dav/principals/user%40domain/ HTTP/1.1" 207 762 0 "-" "Mozilla/5.0 (X11; Linux i686; rv:11.0) Gecko/20120310 Thunderbird/11.0 Lightning/1.3" 0/7026
      1.2.3.4 - - [18/Apr/2012:13:58:42 -0500] "PROPFIND /dav/home/user%40domain/ HTTP/1.1" 207 461 0 "-" "Mozilla/5.0 (X11; Linux i686; rv:11.0) Gecko/20120310 Thunderbird/11.0 Lightning/1.3" 0/11527
      1.2.3.4 - - [18/Apr/2012:13:58:42 -0500] "OPTIONS /dav/home/ HTTP/1.1" 404 23 0 "-" "Mozilla/5.0 (X11; Linux i686; rv:11.0) Gecko/20120310 Thunderbird/11.0 Lightning/1.3" 0/5073


      On the client side, this appears in the error console.

      CalDAV: Status 207 on initial PROPFIND for calendar MyCal
      CalDAV: Authentication scheme for MyCal is Basic
      CalDAV: recv: <?xml version='1.0' encoding='UTF-8'?><D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:M="urn:ietf:params:xml:ns:carddav">
      <D:response>
      <D:href>/dav/home/user@domain/</D:href>
      <D:propstat>
      <D:prop>
      <D:resourcetype><D:collection /></D:resourcetype>
      <D:owner>
      <D:href>/dav/principals/user@domain/</D:href>
      </D:owner>
      <D:supported-report-set><D:supported-report><D:report><D:principal-search-property-set /></D:report></D:supported-report><D:supported-report><D:report><D:principal-property-search /></D:report></D:supported-report><D:supported-report><D:report><D:principal-match /></D:report></D:supported-report><D:supported-report><D:report><D:expand-property /></D:report></D:supported-report><D:supported-report><D:report><D:sync-collection /></D:report></D:supported-report><D:supported-report><D:report><C:calendar-multiget /></D:report></D:supported-report><D:supported-report><D:report><C:calendar-query /></D:report></D:supported-report><D:supported-report><D:report><M:addressbook-multiget /></D:report></D:supported-report><D:supported-report><D:report><M:addressbook-query /></D:report></D:supported-report></D:supported-report-set>
      <F:getctag xmlns:F=" http://calendarserver.org/ns/ ">"1334776461000.9"</F:getctag>
      </D:prop>
      <D:status>HTTP/1.1 200 OK</D:status>
      </D:propstat>
      <D:propstat>
      <D:prop>
      <C:supported-calendar-component-set />
      </D:prop>
      <D:status>HTTP/1.1 404 Not Found</D:status>
      </D:propstat>
      </D:response>
      </D:multistatus>
      CalDAV: Collection has webdav sync support
      Warning: There has been an error reading data for calendar: MyCal. However, this error is believed to be minor, so the program will attempt to continue. Error code: DAV_DAV_NOT_CALDAV. Description: The resource at https://server.host.name/dav/home/user@domain is a DAV collection but not a CalDAV calendar
      Warning: There has been an error reading data for calendar: MyCal. However, this error is believed to be minor, so the program will attempt to continue. Error code: READ_FAILED. Description:


      User reports that Lightning was set up and working a week before these errors started.

      Does anyone know what could be triggering Lightning to experience this problem?
        • 1. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
          arnaudq
          This looks like a misconfiguration of the client. I have no idea why this would have worked before. It would be interesting to know what exact url the end user entered in Lightning.

          Filled a Lightning bug: https://bugzilla.mozilla.org/show_bug.cgi?id=746962
          • 2. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
            803774
            I will confirm the user's exact settings. He has multiple computers, so maybe I didn't get enough clarification whether his calendar was working on that particular computer.

            I gather from the notes in bugzilla that you able to reproduce this issue. Are you saying that it is normal for the server to return 404 to OPTIONS /dav/home/ requests?
            • 3. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
              arnaudq
              Absolutely (see http://tools.ietf.org/html/rfc4918#section-5.1 ) although that is irrelevant in our context:
              * the client would still behave badly if the server would return 200. The issue is with the fact that the configured uri corresponds to a webdav collection and not a caldav one.
              * no matter what the server returns, clients are not supposed to flood the server with requests.
              • 4. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
                803774
                This is an example technique to protect against this Lightning bug. Essentially, if the server process notices that a single client has requested /dav/home/something/ more than X times during its lifetime, then it issues a 403 error, which causes Lightning to break out of its loop. Keep in mind that the code below hasn't been completely tested.

                It requires:

                <ul>
                <li>Use Apache with mod_proxy in front of the CalDAV server</li>
                <li>Use mod_perl (version 2) in order to hook into the early stages of the HTTP request cycle</li>
                <li>Apache is built to use the "worker" MPM</li>
                <li>MaxRequestsPerChild is configured appropriately in proportion to what you configure "PerlSetVar block_lim" below.</li>
                </ul>

                Install this Perl module in your Perl INC path:

                <blockquote>
                package DOSProtect;

                use strict;
                use warnings;

                use Apache2::Const qw(:common :log);
                use Apache2::RequestRec;
                use Apache2::Request;
                use Apache2::Connection;
                use Apache2::Log;

                # this is a global variable that does not lose state
                # during the life of the apache process
                my %state;

                sub handler {
                my $r = shift;

                # set this in Apache config with:
                # PerlSetVar block_uri /regex/uri/(to_match)/
                # (make sure it contains a capture)
                my $block_uri = $r->dir_config('block_uri');

                # set this in Apache config with:
                # PerlSetVar block_lim num
                my $block_lim = $r->dir_config('block_lim');

                # get the URI and IP from the apache request
                my $uri = $r->uri;
                my $ip = $r->connection->remote_ip();

                # misconfiguration? - bail out
                return DECLINED unless ( $block_uri and $block_lim and $uri and $ip );

                # return 403 forbidden if the URI is requested from an IP more than the
                # limit during the life of the apache process

                if ( $uri =~ $block_uri and ++$state{$1}{$ip} >= $block_lim ) {

                $r->log_error("Requests to $uri from $ip exceeded $block_lim");
                return FORBIDDEN;

                }

                # this means that the request continues uninhibited
                return DECLINED;
                }

                1;
                </blockquote>

                In the Apache conf:

                <blockquote>
                <Location /dav/home>
                PerlAccessHandler DOSProtect
                PerlSetVar block_uri /dav/home/[^/]+/
                PerlSetVar block_lim 10
                </Location>
                </blockquote>
                • 5. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
                  cjoy
                  BTW, for ways to block a client see https://wikis.oracle.com/display/CommSuite/Calendar+Server+7+Administration+Guide#CalendarServer7AdministrationGuide-BlockingCalDAVandWCAPClients
                  • 6. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
                    803774
                    Thanks Ciny. However, we're dealing with supported CalDAV clients, so we can't block them.
                    • 7. Re: Lightning request to OPTIONS /dav/home/ 404 causes excessive logging
                      cjoy
                      I understand. Our hope was to achieve that with throttling capabilities at the Application Server level. Unfortunately I think one needs to support Glassfish 3 to get such capabilities. Will definitely consider the filed RFE as soon as possible.