6 Replies Latest reply: Feb 21, 2014 5:30 AM by Pierre Timmermans RSS

RESTFul URI parameter limitations/bugs

TobiasE Newbie
Currently Being Moderated
Hi,

I've run into some limitations (or bugs, don't know which is the most appropriate term at this point) regarding URI template parameters for RESTful Services. Here are a few examples to illustrate:

First limitation:

To start with, I define the following Resource Template: *example?p1={p1}&p2={p2}*
I use the following query as the GET handler: select :p1, :p2 from dual
This works fine, the URL example?p1=foo&p2=bar gets a json response like this: {":1": "foo",":2": "bar"}

I then create a second Resource Template with only the first parameter from the previous template: *example?p1={p1}*
I use the following query as the GET handler: select :p1 from dual
This works as expected, however now the previous template returns the following: {":1": "foo&p2=bar"}, i.e. the part of the URI string after p1= gets parsed as a single parameter.

Second limitation:

I define the following resource template: *example1/{p1}*
I use the following query as the GET handler: select :p1 from dual
Works fine.

I then define the following resource template: *example1/{p1}/example2*
I use the same query again as the GET handler: select :p1 from dual. This time the whole part of the URI string from {p1} forward gets parsed as the p1 parameter, which means it returns {":1":"foo/example2"} rather than {":1":"foo"}. However, if I remove the example1/{p1} template, the example1/{p1}/example2 works as expected.

In both cases, it seems like the rules for URI templates are excessively strict regarding URI similarities with other templates. I've been unable to find any reference to this in the documentation, other than a vague reference to a lack of support for "optional parameters".

Assuming these are known limitations rather than bugs, might they be removed in a future release?

Versions:

APEX Listener 2.0.1
WLS 10.3.6 on Windows 2008 R2 (64-bit)
Oracle DB 11.2.0.3
APEX 4.2.1

Thanks,
Tobias
  • 1. Re: RESTFul URI parameter limitations/bugs
    Colm Divilly Journeyer
    Currently Being Moderated
    TobiasE wrote:
    Hi,

    I've run into some limitations (or bugs, don't know which is the most appropriate term at this point) regarding URI template parameters for RESTful Services. Here are a few examples to illustrate:

    First limitation:

    To start with, I define the following Resource Template: *example?p1={p1}&p2={p2}*
    I use the following query as the GET handler: select :p1, :p2 from dual
    This works fine, the URI example?p1=foo&p2=bar gets a json response like this: {":1": "foo",":2": "bar"}

    I then create a second Resource Template with only the first parameter from the previous template: *example?p1={p1}*
    I use the following query as the GET handler: select :p1 from dual
    This works as expected, however now the previous template returns the following: {":1": "foo&p2=bar"}, i.e. the part of the URI string after p1= gets parsed as a single parameter.
    This is when you need to use the priority value to help the listener choose the most appropriate URI Template. Hopefully I can explain what is actually happening and make this a bit clearer.

    The important thing to realise is that a URI Template matches any characters, characters which you might assume are significant to the matching algorithm, the characters which act as delimiters in the URI and query string syntax are not treated as delimiters, they have no special significance, *&*,*=*,*/*,*?* are all treated just the same as any other character . If you look at the final URI Template RFC [1] you can get a feel for how involved it gets trying to specify a URI Template syntax that can robustly match against all common URI patterns, and actually describe a syntax that is aware of the significance of these delimiters.

    So it's your second Resource Template that is matching both URIs and the first one is not getting invoked at all.

    *example?p1={p1}&p2={p2}* matches example?p1=foo&p2=bar, i.e. *{p1}* is mapped to foo and *{p2}* is mapped to bar
    *example?p1={p1}* matches both example?p1=foo and example?p1=foo&p2=bar, i.e. in the first case *{p1}* is mapped to foo and in the second case *{p1}* is matched to: foo&p2=bar

    In plain english the second template means ' match *{p1}* against any text occuring after: example?p1= '

    So when faced with trying to find the appropriate resource template for the URI, listener finds two matching resource templates, which one does it choose?

    The answer is it chooses the one with the highest priority, but if they are both the same priority, then (and admittedly this isn't documented anywhere, it may be subject to change in future releases) it chooses the resource template that requires the least number of parameters to achieve a match (this is just a basic heuristic to help make a choice, it's a bit of a coin toss whether it matches what the developer might have intended).

    So the solution is to give the Resource Templates different priorities, it's up to you to figure out the order in which you want listener to match your templates that match a given URI. Give the template you want checked first the highest priority, the template you want checked second a lower priority.

    So to fix this problem give *example?p1={p1}&p2={p2}* priority of 9 (or any value > the default of 0) and that should solve the problem
    Second limitation:

    I define the following resource template: *example1/{p1}*
    I use the following query as the GET handler: select :p1 from dual
    Works fine.

    I then define the following resource template: *example1/{p1}/example2*
    I use the same query again as the GET handler: select :p1 from dual. This time the whole part of the URI string from {p1} forward gets parsed as the p1 parameter, which means it returns {":1":"foo/example2"} rather than {":1":"foo"}. However, if I remove the example1/{p1} template, the example1/{p1}/example2 works as expected.
    This is the same issue again really, change the priority of *example1/{p1}/example2* to be greater than the priority of *example1/{p1}*.
    In both cases, it seems like the rules for URI templates are excessively strict regarding URI similarities with other templates. I've been unable to find any reference to this in the documentation, other than a vague reference to a lack of support for "optional parameters".

    Assuming these are known limitations rather than bugs, might they be removed in a future release?
    So as you can see these are known limitations, rather than bugs.

    Without committing to anything, I would certainly like to support more of the final URI Template RFC, but as you can see if you read it, there is a fair bit of complexity involved in doing so. A large part of the problem is that URI Templates were really envisaged for the inverse of how we use them (and how other server side frameworks like RESTlet and .NET use them). URI Templates as specified are primarily intended to be used on the client side to form request URIs from a URI Template and parameter values. We're using them the other way round, trying to decode request URIs to match against templates and thereby extract parameter values (to pass into our queries).

    For now I'm not in a hurry to add more support, until we see compelling cases where the above prioritization mechanism cannot solve the matching problem, so would be glad to hear back from you if find scenarios that are not supportable at the moment.

    Regards,
    Colm

    [1] http://tools.ietf.org/html/rfc6570
  • 2. Re: RESTFul URI parameter limitations/bugs
    TobiasE Newbie
    Currently Being Moderated
    Hi Colm,

    thank you very much for your thorough reply. Everything's working great when I set different priority values, I hadn't realized that's what the priority field is for (although now that I check, it's actually stated pretty clearly in the SQL Workshop Guide for APEX 4.2). Just one minor niggle: in the APEX help text for the Priority field, it says "Lower numbers are evaluated first.", which seems to be the opposite of what actually happens.

    Thanks again,
    Tobias
  • 3. Re: RESTFul URI parameter limitations/bugs
    Colm Divilly Journeyer
    Currently Being Moderated
    TobiasE wrote:
    thank you very much for your thorough reply. Everything's working great when I set different priority values, I hadn't realized that's what the priority field is for (although now that I check, it's actually stated pretty clearly in the SQL Workshop Guide for APEX 4.2). Just one minor niggle: in the APEX help text for the Priority field, it says "Lower numbers are evaluated first.", which seems to be the opposite of what actually happens.
    Thanks for spotting that, must have got changed at some point, strangely a lot of software folks are conditioned to think of 0 as highest priority (e.g. a P0 defect is the highest priority kind), so it's hard to counter that sometimes :)
  • 4. Re: RESTFul URI parameter limitations/bugs
    Simon Collins Newbie
    Currently Being Moderated

    Having read the above thread I'm still not 100% sure whether the API presently supports optional paramaters. Ie I've got URI templates set up at the moment with 5 or 6 variables that could be passed into some of them. At the moment I'm assuming that it is mandatory to state every variable when calling the method (even if you actually only need in some calls to set 1 or 2 variables). Is this the case?

     

    Thanks

     

    Simon

  • 5. Re: RESTFul URI parameter limitations/bugs
    PeterT Newbie
    Currently Being Moderated

    Hi.

     

    A couple of questions/thoughts regarding the parsing of URI templates. I am a bit new in this area so please bear with me.

     

    1)

    From the above post:

    "The answer is it chooses the one with the highest priority, but if they are both the same priority, then (and admittedly this isn't documented anywhere, it may be subject to change in future releases) it chooses the resource template that requires the least number of parameters to achieve a match..."

     

    Why is it implemented like this?

     

    If one has two different requests like

     

    a) example?p1=foo&p2=bar

     

    b) example?p1=foo

     

    then, to me, request a) is more specified than request b) and should therefore be used (otherwise, why did caller supply parameter p2?).

     

    2)

    From the above post:

    "The important thing to realise is that a URI Template matches any characters, characters which you might assume are significant to the matching algorithm, the characters which act as delimiters in the URI and query string syntax are not treated as delimiters, they have no special significance, *&*,*=*,*/*,*?* are all treated just the same as any other character ."

     

    Without the time to dig in to all the details in APEX REST or RFC 6570, I believe that the way I have to implement the different kind of templates gets kind of tedious. For some reason (I can't figure out why), the templates have to make exact matches of all the possible URI's? If one compares this to some ordinary web server parsing, this is not the case I believe. I am free to request example?p1=foo&p4=bar, or example?p2=bar&p1=foo&p7=foobar etc. The different requests get parsed and the parameters get expanded.

     

    The web server approach sounds like this snippet from the RFC:

    RFC 6570:

    "The URI Template syntax has been designed to carefully balance the

      need for a powerful expansion mechanism with the need for ease of

      implementation. The syntax is designed to be trivial to parse while

      at the same time providing enough flexibility to express many common

      template scenarios."

     

    In my latest APEX REST application I hade like 7-9 parameters, and the number of templates should have been too large for me. Instead I implemented the example?{params} template and parsed the parameters myself by something like

     

    v_array1 := apex_util.string_to_table(p_params, '&');
    for i in 1..v_array1.count loop
        v_array2 := apex_util.string_to_table(v_array1(i), '=');
        case v_array2(1)
            when 'p1' then p_1 := v_array2(2);
            when 'p2' then p_2 := v_array2(2);
            when 'p3' then p_3 := v_array2(2);
            when 'p4' then p_4 := v_array2(2);
            ...
        end case;
    end loop;

     

    In the case of the example?... template, could one not in the APEX REST api have some list of possible parameters that get taken care of, like the header binding?

     

    Br Peter

  • 6. Re: RESTFul URI parameter limitations/bugs
    Pierre Timmermans Newbie
    Currently Being Moderated

    I was about to drop the apex listener restful service when I found this blog that solved all my issues, this template matching based on priority subtility should really be highlighted in the doc. I am glad to find this very good explanation. What should also be improved in the doc, in fact in the nice demo application, is to demonstrate with POST/PUT/DELETE and not only with GET

     

    What I could not implement (I had to use workaround) was to change the http result code (for ex. 201 for resource created). Also when doing a post, I would like to send back the update record in the response but this is not possible to have output parameters with the same name as the input variables sent with the request. There are in/out parameters but only in the http headers.

     

    Thanks and regards,

    Pierre

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points