Forum Stats

  • 3,734,238 Users
  • 2,246,919 Discussions
  • 7,857,196 Comments

Discussions

Oauth token - Reuse active token

We have some client apps that do not maintain token state so they always call the ORDS /oauth/token endpoint and use the Bearer token to authorize for the subsequent ORDS API call. Since each ORDS bearer token is valid for 3600 seconds. Lots of unnecessary tokens are generated. Although the ORDS_HOUSEKEEPING_JOB cleans up unused tokens, this is wasteful

Questions

  1. Can the ORDS /oauth/token endpoint to re-use the same token IF the client provides it in a custom x-token or such header, instead of generating a new token?
  2. Can the expires_in attribute of the token be configured or is it hard-wired to 3600?

Thanks

Answers

  • Billy Verreynne
    Billy Verreynne Member Posts: 28,280 Red Diamond

    Is there a problem resource or performance or security wise with lots of tokens?

    Not liking that many tokens is not exactly a technical problem needing to be addressed.

    As for your 2nd question, refer to https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/20.4/aelig/about-REST-configuration-files.html for the ORDS config.xml file parameters.

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy
    edited May 18, 2021 12:08PM

    Suppose there's a query that takes 3 seconds to execute. The overall application is well within the performance SLA so there's no technical issue but I know that the query can be optimized to run in under a second. Not liking something is very often a itch to scratch is all but in my experience it has benefits

    The matter at hand is that we have 2 Linux nodes running ORDS standalone fronted by a load balancer. The ORDS APIs are called by an API gateway server (WSO2) that behaves like I described above (/oauth/token to get bearer token followed by the actual API). This suceceds 6 times out of 10. The other times it gets a HTTP/404 with a [Could not find any dispatcher to handle request]. This tells me that ORDS is receiving the request but the intermittent nature of the error is tricky to nail down. If there are network connectivity issues, none of the calls would work. The fact that it is intermittent makes me think some resource limits are being hit, hence the question. I reached out to @Kris Rice-Oracle on email few days back

    My 2nd question - I was able to locate the documentation but I could not find a parameter that controls the oauth token expiry time, did you?

  • Billy Verreynne
    Billy Verreynne Member Posts: 28,280 Red Diamond

    The other times it gets a HTTP/404 with a [Could not find any dispatcher to handle request]. 

    Sounds like an ORDS connection thread pool, or database server session issue.

    In our case I have seen browser connection madness with clients trying up to 17 HTTP GETs (using APEX URLs) per seconds. No sweat, no errors, on the Apache Tomcat, ORDS, or the database side.

    According to the user agent identification in the APEX log, the Russian Yandex bot and WhatsApp on a Nokia were to blame for 10's of 1000's calls to APEX over a periods of a couple of hours each. Not DoS attacks, as it seems to be more of a browser agent bug stuck-in-a-loop issue.

    Point is that at peaking at 17 browser calls/second to ORDS and serviced by APEX, there were no connectivity issues, no performance issues, and no errors reported.

    Thus the Tomcat-ORDS-APEX-Oracle-db architecture is fairly robust in my view.

    My 2nd question - I was able to locate the documentation but I could not find a parameter that controls the oauth token expiry time, did you?

    Nope, but then I have not looked at the latest ORDS version - or at the Java class architecture used for OAuth2 by ORDS that could be supporting its own distinct set of parameters.

    But I would not expect a performance issue with OAuth tokens as long as the memory available suffices - that said we do not use OAuth on our side as the only JSON services we expose are at this stage for limited company internal use.

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Sounds like an ORDS connection thread pool, or database server session issue.

    How can I troubleshoot this? Again, if there were a connection pool issue or invalid password, proxy user or any of those things, the connection pool would not be established to begin with. A intermittent error indicates some sort of capacity problem but where would that be reported? The ORDS service logs (nohup.out or such) are clean, no errors. The Jetty access log only shows HTTP 200, 301, 302 and such, it does not record HTTP 404 which seems to be trapped by ORDS before it hits Jetty.

    This has always been my frustration with ORDS, as I mentioned in the past. As I said, we are so used to the high level of instrumentation and maturity that Oracle Database provides (setting events, tkprof, _parameters, etc.) so it is quite frustrating when debug.PrintDebugToScreen is all we have to work with in ORDS and all that does is dump the same generic stack trace when the root cause is wildly different.

    The 404/Unable to find dispatcher is one of those catch-all errors that could have wildly different root causes. Kris pointed me to some debugging facilities which I enabled but even at the lowest (DEBUG/FINEST) levels, I could not spot anything that could explain the intermittent 404 errors.

    Any ideas appreciated.

  • Billy Verreynne
    Billy Verreynne Member Posts: 28,280 Red Diamond

    I would be hesitant running ORDS in standalone mode. Do not like Java much and prefer it to be managed via a container such as Tomcat.

    I would run tcpdump on the ORDS http port, and analyse the traffic using WireShark. With this one can at least identify the exact HTTP client calls (with HTTP header and payload) that result in a 404.

    One then can compare these HTTP calls for commonalities, and manually craft duplicate calls using curl in an attempt to further isolate the problem.

    If these HTTP 404 calls do not differ from the HTTP 200 calls, and curl testing does not result in 404s, then it would seem that the issue is an ORDS bug and not an issue triggered by something in the HTTP call.

  • thatJeffSmith-Oracle
    thatJeffSmith-Oracle Distinguished Product Manager Posts: 7,701 Employee

    <entry key="security.oauth.tokenLifetime">3600</entry>

    Billy Verreynne
  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy
    edited May 19, 2021 5:52PM

    Jeff - Thanks.

    This is one of my pet peeves about ORDS. For a mature product, the documentation is not updated. Could you share a complete list of all the configuration parameters used by ORDS?

    Does a value of 0 indicate a one time use token or always valid token?

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Not liking Java is not a technical problem needing to be addressed. Sorry couldn't resist. As I understand it, standalone mode is supported for Production deployments and Jetty is a robust well regarded web server.


    I am in touch with Kris on this issue and I am baffled. The same curl call works from one server intermittently but consistently succeeds from other servers. When it fails, the HTML document returned by ORDS shows the 404 error which is generic in nature and not helpful to identify root cause.

    I'll try tcpdump and such but since ORDS receives the request successfully that rules out network issues. Intermittent errors point to capacity issues but I don't see them reported in ORDS logs.

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    One more comment - this is not just an issue with the token endpoint. API unprotected by any privilege also show the same 404 error. Basically the pattern is that dozens of calls from the same client work fine and then they start throwing 404 for a while and then things are back to normal.

    JDBC pools are set to 100 sessions. No back end errors reported on the database side.

  • thatJeffSmith-Oracle
    thatJeffSmith-Oracle Distinguished Product Manager Posts: 7,701 Employee

     For a mature product, the documentation is not updated

    It's intentionally not documented, you're not advised to be messing with it. I only mentioned it here b/c if you search this forum history, you can find it.

    Also, if you're already talking to Kris on this topic...that would be good to know when starting the thread.

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Why are we not advised to mess with it? Different customers may have different security needs so having ability to adjust the token timeout is useful, no?

    I did mention that I'm working with Kris in my second post on this thread.

    When ORDS throws a 404, what logs can I inspect to determine the exact reason?

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    I came across this article. I am using customURL in my defaults.xml to identify my database.

    Could this be the cause of the 404s I'm seeing? I'm using ORDS 20.4 and the article says the bug has been fixed in 19.4 but can you please check?

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Suppose my JDBC connection pool settings are inadequate. Min, max, idle, etc. Would this also manifest as a 404/URLNotfoundException I'm seeing? How can I verify this and what log level would show this?

  • thatJeffSmith-Oracle
    thatJeffSmith-Oracle Distinguished Product Manager Posts: 7,701 Employee

    When ORDS throws a 404, what logs can I inspect to determine the exact reason?

    Enable debug for ords, and observe the logs.

    AFAIK 404's are only returned when you ask for something ORDS isn't able to resolve, it wouldn't send a 404 if the pool isn't available for another request for example.

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Enable debug for ords, and observe the logs.

    I did this already. There is nothing in the logs. I followed the logging steps at https://is.gd/ords_debug section Configuring Logging output in Standalone Mode but there was nothing in the logs but the logs are very noisy so I may have missed it. Is there a unique request ID in the 404 output (HTML document) that can be used to locate the corresponding log entries?

  • thatJeffSmith-Oracle
    thatJeffSmith-Oracle Distinguished Product Manager Posts: 7,701 Employee

    Not that I can tell, not like there are for other types of error responses

  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy
    edited May 21, 2021 7:05PM

    Is it possible that ORDS throws a 404 error and does not log it? That's what I am seeing, just want to make sure I'm not losing my mind. The other odd thing is that the error trace refers to Tomcat/Catalina libraries which is strange since I am using ORDS in standalone mode

    Debug Trace

    [TE] POST /apex/edbrest/oauth/token start: 2021-05-17T18:45:01.665Z duration: 1556ms

     

     


    URLMappingNotFoundException [statusCode=404, reasons=[The request could not be mapped to any database. Check the request URL is correct, and that URL to database mappings have been correctly configured]]
            at oracle.dbtools.url.mapping.filter.URLMappingFilter.doFilter(URLMappingFilter.java:125)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.auth.external.ExternalSessionFilter.doFilter(ExternalSessionFilter.java:59)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.rt.authentication.apex.ApexSessionQueryRewriteFilter.doFilter(ApexSessionQueryRewriteFilter.java:58)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.cors.CORSResponseFilter.doFilter(CORSResponseFilter.java:83)
            at oracle.dbtools.http.filters.HttpResponseFilter.doFilter(HttpResponseFilter.java:45)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.filters.AbsoluteLocationFilter.doFilter(AbsoluteLocationFilter.java:65)
            at oracle.dbtools.http.filters.HttpResponseFilter.doFilter(HttpResponseFilter.java:45)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.errors.ErrorPageFilter.doFilter(ErrorPageFilter.java:85)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.secure.ForceHttpsFilter.doFilter(ForceHttpsFilter.java:74)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.auth.ForceAuthFilter.doFilter(ForceAuthFilter.java:44)
            at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
            at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
            at oracle.dbtools.http.filters.Filters.filter(Filters.java:67)
            at oracle.dbtools.http.entrypoint.EntryPoint.service(EntryPoint.java:82)
            at oracle.dbtools.http.entrypoint.EntryPointServlet.service(EntryPointServlet.java:102)
            at oracle.dbtools.entrypoint.WebApplicationRequestEntryPoint.service(WebApplicationRequestEntryPoint.java:50)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
    


  • partlycloudy
    partlycloudy Member Posts: 8,024 Silver Trophy

    Jeff - I am happy to take this offline with you and Kris since I am already communicating with Kris over email from my work email.

Sign In or Register to comment.