Forum Stats

  • 3,759,958 Users
  • 2,251,621 Discussions
  • 7,870,883 Comments

Discussions

open tracing with jaeger and monitoring with linkerd: Kindly assist

I have a couple of microprofile microservices (helidon mp 1.4x) deployed to my oke cluster.

I am however trying to implement tracing and health checks and it's not been very easy.

Having followed the following live labs; linkerd and health checks setup and configuration

(https://oracle.github.io/cloudtestdrive/AppDev/cloud-native/livelabs/individual/kubernetes/kubernetes-complete/index.html?lab=log-capture-for-archive&nav=open)

And I have attempted to replicate what's been implemented in the code here (https://docs.oracle.com/en/learn/build_microservices_ocdb/data-centric-microservices-helidon/data-centric-microservices-helidon.html#verify-tracing) for open tracing using jaeger.

Having done everything below; my jaeger UI still fails to

  1. display any tracing
  2. list any service in my cluster


note:

  1. I have installed jaeger via linkerd thus in namespace linkerd-jaegar.
  2. I have also have nginx controller setup in my cluster
  3. Added opentracing annotations to my nginx controller deployment file and redeployed
  4. I have a partial mtls setup between my services which works using a CA signed certificate for my domain.

I have added updated the pom.xml files for my services with

<!-- Jaeger Dependencies -->
<dependency>
    <groupId>io.helidon.tracing</groupId>
    <artifactId>helidon-tracing</artifactId>
</dependency>
<dependency>
    <groupId>io.helidon.tracing</groupId>
    <artifactId>helidon-tracing-jaeger</artifactId>
</dependency>


my application.yaml (& microprofile-config.yaml)

tracing:
  service: metro-acctusr-svc
  protocol: https
  host: collector.linkerd-jaeger
  port: 55678
  path: /api/traces
  propagation: b3
  log-spans: true
  max-queue-size: 42
  flush-interval-ms: 10001
  sampler-type: const
  sampler-param: 1
  components:
    tracing:
      enabled: false

my deployment.yaml file template annotation

template:
  metadata:
    labels:
      app: metro-accountuser-service
    annotations:
      linkerd.io/inject: enabled
      config.linkerd.io/trace-collector: collector.linkerd-jaeger:55678
      config.alpha.linkerd.io/trace-collector-service-account: collector


Linkerd jaeger initially automatically injected the following into my pods and thus, how i obtained my collector url and ports

linkerd-proxy:
      Environment:
      LINKERD2_PROXY_LOG:                                       warn,linkerd=info
      LINKERD2_PROXY_LOG_FORMAT:                                plain
      LINKERD2_PROXY_DESTINATION_SVC_ADDR:                      linkerd-dst-headless.linkerd.svc.cluster.local.:8086
      LINKERD2_PROXY_DESTINATION_PROFILE_NETWORKS:              10.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16
      LINKERD2_PROXY_INBOUND_CONNECT_TIMEOUT:                   100ms
      LINKERD2_PROXY_OUTBOUND_CONNECT_TIMEOUT:                  1000ms
      LINKERD2_PROXY_CONTROL_LISTEN_ADDR:                       0.0.0.0:4190
      LINKERD2_PROXY_ADMIN_LISTEN_ADDR:                         0.0.0.0:4191
      LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR:                      127.0.0.1:4140
      LINKERD2_PROXY_INBOUND_LISTEN_ADDR:                       0.0.0.0:4143
      LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES:              svc.cluster.local.
      LINKERD2_PROXY_INBOUND_ACCEPT_KEEPALIVE:                  10000ms
      LINKERD2_PROXY_OUTBOUND_CONNECT_KEEPALIVE:                10000ms
      LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION:  25,443,587,3306,5432,11211
      _pod_ns:                                                  cerpsys-stage (v1:metadata.namespace)
      _pod_nodeName:                                             (v1:spec.nodeName)
      LINKERD2_PROXY_DESTINATION_CONTEXT:                       {"ns":"$(_pod_ns)", "nodeName":"$(_pod_nodeName)"}
      LINKERD2_PROXY_IDENTITY_DIR:                              /var/run/linkerd/identity/end-entity
      LINKERD2_PROXY_IDENTITY_TRUST_ANCHORS:                    -----BEGIN CERTIFICATE-----
                                                                RnzdrRKwmC4NpdjSXXfQxhlbA3H3odNjWpkCIFx6LKLn1ksniDAkh/WcTkq5Uq5b
                                                                TNg1TesMpyNbRp+0
                                                                -----END CERTIFICATE-----
      LINKERD2_PROXY_IDENTITY_TOKEN_FILE:                       /var/run/secrets/kubernetes.io/serviceaccount/token
      LINKERD2_PROXY_IDENTITY_SVC_ADDR:                         linkerd-identity-headless.linkerd.svc.cluster.local.:8080
      _pod_sa:                                                   (v1:spec.serviceAccountName)
      _l5d_ns:                                                  linkerd
      _l5d_trustdomain:                                         cluster.local
      LINKERD2_PROXY_IDENTITY_LOCAL_NAME:                       $(_pod_sa).$(_pod_ns).serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)
      LINKERD2_PROXY_IDENTITY_SVC_NAME:                         linkerd-identity.$(_l5d_ns).serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)
      LINKERD2_PROXY_DESTINATION_SVC_NAME:                      linkerd-destination.$(_l5d_ns).serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)
      LINKERD2_PROXY_TAP_SVC_NAME:                              tap.linkerd-viz.serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)
      LINKERD2_PROXY_TRACE_ATTRIBUTES_PATH:                     /var/run/linkerd/podinfo/labels
      LINKERD2_PROXY_TRACE_COLLECTOR_SVC_ADDR:                  collector.linkerd-jaeger:55678
      LINKERD2_PROXY_TRACE_COLLECTOR_SVC_NAME:                  collector.linkerd-jaeger.serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)
    Mounts:
      /var/run/linkerd/identity/end-entity from linkerd-identity-end-entity (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-sc7l7 (ro)