Cannot find logs in Datadog.

[bandisudhir]

Hi everyone,

I'm having trouble viewing my logs in Datadog. I've set up Vector to send logs to Datadog, but I can't see them in the Datadog portal. Below is my current Vector configuration. Could someone help me identify what might be wrong or missing in my setup?

data_dir: /var/lib/vector

api:
  enabled: true
  address: 127.0.0.1:8686

sources:
  my_source:
    type: socket
    address: 0.0.0.0:5170
    mode: tcp
    max_length: 128
    buffer_size: 356
    encoding:
      codec: json

transforms:
  extract_message:
    type: remap
    inputs:
      - my_source
    source: |
      .nxlog_eikon_message, err = parse_json(.message)
      if err == null {
        . = { "nxlog_eikon_message": .nxlog_eikon_message, "nxlog_eikon_type": "json" }
      } else {
        del(.)
      }

  add_tags:
    type: remap
    inputs:
      - extract_message
    source: |
      .ddtags = {
        "mnd-applicationid": "app-xxxx",
        "mnd-applicationname": "data_api",
        "mnd-owner": "[email protected]",
        "mnd-supportgroup": "APP-ANALYTICS-xxx-CLOUD",
        "mnd-projectcode": "xxx-xx",
        "mnd-costcentre": "xxxxxxxxx",
        "mnd-dataclassification": "restricted",
        "mnd-baseimagename": "notapplicable",
        "mnd-envtype": "dev",
        "mnd-envsubtype": "sandbox",
        "mnd-lifecycle": "pre-live",
        "opt-componentname": "xxx_api",
        "opt-environment": "beta_ibc",
        "opt-region": "AMERS"
      }
      .service = "xxx_api"
      .ddsource = "usage"
      .host = "XXX-XXXXXXX"
      .hostname = "XXX-XXXXXX"
      .vector_tag = "yes"
      .timestamp = "2024-11-29T00:00:00.000000000Z"
      .env = "dev"

sinks:
  my_console_sink:
    type: console
    inputs:
      - add_tags
    encoding:
      codec: json

  my_datadog_sink1:
    type: datadog_logs
    inputs:
      - add_tags
    api_key: "your_datadog_api_key_1"
    compression: gzip
    site: "datadoghq.eu"
    tags: ["mnd-applicationid:app-xxxx", "mnd-applicationname:dataxxx_api", "mnd-owner:[email protected]", "mnd-supportgroup:APP-ANALYTICS-XXXX-CLOUD", "mnd-projectcode:1267-10", "mnd-costcentre:XXXXX", "mnd-dataclassification:restricted", "mnd-baseimagename:notapplicable", "mnd-envtype:dev", "mnd-envsubtype:sandbox", "mnd-lifecycle:pre-live", "opt-componentname:xxx_api", "opt-environment:beta_ibcxx", "opt-region:AMERS"]
    skip_ssl_verification: true

  my_datadog_sink2:
    type: datadog_logs
    inputs:
      - add_tags
    api_key: "your_datadog_api_key_2"
    compression: gzip
    site: "datadoghq.eu"
    tags: ["mnd-applicationid:app-xxxxxx", "mnd-applicationname:dataxxx_api", "mnd-owner:[email protected]", "mnd-supportgroup:APP-ANALYTICS-XXX-CLOUD", "mnd-projectcode:1267-10", "mnd-costcentre:XXXXXXX", "mnd-dataclassification:restricted", "mnd-baseimagename:notapplicable", "mnd-envtype:dev", "mnd-envsubtype:sandbox", "mnd-lifecycle:pre-live", "opt-componentname:xxx_api", "opt-environment:beta_ibcxx", "opt-region:AMERS"]
    skip_ssl_verification: true

  my_tcp_sink:
    type: socket
    inputs:
      - extract_message
    address: "xxxxxxxxxxx.com:12201"
    mode: tcp
    encoding:
      codec: json

Any help or suggestions would be greatly appreciated!

vector.txt

[pront]

Hi @bandisudhir,

Does the healthcheck pass on startup? Do you see any errors while running your pipeline?

A quick sanity check is to use the public intake API with your DD api key:

curl -X POST "https://http-intake.logs.datadoghq.com/api/v2/logs" \
-H "Accept: application/json" \
-H "Content-Type: text/plain" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-d @- << EOF

Is this successful? Can you see the log in DD?

https://docs.datadoghq.com/api/latest/logs/?code-lang=curl

[bandisudhir]

From the cmd prompt I can see the request is accepted as per the console message.

Also the health check is passed.

But from post with the given quick sanity check curl cmd. The request is forbidden.

I'm I missing something.

[pront]

Hmm, the 403 status indicates an issue with your key. The datadog logs sink should not pass the healthcheck though.

Please see https://docs.datadoghq.com/api/latest/authentication/#validate-api-key. Once you have a valid key that passes the validation check, you can plug it into your Vector config and try again. Let me know how that goes.

[bandisudhir]

The issue is related to datadog in this case it expects some tags to populate the date in the logs explorer. So, it is not related to vector. Thanks for the reponses