Fluentd without buffer section or with encryption in buffer

[mikromas]

What is a problem?

Is it possible to run Fluentd without a buffer section? Additionally, can data be encrypted in the buffer when using @type file?

Thank you in advance!

Describe the configuration of Fluentd

No response

Describe the logs of Fluentd

No response

Environment

- Fluentd version:
- TD Agent version:
- Fluent Package version:
- Docker image (tag):
- Operating system:
- Kernel version:

[Watson1978]

Is it possible to run Fluentd without a buffer section?

When outputting, you will always need to use some kind of buffer.
If it flushes the buffer immediately, it might let behavior that you want.

<match **>
  @type file
  path "/PATH/TO/YOUR/LOG"
  append true

  <buffer []>
    @type memory
    flush_mode immediate
  </buffer>
</match>

can data be encrypted in the buffer when using @type file?

I think there is no such function. we welcome pull requests to add the function.

[daipom]

Is it possible to run Fluentd without a buffer section?

When outputting, you will always need to use some kind of buffer.

For out_file plugin, yes, we must use some kind of buffer.

In general, it depends on the output plugin.
In general, output plugins can work with or without buffer.

[mikromas]

Hi @Watson1978 and @daipom
Thanks both for your prompt answers!

Let me clarify more.
I am interested in '@type kafka2' and '@type elasticsearch' kind of output plugins.
Also, is there any "security" workaround regarding data encryption during buffering?
Switching to in-memory buffer (at the cost of persistence), is there any security way out regarding the buffered data?

Thank you in advance!