Can't get it to work

[scalp42]

Hi @echee2

Using td-agent 1.4.2, we can't get it to work (and nothing showing up in the logs either).

How to replicate our setup, set up a syslog input:

<source>
  @type syslog
  tag syslog.docker.containers
  port 5140
  protocol_type udp
  <parse>
    message_format rfc5424
  </parse>
  source_hostname_key hostname
  source_address_key source_ip
  priority_key priority
  facility_key facility
  @log_level info
</source>

Set up an output with Scalyr:

<match syslog.docker.containers.**>
  @type copy
  <store>
    @type scalyr
    api_write_token xxxxxx
    compression_type bz2
    use_hostname_for_serverhost true
    scalyr_server https://agent.scalyr.com/
    ssl_ca_bundle_path /etc/ssl/certs/ca-certificates.crt
    ssl_verify_peer true
    ssl_verify_depth 5
    message_field message
  </store>
  <store>
    @type stdout
  </store>
</match>

Send some logs to FluentD input:

date ; docker run --rm --name test --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:5140 --log-opt tag="{{.ImageName}}/{{.Name}}/{{.ID}}" --log-opt syslog-format=rfc5424  alpine echo "{\"antho\": \"scalyr-$RANDOM\"}"

Looking at FluentD logs, we can see the request:

2019-07-18 16:57:20.000000000 +0000 syslog.docker.containers.daemon.info: {"host":"default-ubuntu-1804","ident":"alpine/test/ce35f462371d","pid":"16831","msgid":"alpine/test/ce35f462371d","extradata":"-","message":"{\"antho\": \"scalyr-32613\"}","priority":"info","facility":"daemon","source_ip":"127.0.0.1","hostname":"localhost"}

Unfortunately, the logs are not appearing on Scalyr side.

Any ideas?

[scalp42]

Only unrelated thing:

2019-07-18 17:03:09 +0000 [warn]: parameter 'use_hostname_for_serverhost' in <store>
  @type "scalyr"
  message_field "message"
  <snip>
</store> is not used.

[jmorascalyr]

Hello,

Thanks for reaching out. Our engineering team is currently looking into this for you and we will let you know when we have gotten to the root of the issue.

Thanks,

[weilliu]

@scalp42

Our developer @imron found the root cause of the issue. This error appears to have been caused because the use_hostname_for_serverhost is available from version 0.8.8 of the plugin only and that hadn't been pushed to rubygems yet.

This is now pushed to rubygems.

Running fluent-gem update fluent-plugin-scalyr (or equivalent command for td-agent) should update the Scalyr fluentd plugin to version 0.8.8. This can be confirmed in the output of fluentd which will show a line like:

2019-07-19 07:12:09 +0000 [info]: gem 'fluent-plugin-scalyr' version '0.8.8'

Please let us know if this resolves the problem, and if not can you provide the full log output from fluentd (with API key redacted).

[scalp42]

Trying right now. Will keep you updated.

[scalp42]

Confirmed, it's working.

Closing this issue, thanks @weilliu and @imron 💯