How to fix garbled characters received in Kafka Audit Log Listener

kafka-screenshot.png We have setup Kafka as audit log listener and we are seeing the logs. Following is the way we have configured

  1. Wrote extension point with target: org.nuxeo.runtime.stream.kafka.service point: kafkaConfig

    <?xml version="1.0"?>
    <component name="com.softcell.kafka.contrib.kafkacontrib">
    
    <extension target="org.nuxeo.runtime.stream.kafka.service" point="kafkaConfig">
    <kafkaConfig name="default" topicPrefix="nuxeo-">
      <producer>
        <property name="bootstrap.servers">localhost:9092</property>
      </producer>
      <consumer>
        <property name="bootstrap.servers">localhost:9092</property>
        <property name="request.timeout.ms">65000</property>
        <property name="max.poll.interval.ms">60000</property>
        <property name="session.timeout.ms">20000</property>
        <property name="heartbeat.interval.ms">1000</property>
        <property name="max.poll.records">50</property>
      </consumer>
    </kafkaConfig>
    </extension>
    </component>
    
  2. Added following config to this contributions nuxeo.defaults:

    #kafkacontrib.target=
    kafka.enabled=true
    nuxeo.template.includes=common
    nuxeo.stream.audit.enabled=true
    nuxeo.stream.audit.log.config=audit
    nuxeo.stream.audit.batch.size=25
    nuxeo.stream.audit.batch.threshold.ms=500
    

After doing this, I start listening to topic nuxeo-audit using consoleConsumer using this command: bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic nuxeo-audit

I can see the logs getting generated. However there are garbled characters in them. Pasting a sample below:

kafka-screenshot.png

I can see in the source-code that BytesDeserializer is being used for value but I am not sure why am I seeing garbled characters. Any help in this would be deeply appreciated. Thanks

0 votes

0 answers

1227 views

ANSWER

I think you need to add this to nuxeo.conf:

nuxeo.stream.audit.log.codec=legacy

You can find more info here: https://doc.nuxeo.com/nxdoc/nuxeo-stream/#record-codec

There's a warning about using this with Kafka topics but I'm not sure it applies in your situation. https://doc.nuxeo.com/nxdoc/nuxeo-server-release-notes/#nuxeo-streams

07/10/2019

Thanks for your inputs LaraGranite. I added the codec config like you mentioned but didn't see any changes. I can see that there are a few more codecs available. I will try them and will post the findings here.
07/12/2019