# Better Log Parsing with Logstash and Google Protocol Buffers

DevFeed: [Better Log Parsing with Logstash and Google Protocol Buffers](<https://devfeed.tech/articles/better-log-parsing-with-logstash-and-google-protocol-buffers-27939.md>)

Original publisher: [Read original article](<https://tech.trivago.com/post/2016-01-19-logstash_protobuf_codec/>)

Author: Inga Feick Follow

Published: 2016-01-19T00:00:00Z

Content type: tutorial

Language: en

Sources: [Trivago](<https://devfeed.tech/sources/trivago.md>)

Topics: [logstash](<https://devfeed.tech/topics/logstash.md>), [Zig](<https://devfeed.tech/topics/zig.md>), [elasticsearch](<https://devfeed.tech/topics/elasticsearch.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [Development](<https://devfeed.tech/topics/development.md>), [Ruby](<https://devfeed.tech/topics/ruby.md>)

Tags: [backend](<https://devfeed.tech/tags/backend.md>), [data](<https://devfeed.tech/tags/data.md>), [devops](<https://devfeed.tech/tags/devops.md>), [elasticsearch](<https://devfeed.tech/tags/elasticsearch.md>), [google](<https://devfeed.tech/tags/google.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [logs](<https://devfeed.tech/tags/logs.md>), [logstash](<https://devfeed.tech/tags/logstash.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [parsing](<https://devfeed.tech/tags/parsing.md>), [performance](<https://devfeed.tech/tags/performance.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [stream](<https://devfeed.tech/tags/stream.md>)

## AI overview

This tutorial explains how trivago uses Google Protocol Buffers with Kafka, Elasticsearch, and Logstash for log processing. It describes why protobuf can reduce message size and enforce a stable structure, then outlines creating and using a custom Logstash codec for decoding protobuf messages.

## Source excerpt

At trivago we rely heavily on the ELK stack for our log processing. We stream our webserver access logs, error logs, performance benchmarks and all kind of diagnostic data into Kafka and process it from there into Elasticsearch using Logstash.