Press "Enter" to skip to content

Filebeat or Logstash?

Filebeat和Logstash都是ES套件(ES stack)中的组成部分, 其中, Filebeat还是beats家族的成员之一. Filebeat和Logstash都可以将日志文件输出到ElasticSearch, 且众所周知, Filebeat非常轻量级, 而Logstash由于使用JVM的原因性能堪忧, 那么是不是说我们可以抛弃笨重的Logstash了呢?

What is the difference between Logstash and Beats?

Beats are lightweight data shippers that you install as agents on your servers to send specific types of operational data to Elasticsearch. Beats have a small footprint and use fewer system resources than Logstash.

Logstash has a larger footprint, but provides a broad array of input, filter, and output plugins for collecting, enriching, and transforming data from a variety of sources.





If you’re using ELK as your logging solution, one way to ship these logs is using Filebeat to send the data directly into Elasticsearch. Since Filebeat ships data in JSON format, Elasticsearch should be able to parse the timestamp and message fields without too much hassle. Not only that, Filebeat also supports an Apache module that can handle some of the processing and parsing.

However, as of yet, advanced log enhancement — adding context to the log messages by parsing them up into separate fields, filtering out unwanted bits of data and enriching others — cannot be handled without Logstash.

Tough configuration, resource consumption, frequent crashes, and difficult troubleshooting have all made Logstash the weakest link in the ELK Stack. Yet despite these flaws, Logstash still remains a crucial component of the stack.

Elastic has made big steps in trying to alleviate these pains by introducing Beats (and adding a visual element to Logstash pipelines in the future version 6.0), which has enabled users to build and configure multiple and resilient logging pipelines — and ultimately make logging with ELK much more reliable.



但是,到目前为止,高级日志增强 – 通过将日志消息分解为单独的字段来添加上下文,过滤掉不需要的数据并丰富其他内容 – 如果没有Logstash,则无法处理。

严格的配置,资源消耗,频繁的崩溃和难以排除故障都使得Logstash成为ELK Stack中最薄弱的环节。尽管有这些缺陷,Logstash仍然是堆栈的关键组成部分。

Elastic通过引入Beats(以及在未来版本6.0中向Logstash管道添加可视元素)尝试缓解这些痛苦,已经迈出了重要的一步,这使得用户能够构建和配置多个弹性日志管道 – 并最终使用ELK进行日志记录更可靠。

Logstash is useful if you need to aggregate logs from many servers and apply some common transformations and filtering to your events.

If your log events are already structured and you are ok with indexing them directly, then you can definitely have Filebeat send them directly to ES. If ES goes down (e.g. for maintenance), Filebeat will retry until it can successfully send the events.




What is the difference between Logstash and Beats?
Filebeat vs. Logstash — The Evolution of a Log Shipper
FileBeat directly to ELS or via LogStash?

Leave a Reply

Your email address will not be published. Required fields are marked *