Zeek Logo
current (v6.0.0)

Table of Contents

  • About Zeek
  • Monitoring With Zeek
  • Get Started
  • Zeek Log Formats and Inspection
  • Zeek Logs
  • Introduction to Scripting
  • Frameworks
  • Popular Customizations
    • Log Enrichment
      • Community ID
    • Log Writers
      • Kafka
    • Logging
      • JSON Streaming Logs
      • Long Connections
    • Profiling and Debugging
      • jemalloc profiling
  • Troubleshooting
  • Script Reference
  • Developer Guides
  • Subcomponents
  • Acknowledgements
Index
Zeek
  • Popular Customizations

Popular Customizations¶

This page outlines customizations and additions that are popular among Zeek users.

Note

This page lists externally-maintained Zeek packages. The Zeek team does not provide support or maintenance for these packages. If you find bugs or have feature requests, please reach out to the respective package maintainers directly.

You may also post in the Zeek Slack #packages channel or forum to get help from the broader Zeek community.

Log Enrichment¶

Community ID¶

New in version 6.0.

Zeek includes native Community ID Flow Hashing support. This functionality has previously been provided through the zeek-community-id package.

Note

At this point, the external zeek-community-id package is still available to support Zeek deployments running older versions. However, the scripts provided by the package cause conflicts with those provided in Zeek 6.0 - do not load both.

Loading the policy/protocols/conn/community-id-logging.zeek and policy/frameworks/notice/community-id.zeek scripts adds an additional community_id field to the Conn::Info and Notice::Info record.

$ zeek -r ./traces/get.trace protocols/conn/community-id-logging LogAscii::use_json=T
$ jq < conn.log
{
  "ts": 1362692526.869344,
  "uid": "CoqLmg1Ds5TE61szq1",
  "id.orig_h": "141.142.228.5",
  "id.orig_p": 59856,
  "id.resp_h": "192.150.187.43",
  "id.resp_p": 80,
  "proto": "tcp",
  ...
  "community_id": "1:yvyB8h+3dnggTZW0UEITWCst97w="
}

The Community ID Flow Hash of a conn_id instance can be computed with the community_id_v1 builtin function directly on the command-line or used in custom scripts.

$ zeek -e 'print community_id_v1([$orig_h=141.142.228.5, $orig_p=59856/tcp, $resp_h=192.150.187.43, $resp_p=80/tcp])'
1:yvyB8h+3dnggTZW0UEITWCst97w=

Log Writers¶

Kafka¶

For exporting logs to Apache Kafka in a streaming fashion, the externally-maintained zeek-kafka package is a popular choice and easy to configure. It relies on librdkafka.

redef Log::default_writer = Log::WRITER_KAFKAWRITER;

redef Kafka::kafka_conf += {
    ["metadata.broker.list"] = "192.168.0.1:9092"
};

Logging¶

JSON Streaming Logs¶

The externally-maintained json-streaming-logs package tailors Zeek for use with log shippers like Filebeat or fluentd. It configures additional log files prefixed with json_streaming_, adds _path and _write_ts fields to log records and configures log rotation appropriately.

If you do not use a logging archive and want to stream all logs away from the system where Zeek is running without leveraging Kafka, this package helps you with that.

Long Connections¶

Zeek logs connection entries into the conn.log only upon termination or due to expiration of inactivity timeouts. Depending on the protocol and chosen timeout values this can significantly delay the appearance of a log entry for a given connection. The delay may be up to an hour for lingering SSH connections or connections where the final FIN or RST packets were missed.

The zeek-long-connections package alleviates this by creating a conn_long.log log with the same format as conn.log, but containing entries for connections that have been existing for configurable intervals. By default, the first entry for a connection is logged after 10mins. Depending on the environment, this can be lowered as even a 10 minute delay may be significant for detection purposes in streaming setup.

Profiling and Debugging¶

jemalloc profiling¶

For investigation of memory leaks or state-growth issues within Zeek, jemalloc’s profiling is invaluable. A package providing a bit support for configuring jemalloc’s profiling facilities is zeek-jemalloc-profiling.

Some general information about memory profiling exists in the Troubleshooting section.

Previous Next

© Copyright 2019-2021, The Zeek Project. Revision 841cbf0b. Last updated on July 05, 2023.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: current (v6.0.0)
Versions
master
v6.0.0
v5.2.2
v5.2.0
v5.1.3
v5.1.2
v5.1.1
v5.1.0
v5.0.9
v5.0.7
v5.0.6
v5.0.5
v5.0.4
v5.0.3
v5.0.2
v5.0.1
v5.0.0
v4.2.2
v4.1.1
v4.0.9
v3.2.3
v3.1.4
v3.0.14
topic-awelzel-js-docs
lts
current
Downloads
html
On Read the Docs
Project Home
Builds