Zeek Documentation
The documentation repo at zeek-docs contains version-specific Zeek documentation source files that are ultimately used as the basis for content hosted at https://docs.zeek.org.
Markup Format, Style, and Conventions
For general guidance on the basics of how the documentation is written, consult this Zeek wiki:
https://github.com/zeek/zeek/wiki/Documentation-Style-and-Conventions
Source-Tree Organization
The zeek-docs repo containing this README file is the root of a Sphinx source
tree and can be modified to add more documentation, style sheets, JavaScript,
etc. The Sphinx config file is conf.py
. The typical way new documents get
integrated is from them being referenced directly in index.rst
or
indirectly from something in the toctree
(Table of Contents Tree) specified
in that main index.
There is also a custom Sphinx domain implemented in ext/zeek.py
which adds
some reStructureText (reST) directives and roles that aid in generating useful
index entries and cross-references. This primarily supports integration with
the script-reference sections, some of which are auto-generated by Zeek’s
Doxygen-like feature, named “Zeekygen”. The bulk of auto-generated content
lives under the scripts/
directory or has a file name starting with
“autogenerated”, so if you find yourself wanting to change those, you should
actually look at at doing those changes within the Zeek repo itself rather
than here, so see the next section for how Zeekygen docs can be (re)generated.
Generating Zeekygen Reference Docs
All Zeekygen-generated docs get committed into Git, so if you don’t have to perform any changes on it and just want to preview what’s already existing, you can skip down to the next Local Previewing section.
The Zeekygen documentation-generation feature is a part of Zeek itself, so you’ll want to obtain the Zeek repo from Git, read the INSTALL file directions to install required dependencies, and build Zeek:
git clone --recursive https://github.com/zeek/zeek
cd zeek
# Read INSTALL file and get dependencies here
./configure && make -j $(nproc)
# Make desired edits to scripts/, src/, etc.
./ci/update-zeekygen-docs.sh
The last command runs a script to generate documentation, which will end up in
the doc/
subdirectory. Note that doc/
is just a Git submodule of this
this zeek-docs repository, so you can run git status
there to find exactly
what changed.
Also note that the documentation-generation script is run automatically
on a daily basis to incorporate up any documentation changes that people make
in Zeek itself without them having to necessarily be aware of the full
documentation process. The GitHub Action that does that daily task is
located in the Zeek repo’s .github/workflows/generate-docs.yml
file.
Local Previewing (How To Build)
First make sure you have the required dependencies used for building docs:
Python interpreter >= 3.7
Read the Docs Sphinx Theme: https://github.com/rtfd/sphinx_rtd_theme
GitPython: https://github.com/gitpython-developers/GitPython
If you have pip, you may just use the command pip3 install -r
requirements.txt
to install all the dependencies using the
requirements.txt
from zeek-docs.
Now run make
within the zeek-docs repository’s top-level to locally render
its reST files into HTML. After the build completes, HTML documentation is
symlinked in build/html
and you can open the index.html
found there in
your web browser.
There’s also a make livehtml
(requires pip3 install sphinx-autobuild
)
target in the top-level Makefile that is useful for editing the reST files and
seeing changes rendered out live to a separate browser.
Hosting
Documentation is hosted by Read the Docs (RTD), so you can generally read about how it works there. The web-interface is accessible via https://readthedocs.org/projects/zeek-docs.
How zeek-docs is configured to use RTD is a combination of some custom
settings in its .readthedocs.yml
file and others only accessible through
RTD’s web-interface (e.g. domain and subproject settings). Most config
settings are likely understandable just by browsing the web-interface and
RTD’s guides, but a few particular points to mention:
There is an associated, always-failing project at https://readthedocs.org/projects/zeek. It’s always-failing because RTD redirects only activate when pages 404 and this project exists so that all attempts to use https://zeek.rtfd.io or https://zeek.readthedocs.io get redirected to https://docs.zeek.org. Those would have been the project URLs if ownership of the RTD ‘zeek’ project was had from the start, but it was only obtained later, after documentation already started development in the ‘zeek-docs’ RTD project slug.
Over time, page redirects have accrued into
redirects.yml
as a way to help document what they are and why they happened and also as a potential way to automate addition/reinstantiation of a large number of redirects, but typically redirects can be manually added via the RTD web interface first and then noted inredirects.yml
There are RTD subprojects for things like Broker, Package Manager, and Spicy. The use of subprojects simply allows access to their RTD docs via the custom domain of https://docs.zeek.org
RTD will auto-build any newly-pushed commits to zeek-docs (i.e. a webhook is configured), but if a tag is changed to point somewhere different, you’ll typically have to go into the RTD web interface, “Edit” the associated version under “Versions”, “wipe” the existing docs, and then manually trigger a rebuild of that version tag under “Builds”.