git: 5f73d750c276 - main - security/zeek: Update to 7.0.0
Date: Thu, 01 Aug 2024 21:05:06 UTC
The branch main has been updated by leres:
URL: https://cgit.FreeBSD.org/ports/commit/?id=5f73d750c276dea55de2658c340d8e79042f36f2
commit 5f73d750c276dea55de2658c340d8e79042f36f2
Author: Craig Leres <leres@FreeBSD.org>
AuthorDate: 2024-08-01 21:04:36 +0000
Commit: Craig Leres <leres@FreeBSD.org>
CommitDate: 2024-08-01 21:04:36 +0000
security/zeek: Update to 7.0.0
https://github.com/zeek/zeek/releases/tag/v7.0.0
This is the latest major version number Long-Term Support (LTS)
release of Zeek.
- The Telemetry framework has had a major rework, and includes a
number of breaking changes. The biggest change is a move towards
a Prometheus-first model.
- All of the metrics-related script-level options, type, and methods
have been
moved to the Telemetry framework.
- The following options have been removed:
Broker::metrics_export_interval
Broker::metrics_export_topic
Broker::metrics_import_topics
Broker::metrics_export_prefixes
- The unit field has been removed from the telemetry log.
- All of the BROKER_METRICS_* environment variables have been
removed.
- The instruments that previously supported count in scripts and
int64_t in C++ were removed in favor of only providing double
versions.
- The is_sum argument has been removed from the constructors/creation
methods for all of the instruments.
- Zeekctl now sets FileExtract::prefix to spool/extract_files/<node>
to avoid deletion of extracted files when stopping worker nodes.
- Support delete on tables, sets and vectors to clear their contents.
- A new helper function can_load() backed by a new bif find_in_zeekpath()
was added to determine if a non-relative @load directive might
work.
- Zeek packagers can now include a "local" addition into Zeek's
version string.
- SMB2 packets containing multiple PDUs now correctly parse all
of the headers, instead of just the first one and ignoring the
rest.
- The new built-in function lookup_connection_analyzer_id() retrieves
the numeric identifier of an analyzer associated with a connection.
- The from_json() function now supports ingesting JSON representations
of tables as produced by the to_json() function.
- The analyzer.log now optionally supports logging of disabled
analyzers through the new option Analyzer::logging::include_disabling.
- The ftp.log fuid field is now cleared after handling a command
with a fuid associated with it.
- The type_name field populated by global_ids() now aligns with
the value returned by type_name() for each identifier. E.g,
Site::local_nets has a type_name of set[subnet] rather than table.
- The ISO 9660 file signature has been moved into the policy
directory.
- The val_footprint() BiF now factors in the size of strings when
reporting footprints, roughly equating a string's size with the
number of elements comparable to that length.
- The tuning/defaults policy has been deprecated and will be removed
in v7.1.
- If a Spicy protocol analyzers feeds data into file analysis, it
now needs to call Zeek's Files::register_protocol() and provide
a callback for computing file handles.
- The Supervisor's API now returns NodeConfig records with a cluster
table whose ClusterEndpoints have a port value of 0/unknown,
rather than 0/tcp, to indicate that the node in question has no
listening port.
- The --disable-archiver configure flag no longer does anything
and will be removed in 7.1. zeek-archiver has moved into the
zeek-aux repository.
Reported by: Tim Wojtulewicz
---
security/zeek/Makefile | 46 ++++-
security/zeek/distinfo | 6 +-
..._zeekctl_auxil_pysubnettree_SubnetTree__wrap.cc | 156 ---------------
...zeekctl_auxil_pysubnettree_include_SubnetTree.h | 11 --
security/zeek/files/patch-src_threading_Manager.cc | 47 -----
security/zeek/files/patch-src_threading_Manager.h | 23 ---
.../zeek/files/patch-src_threading_MsgThread.cc | 146 --------------
.../zeek/files/patch-src_threading_MsgThread.h | 62 ------
security/zeek/pkg-plist | 211 ++++++++++++++-------
9 files changed, 186 insertions(+), 522 deletions(-)
diff --git a/security/zeek/Makefile b/security/zeek/Makefile
index 791dd314aa14..6aacbcd2de35 100644
--- a/security/zeek/Makefile
+++ b/security/zeek/Makefile
@@ -1,6 +1,5 @@
PORTNAME= zeek
-DISTVERSION= 6.0.4
-PORTREVISION= 1
+DISTVERSION= 7.0.0
CATEGORIES= security
MASTER_SITES= https://download.zeek.org/
@@ -14,7 +13,7 @@ BUILD_DEPENDS= bison>=3.3:devel/bison \
flex>=2.6:textproc/flex \
swig>=4.0.2:devel/swig
LIB_DEPENDS= libcares.so:dns/c-ares
-RUN_DEPENDS= c-ares>=1.18.1:dns/c-ares
+RUN_DEPENDS= c-ares>=1.25.0:dns/c-ares
USES= bison cmake compiler:c++17-lang cpe gettext-runtime perl5 \
python shebangfix ssl
@@ -30,8 +29,44 @@ PORTSCOUT= limit:^[0-9]*\.0\.
CXXFLAGS_powerpc64= -mpower8-vector
-SHEBANG_FILES= auxil/zeekctl/auxil/trace-summary/trace-summary \
- auxil/zeek-client/zeek-client
+SHEBANG_FILES= \
+ auxil/broker/bindings/python/3rdparty/pybind11/docs/conf.py \
+ auxil/broker/bindings/python/3rdparty/pybind11/setup.py \
+ auxil/broker/bindings/python/3rdparty/pybind11/tools/make_changelog.py \
+ auxil/broker/bindings/python/3rdparty/pybind11/tools/setup_global.py.in \
+ auxil/broker/bindings/python/3rdparty/pybind11/tools/setup_main.py.in \
+ auxil/btest/btest \
+ auxil/btest/btest-setsid \
+ auxil/netcontrol-connectors/acld/acld.py \
+ auxil/netcontrol-connectors/command-line/command-line.py \
+ auxil/netcontrol-connectors/openflow/controller.py \
+ auxil/netcontrol-connectors/test/simple-client.py \
+ auxil/package-manager/zkg \
+ auxil/spicy/3rdparty/benchmark/tools/compare.py \
+ auxil/spicy/3rdparty/benchmark/tools/strip_asm.py \
+ auxil/spicy/3rdparty/doctest/scripts/bench/bench.py \
+ auxil/spicy/3rdparty/doctest/scripts/bench/run_all.py \
+ auxil/spicy/doc/scripts/spicy-doc-to-rst \
+ auxil/spicy/tests/Scripts/license-header.py \
+ auxil/spicy/tests/Scripts/stray_baselines.py \
+ auxil/vcpkg/ports/gobject-introspection/portfile.cmake \
+ auxil/zeek-aux/devel-tools/github-manage \
+ auxil/zeek-client/man/build.py \
+ auxil/zeek-client/zeek-client \
+ auxil/zeekctl/ZeekControl/test_cli.py \
+ auxil/zeekctl/auxil/pysubnettree/setup.py \
+ auxil/zeekctl/auxil/trace-summary/trace-summary \
+ auxil/zeekctl/bin/stats-to-csv \
+ auxil/zeekctl/bin/zeekctl.in \
+ auxil/zeekctl/bin/zeekctld.in \
+ auxil/zeekctl/testing/Cfg/bin/zeek__test \
+ auxil/zeekctl/testing/Scripts/diff-to-bytes-output \
+ auxil/zeekctl/util/extract-strictly-local-conns \
+ auxil/zeekctl/util/reformat-stats \
+ ci/collect-repo-info.py \
+ testing/coverage/coverage_cleanup.py \
+ testing/scripts/coverage-calc \
+ testing/scripts/httpd.py
SUB_FILES= pkg-message
NO_MTREE= yes
@@ -40,6 +75,7 @@ CMAKE_ON= BROKER_DISABLE_DOC_EXAMPLES BROKER_DISABLE_TESTS \
BUILD_SHARED_LIBS BUILD_STATIC_BROKER INSTALL_AUX_TOOLS
CMAKE_ARGS= -DCARES_ROOT_DIR:PATH=${PREFIX} \
-DCMAKE_EXE_LINKER_FLAGS="${OPENSSL_LDFLAGS}" \
+ -DDISABLE_JAVASCRIPT:BOOL=ON \
-DINSTALL_BTEST:BOOL=OFF \
-DINSTALL_BTEST_PCAPS:BOOL=OFF \
-DINSTALL_ZKG:BOOL=OFF \
diff --git a/security/zeek/distinfo b/security/zeek/distinfo
index ff14df009050..454a0f21fb3e 100644
--- a/security/zeek/distinfo
+++ b/security/zeek/distinfo
@@ -1,3 +1,3 @@
-TIMESTAMP = 1715883383
-SHA256 (zeek-6.0.4.tar.gz) = 1a7d40d2749a90b914ae3be8df14c17de64c1921b8b272a93a54985aa1080396
-SIZE (zeek-6.0.4.tar.gz) = 60250766
+TIMESTAMP = 1722545605
+SHA256 (zeek-7.0.0.tar.gz) = 186393329b7f53f8087730eb97449e0e868b0dbad5deaea3864d7a7c7d30f8fb
+SIZE (zeek-7.0.0.tar.gz) = 95688958
diff --git a/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_SubnetTree__wrap.cc b/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_SubnetTree__wrap.cc
deleted file mode 100644
index b8aeb6eacbed..000000000000
--- a/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_SubnetTree__wrap.cc
+++ /dev/null
@@ -1,156 +0,0 @@
---- auxil/zeekctl/auxil/pysubnettree/SubnetTree_wrap.cc.orig 2024-05-16 17:25:57 UTC
-+++ auxil/zeekctl/auxil/pysubnettree/SubnetTree_wrap.cc
-@@ -1629,6 +1629,14 @@ SwigPyObject_repr(SwigPyObject *v, PyObject *args)
- return repr;
- }
-
-+/* We need a version taking two PyObject* parameters so it's a valid
-+ * PyCFunction to use in swigobject_methods[]. */
-+SWIGRUNTIME PyObject *
-+SwigPyObject_repr2(PyObject *v, PyObject *SWIGUNUSEDPARM(args))
-+{
-+ return SwigPyObject_repr((SwigPyObject*)v);
-+}
-+
- SWIGRUNTIME int
- SwigPyObject_compare(SwigPyObject *v, SwigPyObject *w)
- {
-@@ -1741,11 +1749,7 @@ SWIGRUNTIME PyObject*
- }
-
- SWIGRUNTIME PyObject*
--#ifdef METH_NOARGS
--SwigPyObject_next(PyObject* v)
--#else
- SwigPyObject_next(PyObject* v, PyObject *SWIGUNUSEDPARM(args))
--#endif
- {
- SwigPyObject *sobj = (SwigPyObject *) v;
- if (sobj->next) {
-@@ -1780,6 +1784,20 @@ SwigPyObject_acquire(PyObject* v, PyObject *SWIGUNUSED
- return SWIG_Py_Void();
- }
-
-+#ifdef METH_NOARGS
-+static PyObject*
-+SwigPyObject_disown2(PyObject* v, PyObject *SWIGUNUSEDPARM(args))
-+{
-+ return SwigPyObject_disown(v);
-+}
-+
-+static PyObject*
-+SwigPyObject_acquire2(PyObject* v, PyObject *SWIGUNUSEDPARM(args))
-+{
-+ return SwigPyObject_acquire(v);
-+}
-+#endif
-+
- SWIGINTERN PyObject*
- SwigPyObject_own(PyObject *v, PyObject *args)
- {
-@@ -1820,12 +1838,12 @@ swigobject_methods[] = {
- #ifdef METH_O
- static PyMethodDef
- swigobject_methods[] = {
-- {(char *)"disown", (PyCFunction)SwigPyObject_disown, METH_NOARGS, (char *)"releases ownership of the pointer"},
-- {(char *)"acquire", (PyCFunction)SwigPyObject_acquire, METH_NOARGS, (char *)"acquires ownership of the pointer"},
-+ {(char *)"disown", (PyCFunction)SwigPyObject_disown2, METH_NOARGS, (char *)"releases ownership of the pointer"},
-+ {(char *)"acquire", (PyCFunction)SwigPyObject_acquire2,METH_NOARGS, (char *)"acquires ownership of the pointer"},
- {(char *)"own", (PyCFunction)SwigPyObject_own, METH_VARARGS, (char *)"returns/sets ownership of the pointer"},
- {(char *)"append", (PyCFunction)SwigPyObject_append, METH_O, (char *)"appends another 'this' object"},
- {(char *)"next", (PyCFunction)SwigPyObject_next, METH_NOARGS, (char *)"returns the next 'this' object"},
-- {(char *)"__repr__",(PyCFunction)SwigPyObject_repr, METH_NOARGS, (char *)"returns object representation"},
-+ {(char *)"__repr__",(PyCFunction)SwigPyObject_repr2, METH_NOARGS, (char *)"returns object representation"},
- {0, 0, 0, 0}
- };
- #else
-@@ -1836,7 +1854,7 @@ swigobject_methods[] = {
- {(char *)"own", (PyCFunction)SwigPyObject_own, METH_VARARGS, (char *)"returns/sets ownership of the pointer"},
- {(char *)"append", (PyCFunction)SwigPyObject_append, METH_VARARGS, (char *)"appends another 'this' object"},
- {(char *)"next", (PyCFunction)SwigPyObject_next, METH_VARARGS, (char *)"returns the next 'this' object"},
-- {(char *)"__repr__",(PyCFunction)SwigPyObject_repr, METH_VARARGS, (char *)"returns object representation"},
-+ {(char *)"__repr__",(PyCFunction)SwigPyObject_repr, METH_VARARGS, (char *)"returns object representation"},
- {0, 0, 0, 0}
- };
- #endif
-@@ -3457,7 +3475,7 @@ SWIGINTERN PyObject *SubnetTree___getitem__(SubnetTree
-
- PyObject* data = self->lookup(cidr, size);
- if ( ! data ) {
-- PyErr_SetString(PyExc_KeyError, cidr);
-+ PyErr_SetObject(PyExc_KeyError, PyBytes_FromStringAndSize(cidr, size));
- return 0;
- }
-
-@@ -4814,27 +4832,27 @@ static PyMethodDef SwigMethods[] = {
- }
-
- static PyMethodDef SwigMethods[] = {
-- { (char *)"SWIG_PyInstanceMethod_New", (PyCFunction)SWIG_PyInstanceMethod_New, METH_O, NULL},
-- { (char *)"inx_addr_sin_set", _wrap_inx_addr_sin_set, METH_VARARGS, NULL},
-- { (char *)"inx_addr_sin_get", _wrap_inx_addr_sin_get, METH_VARARGS, NULL},
-- { (char *)"inx_addr_sin6_set", _wrap_inx_addr_sin6_set, METH_VARARGS, NULL},
-- { (char *)"inx_addr_sin6_get", _wrap_inx_addr_sin6_get, METH_VARARGS, NULL},
-- { (char *)"new_inx_addr", _wrap_new_inx_addr, METH_VARARGS, NULL},
-- { (char *)"delete_inx_addr", _wrap_delete_inx_addr, METH_VARARGS, NULL},
-- { (char *)"inx_addr_swigregister", inx_addr_swigregister, METH_VARARGS, NULL},
-- { (char *)"new_SubnetTree", _wrap_new_SubnetTree, METH_VARARGS, NULL},
-- { (char *)"delete_SubnetTree", _wrap_delete_SubnetTree, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_insert", _wrap_SubnetTree_insert, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_remove", _wrap_SubnetTree_remove, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_lookup", _wrap_SubnetTree_lookup, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_prefixes", _wrap_SubnetTree_prefixes, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_get_binary_lookup_mode", _wrap_SubnetTree_get_binary_lookup_mode, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_set_binary_lookup_mode", _wrap_SubnetTree_set_binary_lookup_mode, METH_VARARGS, NULL},
-- { (char *)"SubnetTree___contains__", _wrap_SubnetTree___contains__, METH_VARARGS, NULL},
-- { (char *)"SubnetTree___getitem__", _wrap_SubnetTree___getitem__, METH_VARARGS, NULL},
-- { (char *)"SubnetTree___setitem__", _wrap_SubnetTree___setitem__, METH_VARARGS, NULL},
-- { (char *)"SubnetTree___delitem__", _wrap_SubnetTree___delitem__, METH_VARARGS, NULL},
-- { (char *)"SubnetTree_swigregister", SubnetTree_swigregister, METH_VARARGS, NULL},
-+ { "SWIG_PyInstanceMethod_New", SWIG_PyInstanceMethod_New, METH_O, NULL},
-+ { "inx_addr_sin_set", _wrap_inx_addr_sin_set, METH_VARARGS, NULL},
-+ { "inx_addr_sin_get", _wrap_inx_addr_sin_get, METH_VARARGS, NULL},
-+ { "inx_addr_sin6_set", _wrap_inx_addr_sin6_set, METH_VARARGS, NULL},
-+ { "inx_addr_sin6_get", _wrap_inx_addr_sin6_get, METH_VARARGS, NULL},
-+ { "new_inx_addr", _wrap_new_inx_addr, METH_VARARGS, NULL},
-+ { "delete_inx_addr", _wrap_delete_inx_addr, METH_VARARGS, NULL},
-+ { "inx_addr_swigregister", inx_addr_swigregister, METH_VARARGS, NULL},
-+ { "new_SubnetTree", _wrap_new_SubnetTree, METH_VARARGS, NULL},
-+ { "delete_SubnetTree", _wrap_delete_SubnetTree, METH_VARARGS, NULL},
-+ { "SubnetTree_insert", _wrap_SubnetTree_insert, METH_VARARGS, NULL},
-+ { "SubnetTree_remove", _wrap_SubnetTree_remove, METH_VARARGS, NULL},
-+ { "SubnetTree_lookup", _wrap_SubnetTree_lookup, METH_VARARGS, NULL},
-+ { "SubnetTree_prefixes", _wrap_SubnetTree_prefixes, METH_VARARGS, NULL},
-+ { "SubnetTree_get_binary_lookup_mode", _wrap_SubnetTree_get_binary_lookup_mode, METH_VARARGS, NULL},
-+ { "SubnetTree_set_binary_lookup_mode", _wrap_SubnetTree_set_binary_lookup_mode, METH_VARARGS, NULL},
-+ { "SubnetTree___contains__", _wrap_SubnetTree___contains__, METH_VARARGS, NULL},
-+ { "SubnetTree___getitem__", _wrap_SubnetTree___getitem__, METH_VARARGS, NULL},
-+ { "SubnetTree___setitem__", _wrap_SubnetTree___setitem__, METH_VARARGS, NULL},
-+ { "SubnetTree___delitem__", _wrap_SubnetTree___delitem__, METH_VARARGS, NULL},
-+ { "SubnetTree_swigregister", SubnetTree_swigregister, METH_VARARGS, NULL},
- { NULL, NULL, 0, NULL }
- };
-
-@@ -5399,9 +5417,9 @@ extern "C" {
- char *ndoc = (char*)malloc(ldoc + lptr + 10);
- if (ndoc) {
- char *buff = ndoc;
-- strncpy(buff, methods[i].ml_doc, ldoc);
-+ memcpy(buff, methods[i].ml_doc, ldoc);
- buff += ldoc;
-- strncpy(buff, "swig_ptr: ", 10);
-+ memcpy(buff, "swig_ptr: ", 10);
- buff += 10;
- SWIG_PackVoidPtr(buff, ptr, ty->name, lptr);
- methods[i].ml_doc = ndoc;
-@@ -5463,8 +5481,8 @@ SWIG_init(void) {
- (char *)"this", &SwigPyBuiltin_ThisClosure, NULL, NULL, NULL
- };
- static SwigPyGetSet thisown_getset_closure = {
-- (PyCFunction) SwigPyObject_own,
-- (PyCFunction) SwigPyObject_own
-+ SwigPyObject_own,
-+ SwigPyObject_own
- };
- static PyGetSetDef thisown_getset_def = {
- (char *)"thisown", SwigPyBuiltin_GetterClosure, SwigPyBuiltin_SetterClosure, NULL, &thisown_getset_closure
diff --git a/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_include_SubnetTree.h b/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_include_SubnetTree.h
deleted file mode 100644
index 5b7a96b92efe..000000000000
--- a/security/zeek/files/patch-auxil_zeekctl_auxil_pysubnettree_include_SubnetTree.h
+++ /dev/null
@@ -1,11 +0,0 @@
---- auxil/zeekctl/auxil/pysubnettree/include/SubnetTree.h.orig 2024-05-16 17:25:57 UTC
-+++ auxil/zeekctl/auxil/pysubnettree/include/SubnetTree.h
-@@ -147,7 +147,7 @@ class SubnetTree (public)
-
- PyObject* data = self->lookup(cidr, size);
- if ( ! data ) {
-- PyErr_SetString(PyExc_KeyError, cidr);
-+ PyErr_SetObject(PyExc_KeyError, PyBytes_FromStringAndSize(cidr, size));
- return 0;
- }
-
diff --git a/security/zeek/files/patch-src_threading_Manager.cc b/security/zeek/files/patch-src_threading_Manager.cc
deleted file mode 100644
index 9bafbd8c8b52..000000000000
--- a/security/zeek/files/patch-src_threading_Manager.cc
+++ /dev/null
@@ -1,47 +0,0 @@
---- src/threading/Manager.cc.orig 2024-05-16 17:25:52 UTC
-+++ src/threading/Manager.cc
-@@ -65,8 +65,12 @@ void Manager::Terminate()
- delete *i;
- }
-
-+ for ( auto* iosource : io_sources )
-+ delete iosource;
-+
- all_threads.clear();
- msg_threads.clear();
-+ io_sources.clear();
- terminating = false;
- }
-
-@@ -79,10 +83,11 @@ void Manager::AddThread(BasicThread* thread)
- StartHeartbeatTimer();
- }
-
--void Manager::AddMsgThread(MsgThread* thread)
-+void Manager::AddMsgThread(MsgThread* thread, iosource::IOSource* iosource)
- {
- DBG_LOG(DBG_THREADING, "%s is a MsgThread ...", thread->Name());
- msg_threads.push_back(thread);
-+ io_sources.push_back(iosource);
- }
-
- void Manager::KillThreads()
-@@ -129,6 +134,18 @@ void Manager::SendHeartbeats()
-
- t->Join();
- delete t;
-+ }
-+
-+ for ( auto it = io_sources.begin(); it != io_sources.end(); /**/ )
-+ {
-+ auto* src = *it;
-+ if ( ! src->IsOpen() )
-+ {
-+ delete src;
-+ it = io_sources.erase(it);
-+ }
-+ else
-+ ++it;
- }
- }
-
diff --git a/security/zeek/files/patch-src_threading_Manager.h b/security/zeek/files/patch-src_threading_Manager.h
deleted file mode 100644
index a75d05d8c79d..000000000000
--- a/security/zeek/files/patch-src_threading_Manager.h
+++ /dev/null
@@ -1,23 +0,0 @@
---- src/threading/Manager.h.orig 2024-05-16 17:25:52 UTC
-+++ src/threading/Manager.h
-@@ -127,8 +127,9 @@ class Manager (protected)
- * MsgThread constructor makes sure to do so.
- *
- * @param thread The thread.
-+ * @param iosource The IO source of the thread.
- */
-- void AddMsgThread(MsgThread* thread);
-+ void AddMsgThread(MsgThread* thread, iosource::IOSource* iosource);
-
- void Flush();
-
-@@ -148,6 +149,9 @@ class Manager (protected)
-
- using msg_thread_list = std::list<MsgThread*>;
- msg_thread_list msg_threads;
-+
-+ using io_source_list = std::list<iosource::IOSource*>;
-+ io_source_list io_sources;
-
- bool did_process; // True if the last Process() found some work to do.
- double next_beat; // Timestamp when the next heartbeat will be sent.
diff --git a/security/zeek/files/patch-src_threading_MsgThread.cc b/security/zeek/files/patch-src_threading_MsgThread.cc
deleted file mode 100644
index 07a7bd863a46..000000000000
--- a/security/zeek/files/patch-src_threading_MsgThread.cc
+++ /dev/null
@@ -1,146 +0,0 @@
---- src/threading/MsgThread.cc.orig 2024-05-16 17:25:52 UTC
-+++ src/threading/MsgThread.cc
-@@ -213,7 +213,77 @@ bool ReporterMessage::Process()
-
- return true;
- }
-+//
-+// The lifetime of the IO source is decoupled from
-+// the thread. The thread may be terminated prior
-+// to the IO source being properly unregistered and
-+// forgotten by the IO manager. Specifically the
-+// threading manager would delete an IO source which
-+// the IO manager still believed to be ready.
-+//
-+// See issue #3682 for more details.
-+class MsgThread_IOSource : public iosource::IOSource
-+ {
-+public:
-+ explicit MsgThread_IOSource(MsgThread* thread) : thread(thread)
-+ {
-+ if ( ! iosource_mgr->RegisterFd(flare.FD(), this) )
-+ reporter->FatalError("Failed to register MsgThread fd with iosource_mgr");
-
-+ SetClosed(false);
-+ }
-+
-+ ~MsgThread_IOSource()
-+ {
-+ if ( IsOpen() )
-+ {
-+ if ( thread )
-+ reporter->Warning("Have thread %s set in MsgThread_IOSource", thread->Name());
-+
-+ if ( ! iosource_mgr->UnregisterFd(flare.FD(), this) )
-+ reporter->FatalError("Failed to unregister MsgThread fd from iosource_mgr");
-+ }
-+ }
-+
-+ void Process() override
-+ {
-+ flare.Extinguish();
-+
-+ if ( thread )
-+ thread->Process();
-+ else
-+ {
-+ // When there's no thread anymore, unregister
-+ // this source from the IO manager and mark
-+ // it as closed. The threading manager will then
-+ // reap it during heartbeat processing or shutdown.
-+ if ( ! iosource_mgr->UnregisterFd(flare.FD(), this) )
-+ reporter->FatalError("Failed to unregister MsgThread fd from iosource_mgr");
-+
-+ SetClosed(true);
-+ }
-+ }
-+
-+ const char* Tag() override { return thread ? thread->Name() : "<MsgThread_IOSource orphan>"; }
-+
-+ double GetNextTimeout() override { return -1; }
-+
-+ void Fire() { flare.Fire(); };
-+
-+ // Fire the flare one more time so that
-+ // the IO manager will call Process() and
-+ // SetClosed(true).
-+ void Close()
-+ {
-+ thread = nullptr;
-+ flare.Fire();
-+ }
-+
-+private:
-+ MsgThread* thread = nullptr;
-+ zeek::detail::Flare flare;
-+ };
-+
- } // namespace detail
-
- ////// Methods.
-@@ -232,19 +302,22 @@ MsgThread::MsgThread() : BasicThread(), queue_in(this,
- child_finished = false;
- child_sent_finish = false;
- failed = false;
-- thread_mgr->AddMsgThread(this);
-
-- if ( ! iosource_mgr->RegisterFd(flare.FD(), this) )
-- reporter->FatalError("Failed to register MsgThread fd with iosource_mgr");
--
-- SetClosed(false);
-+ io_source = new detail::MsgThread_IOSource(this);
-+ thread_mgr->AddMsgThread(this, io_source);
- }
-
- MsgThread::~MsgThread()
- {
-- // Unregister this thread from the iosource manager so it doesn't wake
-- // up the main poll anymore.
-- iosource_mgr->UnregisterFd(flare.FD(), this);
-+ // Unregister this thread from the IO source so we don't
-+ // get Process() callbacks anymore. The IO source is
-+ // freed by separately by the threading manager after its
-+ // last Process() invocation.
-+ if ( io_source )
-+ {
-+ io_source->Close();
-+ io_source = nullptr;
-+ }
- }
-
- void MsgThread::OnSignalStop()
-@@ -319,7 +392,14 @@ void MsgThread::OnKill()
-
- void MsgThread::OnKill()
- {
-- SetClosed(true);
-+ // Ensure the IO source is closed and won't call Process() on this
-+ // thread anymore. The thread got killed, so the threading manager will
-+ // remove it forcefully soon.
-+ if ( io_source )
-+ {
-+ io_source->Close();
-+ io_source = nullptr;
-+ }
-
- // Send a message to unblock the reader if its currently waiting for
- // input. This is just an optimization to make it terminate more
-@@ -432,7 +512,8 @@ void MsgThread::SendOut(BasicOutputMessage* msg, bool
-
- ++cnt_sent_out;
-
-- flare.Fire();
-+ if ( io_source )
-+ io_source->Fire();
- }
-
- void MsgThread::SendEvent(const char* name, const int num_vals, Value** vals)
-@@ -514,8 +595,6 @@ void MsgThread::Process()
-
- void MsgThread::Process()
- {
-- flare.Extinguish();
--
- while ( HasOut() )
- {
- Message* msg = RetrieveOut();
diff --git a/security/zeek/files/patch-src_threading_MsgThread.h b/security/zeek/files/patch-src_threading_MsgThread.h
deleted file mode 100644
index 1daab96b7e0c..000000000000
--- a/security/zeek/files/patch-src_threading_MsgThread.h
+++ /dev/null
@@ -1,62 +0,0 @@
---- src/threading/MsgThread.h.orig 2024-05-16 17:25:52 UTC
-+++ src/threading/MsgThread.h
-@@ -30,6 +30,8 @@ class KillMeMessage;
- class FinishedMessage;
- class KillMeMessage;
-
-+class MsgThread_IOSource;
-+
- }
-
- /**
-@@ -43,7 +45,7 @@ class KillMeMessage;
- * that happens, the thread stops accepting any new messages, finishes
- * processes all remaining ones still in the queue, and then exits.
- */
--class MsgThread : public BasicThread, public iosource::IOSource
-+class MsgThread : public BasicThread
- {
- public:
- /**
-@@ -213,19 +215,13 @@ class MsgThread : public BasicThread, public iosource:
- */
- void GetStats(Stats* stats);
-
-- /**
-- * Overridden from iosource::IOSource.
-- */
-- void Process() override;
-- const char* Tag() override { return Name(); }
-- double GetNextTimeout() override { return -1; }
--
- protected:
- friend class Manager;
- friend class detail::HeartbeatMessage;
- friend class detail::FinishMessage;
- friend class detail::FinishedMessage;
- friend class detail::KillMeMessage;
-+ friend class detail::MsgThread_IOSource;
-
- /**
- * Pops a message sent by the child from the child-to-main queue.
-@@ -291,6 +287,11 @@ class MsgThread : public BasicThread, public iosource:
- */
- virtual const zeek::detail::Location* GetLocationInfo() const { return nullptr; }
-
-+ /**
-+ * Process() forwarded by MsgThread_IOSource.
-+ */
-+ void Process();
-+
- private:
- /**
- * Pops a message sent by the main thread from the main-to-chold
-@@ -367,7 +368,7 @@ class MsgThread : public BasicThread, public iosource:
- bool child_sent_finish; // Child thread asked to be finished.
- bool failed; // Set to true when a command failed.
-
-- zeek::detail::Flare flare;
-+ detail::MsgThread_IOSource* io_source = nullptr; // IO source registered with the IO manager.
- };
-
- /**
diff --git a/security/zeek/pkg-plist b/security/zeek/pkg-plist
index 07ef541c490c..eaf13a68c02f 100644
--- a/security/zeek/pkg-plist
+++ b/security/zeek/pkg-plist
@@ -40,11 +40,14 @@ include/broker/backend.hh
include/broker/backend_options.hh
include/broker/bad_variant_access.hh
include/broker/broker.hh
+include/broker/builder.hh
+include/broker/command_envelope.hh
include/broker/config.hh
include/broker/configuration.hh
include/broker/convert.hh
include/broker/cow_tuple.hh
include/broker/data.hh
+include/broker/data_envelope.hh
include/broker/defaults.hh
include/broker/detail/abstract_backend.hh
include/broker/detail/algorithms.hh
@@ -67,6 +70,7 @@ include/broker/detail/overload.hh
include/broker/detail/peer_status_map.hh
include/broker/detail/pp.hh
include/broker/detail/prefix_matcher.hh
+include/broker/detail/promote.hh
include/broker/detail/radix_tree.hh
include/broker/detail/sink_driver.hh
include/broker/detail/source_driver.hh
@@ -79,23 +83,32 @@ include/broker/endpoint_id.hh
include/broker/endpoint_info.hh
include/broker/entity_id.hh
include/broker/enum_value.hh
+include/broker/envelope.hh
include/broker/error.hh
include/broker/expected.hh
include/broker/filter_type.hh
+include/broker/format/bin.hh
+include/broker/format/json.hh
+include/broker/format/txt.hh
include/broker/frontend.hh
include/broker/fwd.hh
include/broker/internal_command.hh
+include/broker/intrusive_ptr.hh
include/broker/lamport_timestamp.hh
include/broker/mailbox.hh
include/broker/message.hh
include/broker/network_info.hh
include/broker/none.hh
+include/broker/p2p_message_type.hh
include/broker/peer_flags.hh
include/broker/peer_info.hh
include/broker/peer_status.hh
+include/broker/ping_envelope.hh
+include/broker/pong_envelope.hh
include/broker/port.hh
include/broker/publisher.hh
include/broker/publisher_id.hh
+include/broker/routing_update_envelope.hh
include/broker/shutdown_options.hh
include/broker/snapshot.hh
include/broker/span.hh
@@ -105,30 +118,29 @@ include/broker/store.hh
include/broker/store_event.hh
include/broker/subnet.hh
include/broker/subscriber.hh
-include/broker/telemetry/counter.hh
-include/broker/telemetry/fwd.hh
-include/broker/telemetry/gauge.hh
-include/broker/telemetry/histogram.hh
-include/broker/telemetry/metric_family.hh
-include/broker/telemetry/metric_registry.hh
-include/broker/telemetry/metric_registry_impl.hh
include/broker/time.hh
include/broker/timeout.hh
include/broker/topic.hh
+include/broker/variant.hh
+include/broker/variant_data.hh
+include/broker/variant_list.hh
+include/broker/variant_set.hh
+include/broker/variant_table.hh
+include/broker/variant_tag.hh
include/broker/version.hh
+include/broker/visit.hh
include/broker/worker.hh
include/broker/zeek.hh
%%SPICY%%include/hilti/ast/all.h
-%%SPICY%%include/hilti/ast/ast.h
+%%SPICY%%include/hilti/ast/ast-context.h
%%SPICY%%include/hilti/ast/attribute.h
%%SPICY%%include/hilti/ast/builder/all.h
%%SPICY%%include/hilti/ast/builder/builder.h
-%%SPICY%%include/hilti/ast/builder/declaration.h
-%%SPICY%%include/hilti/ast/builder/expression.h
-%%SPICY%%include/hilti/ast/builder/type.h
+%%SPICY%%include/hilti/ast/builder/node-factory.h
%%SPICY%%include/hilti/ast/ctor.h
%%SPICY%%include/hilti/ast/ctors/address.h
%%SPICY%%include/hilti/ast/ctors/all.h
+%%SPICY%%include/hilti/ast/ctors/bitfield.h
%%SPICY%%include/hilti/ast/ctors/bool.h
%%SPICY%%include/hilti/ast/ctors/bytes.h
%%SPICY%%include/hilti/ast/ctors/coerced.h
@@ -166,12 +178,11 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/declarations/global-variable.h
%%SPICY%%include/hilti/ast/declarations/imported-module.h
%%SPICY%%include/hilti/ast/declarations/local-variable.h
+%%SPICY%%include/hilti/ast/declarations/module-uid.h
%%SPICY%%include/hilti/ast/declarations/module.h
%%SPICY%%include/hilti/ast/declarations/parameter.h
%%SPICY%%include/hilti/ast/declarations/property.h
%%SPICY%%include/hilti/ast/declarations/type.h
-%%SPICY%%include/hilti/ast/detail/operator-registry.h
-%%SPICY%%include/hilti/ast/detail/visitor.h
%%SPICY%%include/hilti/ast/doc-string.h
%%SPICY%%include/hilti/ast/expression.h
%%SPICY%%include/hilti/ast/expressions/all.h
@@ -181,7 +192,6 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/expressions/ctor.h
%%SPICY%%include/hilti/ast/expressions/deferred.h
%%SPICY%%include/hilti/ast/expressions/grouping.h
-%%SPICY%%include/hilti/ast/expressions/id.h
%%SPICY%%include/hilti/ast/expressions/keyword.h
%%SPICY%%include/hilti/ast/expressions/list-comprehension.h
%%SPICY%%include/hilti/ast/expressions/logical-and.h
@@ -189,6 +199,7 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/expressions/logical-or.h
%%SPICY%%include/hilti/ast/expressions/member.h
%%SPICY%%include/hilti/ast/expressions/move.h
+%%SPICY%%include/hilti/ast/expressions/name.h
%%SPICY%%include/hilti/ast/expressions/pending-coerced.h
%%SPICY%%include/hilti/ast/expressions/resolved-operator.h
%%SPICY%%include/hilti/ast/expressions/ternary.h
@@ -197,16 +208,19 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/expressions/typeinfo.h
%%SPICY%%include/hilti/ast/expressions/unresolved-operator.h
%%SPICY%%include/hilti/ast/expressions/void.h
+%%SPICY%%include/hilti/ast/forward.h
%%SPICY%%include/hilti/ast/function.h
%%SPICY%%include/hilti/ast/id.h
%%SPICY%%include/hilti/ast/location.h
%%SPICY%%include/hilti/ast/meta.h
-%%SPICY%%include/hilti/ast/module.h
-%%SPICY%%include/hilti/ast/node-ref.h
+%%SPICY%%include/hilti/ast/node-range.h
+%%SPICY%%include/hilti/ast/node-tag.h
%%SPICY%%include/hilti/ast/node.h
+%%SPICY%%include/hilti/ast/operator-registry.h
%%SPICY%%include/hilti/ast/operator.h
%%SPICY%%include/hilti/ast/operators/address.h
%%SPICY%%include/hilti/ast/operators/all.h
+%%SPICY%%include/hilti/ast/operators/bitfield.h
%%SPICY%%include/hilti/ast/operators/bool.h
%%SPICY%%include/hilti/ast/operators/bytes.h
%%SPICY%%include/hilti/ast/operators/common.h
@@ -215,6 +229,7 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/operators/exception.h
%%SPICY%%include/hilti/ast/operators/function.h
%%SPICY%%include/hilti/ast/operators/generic.h
+%%SPICY%%include/hilti/ast/operators/integer.h
%%SPICY%%include/hilti/ast/operators/interval.h
%%SPICY%%include/hilti/ast/operators/list.h
%%SPICY%%include/hilti/ast/operators/map.h
@@ -226,14 +241,12 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/operators/regexp.h
%%SPICY%%include/hilti/ast/operators/result.h
%%SPICY%%include/hilti/ast/operators/set.h
-%%SPICY%%include/hilti/ast/operators/signed-integer.h
%%SPICY%%include/hilti/ast/operators/stream.h
%%SPICY%%include/hilti/ast/operators/string.h
%%SPICY%%include/hilti/ast/operators/struct.h
%%SPICY%%include/hilti/ast/operators/time.h
%%SPICY%%include/hilti/ast/operators/tuple.h
%%SPICY%%include/hilti/ast/operators/union.h
-%%SPICY%%include/hilti/ast/operators/unsigned-integer.h
%%SPICY%%include/hilti/ast/operators/vector.h
%%SPICY%%include/hilti/ast/scope-lookup.h
%%SPICY%%include/hilti/ast/scope.h
@@ -260,6 +273,7 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/types/all.h
%%SPICY%%include/hilti/ast/types/any.h
%%SPICY%%include/hilti/ast/types/auto.h
+%%SPICY%%include/hilti/ast/types/bitfield.h
%%SPICY%%include/hilti/ast/types/bool.h
%%SPICY%%include/hilti/ast/types/bytes.h
%%SPICY%%include/hilti/ast/types/doc-only.h
@@ -273,6 +287,7 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/types/list.h
%%SPICY%%include/hilti/ast/types/map.h
%%SPICY%%include/hilti/ast/types/member.h
+%%SPICY%%include/hilti/ast/types/name.h
%%SPICY%%include/hilti/ast/types/network.h
%%SPICY%%include/hilti/ast/types/null.h
%%SPICY%%include/hilti/ast/types/operand-list.h
@@ -291,53 +306,44 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/ast/types/type.h
%%SPICY%%include/hilti/ast/types/union.h
%%SPICY%%include/hilti/ast/types/unknown.h
-%%SPICY%%include/hilti/ast/types/unresolved-id.h
%%SPICY%%include/hilti/ast/types/vector.h
%%SPICY%%include/hilti/ast/types/void.h
-%%SPICY%%include/hilti/autogen/__ctor.h
-%%SPICY%%include/hilti/autogen/__declaration.h
-%%SPICY%%include/hilti/autogen/__dispatchers.h
-%%SPICY%%include/hilti/autogen/__expression.h
-%%SPICY%%include/hilti/autogen/__node.h
-%%SPICY%%include/hilti/autogen/__operator.h
-%%SPICY%%include/hilti/autogen/__resolved-operator.h
-%%SPICY%%include/hilti/autogen/__statement.h
-%%SPICY%%include/hilti/autogen/__type.h
+%%SPICY%%include/hilti/ast/visitor-dispatcher.h
+%%SPICY%%include/hilti/ast/visitor.h
%%SPICY%%include/hilti/autogen/config.h
%%SPICY%%include/hilti/base/cache.h
%%SPICY%%include/hilti/base/code-formatter.h
%%SPICY%%include/hilti/base/id-base.h
%%SPICY%%include/hilti/base/intrusive-ptr.h
%%SPICY%%include/hilti/base/logger.h
-%%SPICY%%include/hilti/base/optional-ref.h
%%SPICY%%include/hilti/base/preprocessor.h
%%SPICY%%include/hilti/base/result.h
%%SPICY%%include/hilti/base/timing.h
-%%SPICY%%include/hilti/base/type_erase.h
%%SPICY%%include/hilti/base/uniquer.h
%%SPICY%%include/hilti/base/util.h
-%%SPICY%%include/hilti/base/visitor-types.h
-%%SPICY%%include/hilti/base/visitor-util.h
-%%SPICY%%include/hilti/base/visitor.h
-%%SPICY%%include/hilti/compiler/coercion.h
+%%SPICY%%include/hilti/compiler/coercer.h
%%SPICY%%include/hilti/compiler/context.h
+%%SPICY%%include/hilti/compiler/detail/ast-dumper.h
%%SPICY%%include/hilti/compiler/detail/codegen/codegen.h
+%%SPICY%%include/hilti/compiler/detail/constant-folder.h
%%SPICY%%include/hilti/compiler/detail/cxx/all.h
%%SPICY%%include/hilti/compiler/detail/cxx/elements.h
%%SPICY%%include/hilti/compiler/detail/cxx/formatter.h
%%SPICY%%include/hilti/compiler/detail/cxx/linker.h
%%SPICY%%include/hilti/compiler/detail/cxx/unit.h
+%%SPICY%%include/hilti/compiler/detail/optimizer.h
%%SPICY%%include/hilti/compiler/detail/parser/driver.h
%%SPICY%%include/hilti/compiler/detail/parser/scanner.h
-%%SPICY%%include/hilti/compiler/detail/visitors.h
+%%SPICY%%include/hilti/compiler/detail/resolver.h
+%%SPICY%%include/hilti/compiler/detail/scope-builder.h
%%SPICY%%include/hilti/compiler/driver.h
%%SPICY%%include/hilti/compiler/init.h
%%SPICY%%include/hilti/compiler/jit.h
-%%SPICY%%include/hilti/compiler/optimizer.h
%%SPICY%%include/hilti/compiler/plugin.h
%%SPICY%%include/hilti/compiler/printer.h
+%%SPICY%%include/hilti/compiler/type-unifier.h
%%SPICY%%include/hilti/compiler/unit.h
-%%SPICY%%include/hilti/global.h
+%%SPICY%%include/hilti/compiler/validator.h
%%SPICY%%include/hilti/hilti.h
%%SPICY%%include/hilti/rt/3rdparty/ArticleEnumClass-v2/EnumClass.h
%%SPICY%%include/hilti/rt/3rdparty/SafeInt/SafeInt.hpp
@@ -392,6 +398,7 @@ include/broker/zeek.hh
%%SPICY%%include/hilti/rt/types/address.h
%%SPICY%%include/hilti/rt/types/all.h
%%SPICY%%include/hilti/rt/types/any.h
+%%SPICY%%include/hilti/rt/types/bitfield.h
%%SPICY%%include/hilti/rt/types/bool.h
%%SPICY%%include/hilti/rt/types/bytes.h
%%SPICY%%include/hilti/rt/types/enum.h
@@ -424,18 +431,18 @@ include/paraglob/exceptions.h
include/paraglob/node.h
include/paraglob/paraglob.h
include/paraglob/serializer.h
-%%SPICY%%include/spicy/ast/aliases.h
%%SPICY%%include/spicy/ast/all.h
+%%SPICY%%include/spicy/ast/builder/builder.h
+%%SPICY%%include/spicy/ast/builder/node-factory.h
%%SPICY%%include/spicy/ast/ctors/all.h
%%SPICY%%include/spicy/ast/ctors/unit.h
%%SPICY%%include/spicy/ast/declarations/all.h
+%%SPICY%%include/spicy/ast/declarations/hook.h
%%SPICY%%include/spicy/ast/declarations/unit-hook.h
-%%SPICY%%include/spicy/ast/detail/visitor.h
-%%SPICY%%include/spicy/ast/engine.h
-%%SPICY%%include/spicy/ast/expressions/all.h
-%%SPICY%%include/spicy/ast/hook.h
+%%SPICY%%include/spicy/ast/forward.h
+%%SPICY%%include/spicy/ast/node-tag.h
+%%SPICY%%include/spicy/ast/node.h
%%SPICY%%include/spicy/ast/operators/all.h
-%%SPICY%%include/spicy/ast/operators/bitfield.h
%%SPICY%%include/spicy/ast/operators/sink.h
%%SPICY%%include/spicy/ast/operators/unit.h
%%SPICY%%include/spicy/ast/statements/all.h
@@ -443,9 +450,7 @@ include/paraglob/serializer.h
%%SPICY%%include/spicy/ast/statements/print.h
%%SPICY%%include/spicy/ast/statements/reject.h
%%SPICY%%include/spicy/ast/statements/stop.h
-%%SPICY%%include/spicy/ast/types.h
%%SPICY%%include/spicy/ast/types/all.h
-%%SPICY%%include/spicy/ast/types/bitfield.h
%%SPICY%%include/spicy/ast/types/sink.h
%%SPICY%%include/spicy/ast/types/unit-item.h
%%SPICY%%include/spicy/ast/types/unit-items/all.h
@@ -457,10 +462,8 @@ include/paraglob/serializer.h
%%SPICY%%include/spicy/ast/types/unit-items/unresolved-field.h
%%SPICY%%include/spicy/ast/types/unit-items/variable.h
%%SPICY%%include/spicy/ast/types/unit.h
-%%SPICY%%include/spicy/autogen/__dispatchers-productions.h
-%%SPICY%%include/spicy/autogen/__dispatchers.h
-%%SPICY%%include/spicy/autogen/__production.h
-%%SPICY%%include/spicy/autogen/__unit-item.h
*** 393 LINES SKIPPED ***