Windows Event Forwarding (WEF) Collector: Internal Operating Notes

Scope

Internal operating notes for the logsign-wef-collector service. The customer facing setup guide covers certificates, device registration, the Windows side and normal verification. This note covers what support and engineering need on top of that: service flags, log lines, where every dropped event is counted, and the boundaries of the delivery guarantee. Everything below was read from the service source on the develop branch and matches the behaviour of the shipped unit file.

Service and Flags

The service is defined in /usr/lib/systemd/system/logsign-wef-collector.service. Override it with systemctl edit logsign-wef-collector rather than editing the unit in place.

Note the difference between the binary defaults and the values the shipped unit passes. The binary on its own listens on HTTP 5985 and has HTTPS disabled. The unit file inverts that, which is what an installed appliance actually runs.

At least one of -wef.http.port or -wef.https.port must be non-zero, otherwise the service exits at startup with a fatal error.

Reading the Service Log

A successful enrollment produces these three lines, in this order:

journalctl -u logsign-wef-collector -f | grep -E 'HANDSHAKE-OK|SUBSCRIPTION-LIVE|Dropping|rejected'

[HANDSHAKE-OK] Identify ack delivered to 10.0.0.31:49890, client may now Enumerate
[HANDSHAKE-OK] Subscription policies delivered to 10.0.0.31:49890, notifyTo=https://... subs=[...]
[SUBSCRIPTION-LIVE] First event received from 10.0.0.31:49892 (UID=...)

SUBSCRIPTION-LIVE is the line that matters. The two handshake lines only prove that the conversation started, not that any event was delivered.

Two log behaviours are worth knowing before you conclude that a machine is silent:

• Each machine's handshake is logged once at info level. Repeats drop to debug, because a fleet of a few hundred forwarders would otherwise fill the journal with handshakes alone.
• An empty EnumerateResponse is logged as a warning, and it means no subscription policy is configured for that source. The client will connect happily and never deliver anything.

Occasional TLS handshake error ... EOF lines from a healthy machine are normal probe connections.

Where Dropped Events Are Counted

The collector never discards an event without counting it, and the counters are the complete record. The service log names the reason and is rate limited where a misbehaving machine could otherwise flood it, so during a storm the counters keep rising while the journal stays readable. Answer "did we lose anything, and why" from the device page first, and the log second.

Counted per event:

Counted per refused request, because source validation runs before the collector opens the request body. A refused Identify and a refused 50 event batch therefore cost exactly one each, and neither loses data, since the forwarder keeps its bookmark and re-sends:

An unregistered sender belongs to no device, so it appears only in the unfiltered host list, never in a device's own Forwarding machines table.

Undecodable payloads travel to raw_persist only when raw persistence is enabled for that source. If it is switched off, what remains is the journal entry: byte count, digest and a short escaped sample, which identifies a recurring payload but is not a copy of it. Those entries are rate limited to one per machine per minute and to a few payloads within it, and anything skipped meanwhile is reported in the next entry rather than repeated per batch or dropped silently. Refused requests are logged the same way and share the limit. The complete body and headers are available on the [WEF-RAW] line when the service runs at -loglevel=debug, which is also the only level at which the collector spends anything on formatting them.

Limits Outside This Collector

• Acknowledgement means the events were handed to the pipeline, not that they are on disk. The ingress is core NATS, exactly as for every other collector, so a crash between handover and delivery is not covered by the acknowledgement.
• Further down the pipeline, logsign-persist drops when its own queue is saturated. That is counted as raw_persist|failed|<source>|wef|<node> and logged as persist channel full, dropping message.
• Per-machine statistics are written with a 7 day retention. The noisier counters, meaning rejected and unregistered requests, use a 1 day retention.

Certificate Handling

• On the Logsign issued path, the CA is created with a 10 year validity, and server and client certificates with 825 days, which is the maximum Windows accepts.
• The subscription URL is generated as Server=https://<collector>:5986/wsman/SubscriptionManager/WEC,Refresh=60,IssuerCA=<thumbprint>. The refresh interval of 60 seconds is fixed in that string.
• Regenerating or replacing the CA invalidates every issued client certificate. The interface counts the enrolled machines that will stop delivering and asks for confirmation before saving.
• When the customer supplies their own PKI, the three PEM blocks are validated together before storage: key and certificate must match, the certificate must chain to the supplied CA, and it must carry a subjectAltName. Each of these would otherwise surface on the Windows side only as a forwarder that never enrolls.

Quick Commands

systemctl status logsign-wef-collector
journalctl -u logsign-wef-collector -f
ss -lntp | grep 5986                     # is it listening?
systemctl edit logsign-wef-collector     # add -loglevel=debug to ExecStart when debugging one machine