Possible regression in 1.12.0: SIGSEGV during LogEvent value copy on Linux x86_64

[cgsergey]

Summary

After upgrading swift-log from 1.11.0 to 1.12.0, a Linux Swift server
crash-loops with SIGSEGV (exit code 139) every time a code path that
emits a log line through a multiplexed handler chain executes. The
same binary on macOS does not crash. Pinning swift-log back to
1.11.x stops the crash entirely.

Crash signature (identical across hundreds of occurrences)

*** Program crashed: Bad pointer dereference at 0x0000000000000068 ***
Thread N "...default-qos" crashed:
  0  swift_retain + 47
  1  initializeWithCopy for LogEvent + 159
  2  outlined init with copy of LogEvent + 28
  3  closure #1 in <user closure that wraps an HTTP call> + 2095
  4  ...async thunks...
  N  Tracer.withSpan<A>(...)
  N+1 user code that called the logger

Register dump at the fault:

rax 0x0000000000000000
rcx 0x0000000000000068      // offset being dereferenced
rdi 0x0000000200000000      // pointer passed to swift_retain

Environment

• swift-log: 1.12.0 (regression vs 1.11.0)
• swift-otel: 1.1.0 (depends on swift-log >= 1.12.0)
• swift-distributed-tracing: 1.x
• Swift toolchain: 6.x release build
• OS: Ubuntu 24.04 LTS, x86_64
• macOS Apple Silicon: same source, no crash

Setup

Logging is bootstrapped with a MultiplexLogHandler chain that
contains an intercepting wrapper around a stdout handler, plus the
OTel.makeLoggingBackend factory. Any Logger.log(level:_:metadata:)
call from a function that wraps the body in withSpan reliably
triggers the crash within ~2 minutes of process startup.

Suspect

PR #425 in 1.12.0 added error: (any Error)? to LogEvent. The new
field is an existential (~5-word container holding a class reference
in one of its slots). rdi = 0x0000000200000000 looks like
uninitialized memory decoded as Some(class_ref) — high bits set,
low bits zero. swift_retain then dereferences the refcount field
at offset 0x68 from NULL, faulting at address 0x68.

Possible underlying causes (speculation):

• Compiler bug in Optional<any Error> spare-bit / tag-byte encoding
  on Linux x86_64
• @inlinable Logger.log inlining the 1.11.0 LogEvent layout into
  a caller that links 1.12.0 swift-log
• A field-init order issue where the legacy 7-arg LogEvent.init
  delegates to the new 8-arg init and the error slot is observed
  uninitialized under value-witness copy

What we tried

Combination	Result
swift-log 1.11.0 + swift-otel 1.0.5	stable, no restarts in 10+ min
swift-log 1.12.0 + swift-otel 1.1.0	crashes ~every 2 min on every alloc
swift-log 1.12.0 + swift-otel 1.0.5 (downgrade attempt)	still crashes

The third row isolates swift-log 1.12.0 as the regression — the
swift-otel version doesn't matter.

Expected behavior

LogEvent constructed via the legacy 7-arg init (which delegates
to the new 8-arg init with error: nil) should be safe to copy
through a MultiplexLogHandler chain without retaining a bogus
existential.

Happy to provide additional diagnostic output (debug-symbolicated
trace, minimal reproducer) if helpful.

[0xTim]

This is a runtime crash so would be helpful to know the Swift version you're using

[kukushechkin]

@cgsergey thank you for reporting the issue!

The same binary on macOS does not crash

Could you provide a few details about your build setup on both platforms — what toolchain versions are you using, are you building using SPM in both cases and what kind of artifacts do you produce or link against. A minimal reproducer would also help a lot if you're able to put one together.

[cgsergey]

Thanks for the quick response, both of you. Here are the details:

Toolchain

Identical Swift compiler version on both sides:

Side	Toolchain	OS	Arch
Crashes	Swift 6.3.1 release (swift-6.3.1-RELEASE) from the official swift:6.3.1-noble Docker image on Docker Hub. We layer one base image on top of it that only runs apt-get dist-upgrade plus a couple of build-time apt-get install deps, no toolchain changes.	Ubuntu 24.04 LTS (noble)	x86_64
Does not crash	Apple Swift 6.3.1 (swiftlang-6.3.1.1.2 clang-2100.0.123.102), Xcode default toolchain	macOS 26.4 (arm64-apple-macosx26.0)	arm64 (Apple Silicon)

swift --version on the macOS side reports Apple Swift version 6.3.1 (swiftlang-6.3.1.1.2 clang-2100.0.123.102). Inside the Linux container it reports Swift version 6.3.1 (swift-6.3.1-RELEASE). No snapshot toolchain in the mix.

Build setup (SPM on both sides)

Yes, SPM on both. Same Package.swift, same Package.resolved.

Linux is built inside a multi-stage Docker image. The compile step is:

swift build -c release \
  --product Backend \
  --static-swift-stdlib

The runtime image is swift:6.3.1-noble-slim. No custom linker flags beyond --static-swift-stdlib. The executable is the only artifact copied out; we also copy swift-backtrace-static from /usr/libexec/swift/linux/ and set SWIFT_BACKTRACE=enable=yes,sanitize=yes,threads=all,images=all,interactive=no so the crash report you saw in the original issue is produced by swift-backtrace itself, not by gdb / lldb.

macOS is plain swift build / swift run from the same Package.swift, no Xcode project, no special schemes.

Resolved versions when crashing vs. stable

I confirmed this by diffing Package.resolved before and after the pin. These are the only two packages that changed between the crashing build and the stable build:

Package	Crashing build	Stable build
swift-log	1.12.0 (rev 5073617dac96)	1.11.0 (rev 8c0f217f0100)
swift-otel	1.1.0 (rev d1156d2ffc2a)	1.0.5 (rev 1a56b3a854ac)

Everything else in the dependency graph stayed on the same revision across both builds. Notably swift-distributed-tracing stayed on 1.4.1, swift-nio on 2.99.0, swift-nio-http2 on 1.43.0, swift-service-context on 1.3.0, swift-service-lifecycle on 2.11.0, swift-protobuf on 1.37.0, and the gRPC v2 stack on its 2.x versions. Same toolchain bytes (we don't rebuild the Docker base image between runs), same code, same swift build invocation. The only deltas across the broken and fixed runs are the two rows above.

To isolate which of those two is responsible I ran a third combo: swift-log 1.12.0 with swift-otel 1.0.5 pinned back. That still crashed, which is why I'm pointing the finger at swift-log 1.12.0 and PR #425 specifically rather than at swift-otel.

Logging bootstrap

The handler chain installed at startup, in case it matters for the value-witness path:

LoggingSystem.bootstrap { label in
    MultiplexLogHandler([
        InterceptingLogHandler(
            label: label,
            underlying: SimpleLogHandler(label: label, logLevel: logLevel),
            sink: { level, message, metadata, source, file, function, line in
                // forward to in-process telemetry
            }
        ),
        loggingBackend.factory(label)   // OTel.makeLoggingBackend factory
    ])
}

InterceptingLogHandler is internal to our project and conforms to LogHandler in the obvious way. SimpleLogHandler is a stdout pretty-printer. OTel.makeLoggingBackend is from swift-otel 1.1.0 in the crashing build.

Distributed tracing and metrics are bootstrapped the same way through swift-distributed-tracing and swift-metrics, also via swift-otel factories.

Every crashed frame we saw had Tracer.withSpan<A>(...) above the logger frame on the stack, and the offending frame is initializeWithCopy for LogEvent followed by outlined init with copy of LogEvent, called from an async closure inside our HTTP client code path. That stack is identical across every crash report (hundreds of allocs over a 24-hour window before we rolled back), so the trigger is one specific call site, not a race.

Minimal reproducer

I don't yet have a self-contained reproducer. The codebase is a large private monolith and I haven't been able to reduce the trigger to a small swift run script. What I can say:

1. It only fires from a function that wraps its body in withSpan { ... } and then emits one or more log lines from inside the span.
2. With the same source, the same Package.resolved, the same Swift 6.3.1 toolchain, swift-log 1.11.0 is stable for 10+ minutes under the same load; swift-log 1.12.0 crashes within about two minutes on every fresh allocation. Three independent fresh deploys, same result.
3. Switching swift-otel up or down with swift-log 1.12.0 pinned does not change the outcome (always crashes). Switching swift-log down with swift-otel 1.1.0 still pinned does (refuses to resolve, since 1.1.0 requires >= 1.12.0), which is what forced us to pin both.

I'll try to isolate this into a minimal reproducer over the upcoming weekend. Starting point is a fresh swift:6.3.1-noble container with the smallest combination I think still triggers it (async server + withSpan + a MultiplexLogHandler chain + the swift-otel logging backend with an OTLP target), then reducing from there. If I get a clean repro I'll attach it here. If not, I'll attach the next layer of detail (symbolicated backtrace, full Package.resolved, register dumps).

Let me know if there's a specific layer you'd like me to bisect first, or any extra diagnostics to enable in the crashing build.

[kukushechkin]

@cgsergey I start to think there might be some dependency in you project that vendors swift-log of a different version and something somewhere goes terribly wrong. Do you have any frameworks linked? Could you try to bisect those?

I've created a small example with multiplexed logger inside a span: kukushechkin@c15eb7a, but unfortunately cannot reproduce the crash. The compose file runs the service in a x86_64 container. Or did I miss some important detail in my example?

[kukushechkin]

@cgsergey did you have a chance to look further into this?