Getting TraceID from an OpenTracing DatadogSpanContext can throw "Cannot read properties of undefined (reading 'toString') #4987
Comments
|
We too are having trouble with this error sometimes after we upgraded from v5.24.0 to v5.25.0. import tracer from 'dd-trace'
const serviceName = process.env['DD_SERVICE']
if (serviceName) {
tracer.init({
logInjection: true,
service: serviceName,
})
}
...
export function logger() {
// Set req.traceId to http access log
return (req: Request, res: Response, next: NextFunction) => {
// dd-trace v5.25.0 sometimes raise error `Cannot read properties of undefined (reading 'toString')`
// in https://github.com/DataDog/dd-trace-js/blob/497c0b0d4333b6a7e7a434fb4de275595e5950c9/packages/dd-trace/src/opentracing/span_context.js#L39
req.traceId = tracer?.scope().active()?.context().toTraceId()
delegate(req, res, next)
}
}
const app = express()
app.use(logger())
... |
|
This set of libraries is honestly disappointingly unstable. I feel like every time we upgrade another random issue like this appears. We had it pinned since May 2024 to circumvent another bug, but yesterday I was forced to unpin to fix another problem, and now I wake up to several reports of this error reported here. You seriously need to invest in better integration tests. |
|
We're using I've downgraded the layer to 112, which for us is the LKGC. |
|
I've not been able to reproduce this on v5.25.0 with the code snippets provided (tried continuously sending requests to the express app over a 1000 times). Would it be possible for any of you to create a self contained application that successfully reproduces the problem so we can better investigate? |
|
We ran into this issue after upgrading from 5.21 -> 5.30, without any explicit userland code interacting with Downgraded to 5.24 after reading this thread and now the issue is resolved From what I can tell, the only way a Only obvious example I could find was from this commit It could also theoretically happen if the I assume there's a reasonable explanation to why this project isn't using static types (e.g. TS) for its source code, since it would largely eliminate this class of problems |
|
Some debug log snippets from prod: First error: This is immediately followed by attempting to encode a payload that is the first one to be missing a trace_id and then another error which disables the plugin ( Stack (at 2025-01-10):
|
I haven't been able to create a local repro yet. Will post here if I do. |
|
This is a critical issue for us as well. We have to disable ASM + tracing in our apps. @rochdev Any attempt from datadog to fix this. It is a severe issue for us using datadog. |
Calling this
DatadogSpanContext.toTraceId()method on what appears to be a valid span can sometimes error out with an undefined innerthis._traceIdvalue:dd-trace-js/packages/dd-trace/src/opentracing/span_context.js
Lines 33 to 40 in 497c0b0
Suggested Solution
While I am not sure what might cause this scenario, a quick fix would be to make the dot property access of
this._traceIda bit more defensive:Replication Steps
Using:
Initialize the tracer on startup:
Try to defensively get the active span's
traceIDat runtimeExpected result
traceIDis either a valid Trace ID from the current span orundefinedin which case we default to'n/a/based on the code aboveActual result
We regularly encounter an error whereby we seem to have a defined
span(i.e. notnullorundefined) with an empty or missingDatadogSpanContext._traceIDproperty such that when we calltoTraceId()on it we get the following error:Unit Tests
Here is a failing test we can add to
packages/dd-trace/test/opentracing/span_context.spec.jsto demonstrate the issueThe text was updated successfully, but these errors were encountered: