LogRecord pipeline addition #5124
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
noahfalk
approved these changes
Jan 4, 2025
|
(Let me know if you need me to mash the merge button) |
wiktork
reviewed
Jan 11, 2025
| @@ -13,12 +15,21 @@ public class LogObject : IReadOnlyList<KeyValuePair<string, object>>, IStateWith | |||
| public static readonly Func<object, Exception, string> Callback = (state, exception) => ((LogObject)state).ToString(); | |||
|
|
|||
| private readonly string _formattedMessage; | |||
| private List<KeyValuePair<string, object>> _items = new(); | |||
| private KeyValuePair<string, object>[] _items = new KeyValuePair<string, object>[8]; | |||
There was a problem hiding this comment.
Would CollectionsMarshal.AsSpan work here?
wiktork
reviewed
Jan 11, 2025
| if (_logger != null) | ||
| { | ||
| eventSource.Dynamic.AddCallbackForProviderEvent(LoggingSourceConfiguration.MicrosoftExtensionsLoggingProviderName, "MessageJson", (traceEvent) => { | ||
| if (!activityIdToScope.TryGetValue(traceEvent.ActivityID, out LogScopeItem scopeItem)) | ||
| { |
There was a problem hiding this comment.
I'm assuming you kept the two code paths separate because ultimately we will remove the _logger == null path. However, I'm assuming there's a lot of overlap between our reading of the log events and the new code.
wiktork
reviewed
Jan 11, 2025
|
|
||
| namespace Microsoft.Diagnostics.Monitoring.EventPipe | ||
| { | ||
| internal readonly struct LogRecordException : IEquatable<LogRecordException> |
There was a problem hiding this comment.
I think a record struct might give you these semantics automatically.
wiktork
reviewed
Jan 11, 2025
| { | ||
| private static readonly UTF8Encoding s_Utf8Encoding = new(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true); | ||
| [ThreadStatic] | ||
| private static LogObject[]? s_ScopeStorage; |
There was a problem hiding this comment.
Does this state get cleaned up after each session?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The existing
EventLogsPipelineroutes log messages retrieved usingMicrosoft-Extensions-Loggingthrough anILoggershim. There isn't anything necessarily wrong with this, but it adds a lot of extra work and memory pressure to the monitoring host. There also really isn't a way to add data such as Timestamp, TraceId, SpanId, etc., without having to hack ILogger.Working through a prototype of a
dotnet-monitorwhich exports OpenTelemetry format (OTLP) we (@samsp-msft @wiktork and @rajkumar-rangaraj) felt it was better to treat logging as we do the other pipelines and have a simple DTO to store all the log information (independent of ILogger) consumers can use to process logs.What this PR does is add a new interface into
EventLogsPipeline(ILogRecordLogger) which can be used to opt-into this new pipeline behavior.Once consumers have been updated we should be able to remove the old one. But they exist side-by-side for now so nothing currently reliant on the internals breaks.