HathiTrust DEV-1350: /var/log/babel for web apps #719
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, the web applications log to NFS, which is suboptimal for many reasons -- processes have to write to individual files rather than being able to share a log file, and logs can't easily be collected from there and sent to retention or query systems (filebeat, loki). This: * creates /var/log/babel and makes it writable by the web server * adds a logrotate config to ensure logs are rotated (similar to apache logs) * adds an alloy config to send these logs to loki
|
The filebeat config does not need any changes; |
rrotter
approved these changes
Oct 4, 2024
|
The logrotate config appears to be working. I will go ahead and merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the HathiTrust web applications log to NFS, which is suboptimal for many reasons -- processes have to write to individual files rather than being able to share a log file, and logs can't easily be collected from there and sent to retention or query systems (filebeat, loki).
This:
I have been testing this configuration on dev-1.babel.hathitrust.org and the alloy config appears to work; the logrotate config may still have some errors (will check next Monday.)
Once the logrotate config is proven to work as expected, this should be safe to merge at any time; once it is merged, we can deploy the corresponding change for the web applications.