Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-version notebook support #232

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Conversation

MichaelHatherly
Copy link
Collaborator

@MichaelHatherly MichaelHatherly commented Jan 14, 2025

This adds support for running notebook processes that are different Julia versions than the main runner process. To start a different Julia version in a notebook use juliaup channel syntax in the julia.exeflags frontmatter for the notebook. You must already have that channel name installed:

---
engine: julia
julia:
    exeflags: ["+1.11"]
---

Previously this did not work since QNR used Malt.jl for managing them, and that package uses Serialization.jl for the communication between processes. Serialization is limited to only being about to deserialize data from the same version of Julia, hence it blocked supporting running multiple Julia versions in our Quarto notebooks.

This PR vendors Malt.jl and adjusts some code to allow for starting up and sending data across different versions of julia. Instead of Serialization we use BSON for serialization which manages to work across versions for the limited kinds of data that QNR uses, basically just strings, arrays of uint8, namedtuples and numbers.

In addition this also implements what was discussed in #215 to throw more informative errors when the Julia version running and the julia_version field in a Manifest.toml file do not match, which can, if ignored, lead to hard to diagnose errors on notebook startup. We now throw a hard error when there is a mismatch, and guide the user to a solution by either picking a different Juliaup channel, or re-resolving the manifest file.


Take 2 of #230.

Copy link

codecov bot commented Jan 14, 2025

Codecov Report

Attention: Patch coverage is 22.10928% with 613 lines in your changes missing coverage. Please review.

Project coverage is 31.05%. Comparing base (ad04a34) to head (a27d9f7).

Files with missing lines Patch % Lines
...rtoNotebookWorker/src/vendor/BSON/src/anonymous.jl 0.00% 135 Missing ⚠️
...toNotebookWorker/src/vendor/BSON/src/extensions.jl 0.00% 109 Missing ⚠️
src/Malt.jl 58.00% 97 Missing ⚠️
src/QuartoNotebookWorker/src/Malt.jl 0.00% 73 Missing ⚠️
...c/QuartoNotebookWorker/src/vendor/BSON/src/read.jl 0.00% 68 Missing ⚠️
.../QuartoNotebookWorker/src/vendor/BSON/src/write.jl 0.00% 59 Missing ⚠️
src/startup.jl 0.00% 20 Missing ⚠️
...rtoNotebookWorker/src/vendor/BSON/test/runtests.jl 0.00% 17 Missing ⚠️
...c/QuartoNotebookWorker/src/vendor/BSON/src/BSON.jl 0.00% 13 Missing ⚠️
src/QuartoNotebookWorker/src/render.jl 0.00% 8 Missing ⚠️
... and 4 more

❗ There is a different number of reports uploaded between BASE (ad04a34) and HEAD (a27d9f7). Click for more details.

HEAD has 5 uploads less than BASE
Flag BASE (ad04a34) HEAD (a27d9f7)
6 1
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #232      +/-   ##
==========================================
- Coverage   37.14%   31.05%   -6.10%     
==========================================
  Files          41       52      +11     
  Lines        1642     2389     +747     
==========================================
+ Hits          610      742     +132     
- Misses       1032     1647     +615     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MichaelHatherly MichaelHatherly force-pushed the mh/multi-version-2 branch 12 times, most recently from a66055f to 64f3f4b Compare January 15, 2025 22:04
@MichaelHatherly MichaelHatherly force-pushed the mh/multi-version-2 branch 2 times, most recently from 9bb4065 to e550ab1 Compare January 16, 2025 16:54
Add a barrier function in the `show` code path to make stacktrace
cleaning more robust. Previous logic produced inconsistent results
between macOS x64 and ARM due to differences in the frames included
in stacktraces, ie. there were missing `invokelatest` frames that
impacted the stacktrace cutoff point.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant