You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A very frequent pattern for the data scientists I manage is using jupyter notebooks for development and the productionalizing with Metaflow.
Right now the Jupyter Notebook -> Metaflow python conda environment transition is a friction point.
Ideally we'd like to solve the environment once, lock it with a conda lock file, and then use that lock file throughout the life of that version of the code. We also cache our python packages with an artifactory so this adds a fairly nice confidence of reproducibility.
@romain-intel I put some work in trying to figure out how to get the conda lock file input working a while ago but got pulled into other projects. I seem to remember you saying it wouldn't be a big lift. Any chance you could consider implementing this feature?
In the short term I don't anticipate having the bandwidth to get the necessary context in this project to add the functionality myself.
The text was updated successfully, but these errors were encountered:
I think it would be easily possible to do in some cases. If you always go through conda to create your environment, that would work well (ie: conda would know about everything). If, however, you used conda and then pip, it may be harder to determine. It actually came up a few days ago internally as well. It sounds like your use case would fall more in the first category (ie: conda knows everything). I can't promise any timeline but let me see when I can get this in. You are right that it would be a very cool feature.
A very frequent pattern for the data scientists I manage is using jupyter notebooks for development and the productionalizing with Metaflow.
Right now the Jupyter Notebook -> Metaflow python conda environment transition is a friction point.
Ideally we'd like to solve the environment once, lock it with a conda lock file, and then use that lock file throughout the life of that version of the code. We also cache our python packages with an artifactory so this adds a fairly nice confidence of reproducibility.
@romain-intel I put some work in trying to figure out how to get the conda lock file input working a while ago but got pulled into other projects. I seem to remember you saying it wouldn't be a big lift. Any chance you could consider implementing this feature?
In the short term I don't anticipate having the bandwidth to get the necessary context in this project to add the functionality myself.
The text was updated successfully, but these errors were encountered: