How to work offline with Jupyter notebooks on Windows

Support Data Preview 0
, ,
#1

Dear all, I was asking how to work on Jupyter notebooks in the platform, on Windows, if we are offline.
I have found a specific procedure online on how installing the LSST Science Pipelines here:

https://pipelines.lsst.io/getting-started/data-setup.html

and here:

https://pipelines.lsst.io/install/newinstall.html#newinstall-unset-variables

Is it convenient to do it?
I do work on Windows, should I download Cygwin or something in order to create a Linux environment?
Thanks to everyone,
Best,
Silvia

#2

@silvia.pietroni

Rather than cygwin (not sure that would work?) you might like to look at WSL2 (Windows subsystem for Linux).

Using that I installed an ubuntu environment and then installed the pipeline stack. It was all pretty quick. I did it more just to see if it worked though and other than confirming the butler works I’ve haven’t really tested it.

I’m not sure about using that and jupyter notebooks though.

Mike

image
image979Ă—860 71.6 KB

#3

Hi @silvia.pietroni ! I just wanted to check in and see whether the WSL2 suggestion solved your issue or not, in case we should look for a solution? Thanks!

#4

Dear @ChristinaWilliams , I still did not try because @MRead wrote he wasn’t sure cause he did not try it.
I installed Cygwin but I still have to run it.
What do you suggest?

#5

I haven’t weighed in because I have no experience running RSP on Windows. But since there’s been limited input so far, I’ll just add this: I regularly teach a compute-heavy Bayesian data analysis course to both advanced undergrads and grads. It heavily uses Git/GitHub and Jupyter notebooks (all assignments are Jupyter notebooks submitted via a class GitHub org). The OSes on the students’ computers are a mix of Mac, Windows, and, for just a few, Linux.

For the students using Windows, I have tested WSL 2. I was able to install it easily, and have had no problem installing multiple conda environments supporting Jupyter, JupyterLab, and the Spyder IDE. The main reason I explored it was that some assignments have used PyStan (which requires a C++ toolchain), and the Stan team stopped supporting Windows as of PyStan v3. Conda and Python+Jupyter work fine on Windows without WSL 2, but for PyStan 3, WSL 2 was essential, and it worked without much installation trouble. That said, so that I wouldn’t have to require students to use WSL 2, we stuck with PyStan v2, which works under Windows (without WSL 2), provided you use the right magical ingredients in your conda env. :slightly_smiling_face: (No such magic was needed for Python + Jupyter without PyStan.)

Some of my class students were familiar with WSL 2, having used it for other courses or their research. A current grad student I’m working with (on an exoplanet problem) is using WSL 2 on his Windows laptop.

Note that WSL 2 is a Microsoft product, built to allow access to the Windows filesystem from the Linux OS. So a user has access to their regular Windows folders and files from within the Linux environment.

In earlier years, some students would use cygwin for PyStan, and it was a major headache. I haven’t kept up with it, and I presume it’s more mature now. I just wanted to report that Microsoft’s work on WSL 2 offers a pretty smooth ride for Python + Jupyter, including supporting a complex C++ toolchain (for PyStan 3). Whether it’s equally smooth with the full RSP system is not something I can speak to directly.

—Tom

PS: I use macOS and Ubuntu Linux myself. I was able to test the course software under Windows (including WSL 2) by using a Windows virtual machine on macOS with VMware Fusion.

2 Likes
#6

Hi @silvia.pietroni,
Just trying to get a better understanding of your question. So are you trying to run the notebooks from the Rubin Science Platform on your own computer so that you can work on them offline?

I’m not sure if this is possible if you are trying to use the notebooks to access Rubin data offline.

Best,
Ryan

1 Like
#7

Dear @ryanlau , yes, I’d like to know if it’s possible and worthy to install something on my laptop in order to work offline (sometimes it happens that network is down or while we are travelling, etc).
But if it’s not possibile to access Rubin data offline, as you noticed, maybe it is a useless procedure to do.
Thanks
Silvia

#8

Dear all, I discussed the topic together with @DouglasLTucker during the last Stack Club and he showed me how to use the LSST data offline with TOPCAT (thanks!).
Here is the link with the instructions: Introduction to the RSP API Aspect — Vera C. Rubin Observatory Documentation for Data Preview 0.2

3 Likes
#9

Just to be clear, using TOPCAT is still “online” in the sense that a network connection to the Interim Data Facility (IDF) is required to access the TAP service (and a current authentication token for that facility is also needed). It is only “offline” in the sense that no notebook or command line server at the IDF is being used.

If you use TOPCAT to query and download data to your local machine, that data can then be used truly “offline”, disconnected from the Internet.

1 Like
#10

yes @ktl , thanks.