RSP / data.lsst.cloud updates - 2021-12-03

News

Things have been pretty stable across our deployments and RSP teams have been working on Data Preview 0.2 functionality as well as infrastructure components such as monitoring and scaling. Since the last bulletin we have migrated the Notebook service to Jupyterlab 3.x and updated to JupyterHub 1.5.x. Project staff who often have to hop between RSP deployments and containers in particular should note that at the bottom of your Notebook page you now have a status line that tells you where you are and what you are running:

Intermittent delays in the RSP Notebook service

We have been tracking three separate stalls that occur intermittently during Notebook start. They are

  1. From the RSP home page, between clicking on Notebook and getting to the spawner page
  2. During the spawning (waiting for server to start)
  3. After the redirection to the JupyterLab server before the JL page renders (“white page”)

We have determined these all have different root causes.

The first one has been isolated and the fix was rolled out in yesterday’s Patch Thursday. We no longer expect it to occur so please raise an issue if you see it. [1]

The second one has been isolated and we are working on a solution.

The third one is going to be a toughie as it seems to be happening within JupyterLab itself, but we will be looking into it as we can. Meanwhile the best thing to do if you trip it is to wait it out, in particular don’t reload the page, much as it is a natural temptation.

Reminder if you are a Data Preview delegate and you have service issues you can file a ticket here: Issues · rubin-dp0/Support · GitHub or if you have a general question you can ask it on this forum.


  1. It was introduced when we followed an upstream recommendation and switched to a different third party library which turned out to have a different asynchronous behaviour than the one we replaced and we had to do some refactoring to bring back the correct multi-threading behaviour in our authentication service. ↩︎

1 Like