Launching notebook (nublado) gives 500 Internal Server Error

Support Rubin Science Platform
#1

Howdy,

I had a working environment (tacc-spherex) earlier today. I wanted to change a few nublado configs, so I did so, and it was having trouble restarting. It seemed like the helm chart wasn’t happy without a few new options, e.g.

cronjob:
  enabled: false

I re-based my branch (u/mpackard/tacc-spherex-dev) from main thinking maybe I was too far behind.

After refreshing and restarting everything I now get a 500: Internal Server Error when trying to launch notebooks. One error I found in hub log:

[E 2025-07-31 21:50:54.986 JupyterHub log:192] 500 GET /nb/hub/spawn/mpackard (mpackard@192.168.115.244) 50.35ms                                           
[E 2025-07-31 21:56:18.412 JupyterHub _internals:50] Exception raised in REST spawner                                                                      
    Traceback (most recent call last):                                                                                                                     
      File "/usr/local/lib/python3.12/site-packages/rubin/nublado/spawner/_internals.py", line 44, in wrapper                                              
        return await f(spawner, *args, **kwargs)                                                                                                           
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                           
      File "/usr/local/lib/python3.12/site-packages/rubin/nublado/spawner/_internals.py", line 203, in options_form                                        
        r.raise_for_status()                                                                                                                               
      File "/usr/local/lib/python3.12/site-packages/httpx/_models.py", line 829, in raise_for_status                                                       
        raise HTTPStatusError(message, request=request, response=self)                                                                                     
    httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'https://tacc-spherex.lsst.cloud/nublado/spawner/v1/lab-form/mpackard'         
    For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

Some part of it thinks the notebook should live at /nb/hub/spawn/mpackard and some part thinks https://tacc-spherex.lsst.cloud/nublado/… ? Not sure if that is a red herring.

Thanks for any help you can provide.

Regards,
Mike

#2

Hi Mike,

It doesn’t look like the rebase of your branch is successful if one looks at the corresponding GitHub PR. It has tons of conflicts and duplicate commits, which may mean that instead of rebasing you accidentally merged main onto the branch. I suspect that your branch is now in an inconsistent state, which may be causing problems.

Maybe start with a git rebase and see if that helps? It will at least make it easier to see what you intended to change relative to current Phalanx main.

cronjob.enabled should be false by default, so you shouldn’t have to set that explicitly in your values file.

#3

Thank you. I think my branch is in better shape now, and ArgoCD has somewhat recovered. I will check with some more folks before submitting PR to main.

Regards
Mike