Permission changes for users’ “notebooks/tutorial-notebooks” directory

This message is for all @DP0-Delegates.

Action is recommended, depending on your case. See below.

Overview: As of Thu Sep 29 2022, the recommended image for the Rubin Science Platform (RSP; data.lsst.cloud) has been updated to “Weekly 40”. Your “notebooks/tutorial-notebooks” directory may now be read-only. The information provided in this post can also be found in the DP0.2 documentation page “Introduction to the RSP Notebook Aspect”.

All delegates will find their situation under either Case A or Case B. The following details assume that the recommended image (Weekly 40 or later) is selected at the time of login to the Notebook Aspect of the RSP.


Case A: For all existing RSP accounts set up before today (i.e., most delegates), the “notebooks/tutorial-notebooks” directory will remain writable, but a warning file (00_WARNING_README.md) will appear if any other file in that directory has been changed. This warning alerts delegates that they might not have the most up-to-date versions of the tutorials, and should take action.

The recommended action for all existing accounts is to do Step 1a or 1b, and then Step 2, in order to convert to a read-only version of the “notebooks/tutorial-notebooks” directory.

Step 1a: If you want to keep the current version of your “notebooks/tutorial-notebooks” directory, move it to another path in your home directory.

How to move tutorial-notebooks to another path.

In the Notebook Aspect of the RSP, use the Launcher (+ button) and open a Terminal, and execute the following statements. They will make a new sub-directory in your home directory called “my-work”, and then move your current “tutorial-notebooks” directory into it and rename it. It is crucial that you move and not copy the “tutorial-notebooks” directory, because it must be entirely gone from the “notebooks/” directory for Step 2 to work.

cd ~
mkdir my-work
cd ~/notebooks
mv tutorial-notebooks ~/my-work/my-tutorial-notebooks
cd ~/my-work/my-tutorial-notebooks
ls

You will see the contents of your version of the “tutorial-notebooks” listed.

Step 1b: If you don’t care to keep the current version, delete the entire “tutorial-notebooks” directory.

How to delete the tutorial-notebooks directory.

In the Notebook Aspect of the RSP, use the Launcher (+ button) and open a Terminal, and execute the following statements.

cd ~/notebooks
rm -rf tutorial-notebooks

Step 2: Exit the Notebook Aspect (“File” → “Save All and Exit”), log back in using the recommended image, and you will have a new read-only version of the “notebooks/tutorial-notebooks” directory. Case B (below) now applies to you too, even though your account is not new – please read the details for Case B.


Case B: For all new RSP accounts created today or hereafter, the “notebooks/tutorial-notebooks” directory will be read-only. It will always have the most up-to-date versions of the tutorials (i.e., the “prod” branch of the “tutorial-notebooks” repository). Notebooks can be executed and edited in this directory, but these changes are not saved automatically and cannot be saved to this directory by users.

If you want to make and save changes to these tutorial notebooks, the recommended action is to copy or move them to a different path in your home directory (see Case A, Step 1a above for instructions), but remember that files copied out of the “notebooks/tutorial-notebooks” directory will not be updated when Rubin Observatory staff update and release future versions of the tutorial notebooks that are compatible with future versions of the RSP’s recommended image.


How do I know if a directory is read-only?

In the Notebook Aspect of the RSP, use the Launcher (+ button) and open a Terminal, and execute the following statements.

cd ~/notebooks
ls -lah

If, on the output line for “tutorial-notebooks”, you see “dr-xr-xr-x”, this directory is read-only. If you see “drwxr-xr-x”, this directory is writable by the user.

What happens if I don't do the recommended action?

Taking no action and leaving the “notebooks/tutorial-notebooks” directory with writable permissions is an option, and support will be provided for delegates who prefer this option. Should you encounter issues with the tutorials not updating as new and improved versions are released, see these troublshooting tips or submit GitHub Issues for help.

Questions about this? Please post them as replies in this thread.

1 Like

I’m trying to copy a file from my home directory to my scratch directory, but I’m getting “Permission denied.” Also, I can’t “chmod 777” in the /scratch/bobabel directory. @DouglasLTucker encountered the same issue. Here’s a sample:
[bobabel2023@nb-bobabel2023 bobabel]$ pwd
/scratch/bobabel
[bobabel2023@nb-bobabel2023 bobabel]$ cp ~/WORK/Regression/StellarClassification/10000StarsFits3.ipynb .
cp: cannot create regular file ‘./10000StarsFits3.ipynb’: Permission denied
[bobabel2023@nb-bobabel2023 bobabel]$ chmod 777 10000StarsFits2.ipynb
chmod: changing permissions of ‘10000StarsFits2.ipynb’: Operation not permitted

Hey, Bob!

I am having a similar situation, in which I get a “Permission Denied” when I try to copy anything to my scratch area (/scratch/douglasleetucker) on data.lsst.cloud.

That said, I did find I could create a new directory on /scratch and copy stuff to the new scratch directory (in this case /scratch/DouglasLTucker).

E.g.:

douglasleetucker@nb-douglasleetucker scratch]$ pwd
/scratch
[douglasleetucker@nb-douglasleetucker scratch]$ mkdir DouglasLTucker
[douglasleetucker@nb-douglasleetucker scratch]$ ls
ajax6255  athornton     christinaadair    ebusa            jcarlin     kaehler   lionandjelka   mpwiesner  old-home       sustamujic                         t-ferreira
alxogm    bjwhite-fnal  dicrisci          gschwend         jeffcarlin  kunalfzu  maxdallora     mrmonroy   salnikov       sylvielsstfr                       tloredo
ameisner  bobabel       douglasleetucker  hcferguson       jvazquez77  leannep   melissagraham  ncaplar    shsuyu         TAP_verify_DP0.1-object_cat.ipynb  trianaa
annis     caganze       DouglasLTucker    jacob-hjortlund  kadrlica    ledwar04  morsony        nsevilla   simonkrughoff  TAP_verify_DP01.object.csv
[douglasleetucker@nb-douglasleetucker scratch]$ cd
[douglasleetucker@nb-douglasleetucker ~]$ cp 10000S* /scratch/DouglasLTucker/
[douglasleetucker@nb-douglasleetucker ~]$

I wonder if something happened with the old /scratch directory ownerships/permissions when we changed to the new RSP login system?

Hey, Bob!

I’ve alerted folks in the DM group and the suggestion is to create new scratch directories.
It sounds like the DM folks might be moving the old /scratch/ directories out of the way; so you might want to copy the contents of your original /scratch/ directory elsewhere, if you want to ensure you save the contents, just in case.

Thank you all for resolving this so fast!
bob