New advanced command-line tutorial: custom coadd

Support Data Preview 0
, , ,
#1

We have now added a new command-line tutorial to the DP0.2 website:

02. Custom Coadds from the Command Line (advanced)

This new command-line tutorial shows one way of using Rubin/LSST command-line tools (particularly pipetask) within the RSP terminal to build the same custom coadd described in tutorial notebook 9a.

Please let me know of any suggestions/questions/feedback. Thanks!

#2

In case it’s of interest, I’ve also recast this command line tutorial’s custom coadd processing as a single shell script that can be executed from the RSP terminal:

# make sure that you're in the directory from which you wish to run/launch your
# custom coadd processing

# set up the LSST pipelines environment
setup lsst_distrib

# make the custom coadd QuantumGraph visualization
pipetask build \
-p $DRP_PIPE_DIR/pipelines/LSSTCam-imSim/DRP-test-med-1.yaml#makeWarp,assembleCoadd \
--pipeline-dot pipeline.dot; \
dot pipeline.dot -Tpdf > makeWarpAssembleCoadd.pdf

# remove temporary file
rm pipeline.dot 

# specify the directory for output log files
LOGDIR=logs

# make the directory for output log files
mkdir $LOGDIR

# run the custom coadd processing
LOGFILE=$LOGDIR/makeWarpAssembleCoadd-logfile.log; \
date | tee $LOGFILE; \
pipetask --long-log --log-file $LOGFILE run --register-dataset-types \
-b dp02 \
-i 2.2i/runs/DP0.2 \
-o u/$USER/custom_coadd_window1_cl00 \
-p $DRP_PIPE_DIR/pipelines/LSSTCam-imSim/DRP-test-med-1.yaml#makeWarp,assembleCoadd \
-c makeWarp:doApplyFinalizedPsf=False \
-c makeWarp:connections.visitSummary="visitSummary" \
-d "tract = 4431 AND patch = 17 AND visit in (919515,924057,924085,924086,929477,930353) AND skymap = 'DC2'"; \
date | tee -a $LOGFILE
#3

An updated version of this command line tutorial which avoids POSIX-style interaction with the pipeline YAML definition is now available at the same DP0.2 website URL:

02. Custom Coadds from the Command Line (advanced)

#4

In the GitHub repository for DP0.2 delegate contributions, I have now posted “solutions” to the command line optional exercises from this DP0.2 command line tutorial:

#5

Just a quick note to say that we’ve made substantive updates to DP0.2 tutorial notebook 09b, so that it now uses PipelineTasks. We also updated DP0.2 command line tutorial 02 so that it now includes instructions to run source detection on the custom coadd from the command line using pipetask. Thanks…

#6

¡Hi! I wonder if there is information regarding when it would be possible to make custom coadd with the current/latest pipeline version instead of using w_2022_40? will that pipeline version be usable only with DP1 or will be usable with DP0?

thanks

#7

Hi @alxogm, thanks for this question.

Due to great evolution and progress in Rubin’s data pipelines and services, the DP0.2-era tutorial notebook on custom coadds (09a/b) is no longer a useful learning tool (version Weekly 2022_40 will not work with DP1). We’re going to deprecate it (remove it from the set of DP0.2 notebooks). Better, up-to-date examples of image (re)-processing will be released as part of the DP1 tutorial set later this year.

#8

Thanks Melissa! I think part of my question was the other way around, would the new pipeline for re-processing images will work with DP0? If that is the case, is there a way to start using a preliminar version, even if it is under development? I ask about it, because there is some work I’m doing that would benefit to move on the new version if possible.

#9

Thank you for clarifying your question. No, the newest versions of the pipelines will not work for creating custom coadds out of the DP0 processed visit images, which are from older versions of the pipelines. If it was possible we would have updated the DP0 custom coadd tutorial notebooks.

1 Like