User Guide for creating our own Butler repository and all associated steps

In the spirit of Sherlock’s post about ‘TAGGED’ class, I am curious to know if there is a document that explains the chronology from the initial creation of a Butler repository, through the ingesting of raw frames, darks, flats, biases, etc.,…all the steps that a scientist might execute to then perform the steps [like those] shown in the tutorial, like single_frame, isr, characterization, coadds, etc…
So, is there a document/user guide on such steps? And, I’m not referring to “docstrings”.

I made some notes about my process in doing this on the Spin cluster at NERSC here:

Log In - SLAC Confluence

This is on the DESC confluence, so it will only be accessible to people who can log into it.

1 Like

Thank you Rob. I requested login…see below.
Fred Klich, Dallas

SLAC Jira/Confluence user signup

An e-mail has been sent to the administator(s) of group LSST Dark Energy Science Collaboration. You should receive an e-mail when your account has been approved.

The “Getting Started” documentation is helpful: Getting started with the LSST Science Pipelines — LSST Science Pipelines
As well as the documentation for obs_decam: The Gen3 workflow — LSST Science Pipelines

I had some trouble with that tutorial. I’m not sure if it is me or if the tutorial needs to be updated?

Thanks Christina. So far, I’ve been able to create my own repository, define the HSC instrument and ingest the curated calibrations for the HSC. I hope next to ingest the raws (I’ll use the raws from the current Gen3 file tree just to keep me honest). I’ll hope to transition to doing the rest of the normal pipeline steps after that…
I’m hoping to see the docs that Rob Knop referenced, however, I still have not received any status on my request for a login to the SLAC Confluence arena. Do you have any view to the doc Rob is mentioning?
Fred, Dallas

Rob, Would love to see the Confluence doc(s) you’re referring to…Are you able to help with my user account…or, even just send to my email account?
Fred, Dallas

That confluence is for DESC, which will only be available to DESC members.

Here is a resource I leaned on heavily when making it :

Many thanks Rob. This will take me parsecs down the road to setting up my own pipeline.
Fred, Dallas