I'm trying to process PhoSim LSST raw images with the pipeline and obtain calibrated exposures (calExps). I intend to do some further processing with calExps and experiment with some algorithms. I could also use HSC or DECam images (if I could obtain them, haven't investigated yet), but LSST images are preferred.
So I'm first trying to produce some calExps but I have several problems. I downloaded flatr, bias and dark fits files from the simulation done at Purdue (https://bitbucket.org/phosim/phosim_release/wiki/Example%20Images). I ingested the flatr images into a local registry using ingestImages.py task. The next step, I believe, is to ingest bias and dark FITS files using ingestCalibs.py. But there I have some problems.
ingestCalibs.py /data/folder /bias/folder/*.fits.gz --calibType bias --validity 1000 --create
It seems that I'm missing "translations" from HDU field names to registry column names. How and where do I specify those?
I'm using the LsstSimMapper for ingesting biases. Should I be using a different one?
But even if this is solved, I believe I will have a problem down the road anyway because the dataset from Purdue is missing flats for some reason. Can you please tell me how to obtain the flats?
By the way, what do constructBias.py, constructDark.py and others do exactly? I wasn't able to find any real documentation.
Regarding the reference database, I downloaded
ps1_pv3_3pi_20170110 fits files for the relevant area (using the instructions here: https://github.com/lsst/ci_hsc/tree/master/ps1_pv3_3pi_20170110). But again, I think I'll have problems down the road because I think I will need the Astrometry.net format, and
buildAndCatalog.py command fails by reporting that some intermediary files contain no data.
I'm a bit stuck here so I wonder if I could just avoid all of this and obtain calExps directly. Are there any simulated LSST calExp images available for download anywhere?
Are there any other publicly available simulated LSST images? (Purdue dataset contains just one snap, I believe)
Is the stack ready to process LSST simulated images at all or would you recommend I switch to HSC or DECam?
Is there any detailed documentation available for processing simulated LSST images?
Lot's of questions, but any help is appreciated.