Run pipeline task on FITS images

Hi,
I would like to run an LSST pipeline task on an external .fits image with a code like:

from astropy.io import fits
from lsst.pipe.tasks.characterizeImage import CharacterizeImageTask

with fits.open('myImage.fits') as hdu:
  header = hdu[0].header
  image = hdu[0].data

config = CharacterizeImageTask.ConfigClass()
config.psfIterations = 1
charImageTask = CharacterizeImageTask(config=config)
result = charImageTask.run(image)

but it raises an error. From what I understand the problem is that the run method accepts only ‘lsst.afw.image.ExposureF’ objects. I try opening the fits file as an ExposureF object but I have some problems doing that. Can you please help me with this? What is the best way to open a fits file so that can be processed by the LSST pipeline?

Hi Salvatore,

In order to process non-LSST images with the LSST Science Pipelines, they must be ingested into a Butler repository. I haven’t tried to do this myself, but others in this forum will be able to help.

In the meantime, here are some additional resources.

I think that there might be more to it than just ingesting the fits files into a Butler repo, too; for example, CharacterizeImageTask might expect images with specific header information.

And searching the forum will yield a few more related topics about processing external images with the LSST Science Pipelines.

You don’t report the error but something like CharacterizeImageTask expects certain components to exist in the fits file such as variance and mask planes. If your FITS file is a single image plane then you likely will run into problems. This will be even more constrained when WCS and PSF information has to be available in the file.

You can not use astropy.io.fits to read the FITS file.

I also found Merian Data Processing Using the LSST Science Pipelines - HackMD a useful how-to.

1 Like

Thank you very much for these resources. They have been very useful for understanding more about Butler.

Nevertheless, I still have some problems ingesting non-LSST images into a Butler repository. I wasn’t able to create a metadata translator for the instrument (I was using some VST OmegaCAM and FORS2 images) but I will keep studying to understand how it works (I have little experience with Python so I need a bit more time).

Beyond that, I think I found another way of running pipeline tasks on FITS images even though it is probably far from being the best solution. Despite this, it seems to work at least for what I had in mind, so I share it here, probably it could be helpful for someone else too.

Details here

Essentially, what I have done is using the readFits method of the ExposureF class:

import lsst.afw.image as afwImage
image = afwImage.exposure.ExposureF.readFits('myImage1.fits')

where 'myImage1.fits' is the relative path of my FITS image. It complains about the mask which I think is not fatal:

lsst.afw.image.MaskedImageFitsReader WARNING: Expected extension type not found: IMAGE

but for the rest, it seems to work. I was able to do a cut of the image:

import lsst.geom as afwGeom
x_target, y_target = 1000, 500
width,height=600,500
xmin,ymin = x_target-width//2, y_target-height//2
bbox = afwGeom.Box2I()
bbox.include(afwGeom.Point2I(xmin, ymin))
bbox.include(afwGeom.Point2I(xmin+width, ymin+height))
cutout = image.Factory(image, bbox, origin=afwImage.LOCAL, deep=False)

Additionally, I was able to run some pipeline tasks such as:

from lsst.pipe.tasks.characterizeImage import CharacterizeImageTask
config = CharacterizeImageTask.ConfigClass()
config.psfIterations = 1
charImageTask = CharacterizeImageTask(config=config)
result = charImageTask.run(cutout)

and finally, I get the sources identified and overplotted on the original image:

See this other discussion: Minimum fits header data required for butler ingest-raws - #12 by timj

The steps to allow ingest of other instrument data are described in that post: creating a metadata translator, creating an Instrument class that butler can use to understand how to read the file (effectively what we call an “obs” package).