Pan-STARRS reference catalog in LSST format

refcat
panstarrs
Tags: #<Tag:0x00007ff62c660948> #<Tag:0x00007ff62c660808>

(Paul Price) #1

The Pan-STARRS 3pi stack data (images and catalogs) were released to the world as DR1 on 2016 December 19. This provides a high-quality network of calibration sources over three quarters of the sky to a modest depth. Telescopes both large and small (depths overlapping i ~ 17 to 21 mag) observing fields north of Dec=-30 deg in the optical will have good PS1 calibration sources in the field, observed through the same column as the target sources, allowing simple relative calibration for astrometry, photometry and spectroscopy.

To allow us (and others) to take advantage of this, I have produced a point source catalog in the LSST reference catalog format and also in the astrometry.net format we have used in the past. Details on the contents, construction and use of the catalog are available from the README.txt file included in the catalog, which I will summarise here.

Overview

This reference catalog contains astrometry and grizy photometry for 2,990,470,528 point sources at Dec > -30 deg to i ~ 22.5 mag.

Users of this catalog should include the following acknowledgement in publications:

The Pan-STARRS1 Surveys (PS1) have been made possible through contributions of the Institute for Astronomy, the University of Hawaii, the Pan-STARRS Project Office, the Max-Planck Society and its participating institutes, the Max Planck Institute for Astronomy, Heidelberg and the Max Planck Institute for Extraterrestrial Physics, Garching, The Johns Hopkins University, Durham University, the University of Edinburgh, Queen’s University Belfast, the Harvard-Smithsonian Center for Astrophysics, the Las Cumbres Observatory Global Telescope Network Incorporated, the National Central University of Taiwan, the Space Telescope Science Institute, the National Aeronautics and Space Administration under Grant No. NNX08AR22G issued through the Planetary Science Division of the NASA Science Mission Directorate, the National Science Foundation under Grant No. AST-1238877, the University of Maryland, and Eotvos Lorand University (ELTE), the Los Alamos National Laboratory, and the Gordon and Betty Moore Foundation.

Relevant papers for information and citation include:

LSST-style format

The LSST-style format uses FITS files divided by HTM. You can download the LSST-style catalog here. Be warned that it’s rather large: 423 GB. You can download a subset of the catalog by grabbing the appropriate FITS files; see the section in the README.txt entitled “Field selections” for details.

You need to install it in your data repo:

ln -s /path/to/ps1_pv3_3pi_20170110/ /path/to/DATA_REPO/ref_cats

Then you need to adjust your configuration. Here’s a configuration for processCcd.py (either to put in your obs package as config/processCcd.py, or to specify on the command-line with --configfile). See also the section below on “Common configuration” for colorterms and filterMap.

from lsst.meas.algorithms import LoadIndexedReferenceObjectsTask
config.calibrate.astromRefObjLoader.retarget(LoadIndexedReferenceObjectsTask)
config.calibrate.astromRefObjLoader.ref_dataset_name = "ps1_pv3_3pi_20170110"
config.calibrate.photoRefObjLoader.retarget(LoadIndexedReferenceObjectsTask)
config.calibrate.photoRefObjLoader.ref_dataset_name = "ps1_pv3_3pi_20170110"
config.calibrate.photoCal.photoCatName = "ps1_pv3_3pi_20170110"

Astrometry.net-style format

The astrometry.net-style format uses custom FITS files divided by HEALPix and indexed using tools provided by astrometry.net. You can download the astrometry.net-style catalog here. This format is a bit smaller than the LSST-style format (it does not include as much detail, e.g., proper motions), but is still quite hefty at 396 GB. If you want a subset of the catalog, you will need to figure out the appropriate HEALPix using the get-healpix tool from astrometry.net (I’m not entirely sure whether it’s the “XY”, “RING” or “NESTED” scheme you want) and grab the *_and_<healpix>_[012].fits files. Or you can grab the appropriate files from the LSST-style format and generate your own astrometry.net-style catalog using the included scripts; see the README.txt under “Astrometry.net format” for details.

The downloaded catalog doesn’t need to be installed in the data repo, but it should be declared to eups:

eups declare astrometry_net_data ps1_pv3_3pi_20170110-and -r /path/to/ps1_pv3_3pi_20170110-and

Then, to use it, you need to:

setup -j astrometry_net_data ps1_pv3_3pi_20170110-and

No configuration changes are currently required, because astrometry.net-style catalogs are currently the default (this will likely change soon), but do see the section below on “Common configuration” for colorterms and filterMap.

Common configuration

Regardless of which style of catalog you use, you need to set the colorterms and filterMap appropriately for your camera. Here is an example configuration for processCcd.py:

# These colorterms are for HSC, included as an example
colorterms = config.calibrate.photoCal.colorterms
from lsst.pipe.tasks.colorterms import ColortermDict, Colorterm
colorterms.data["ps1*"] = ColortermDict(data={
    'g': Colorterm(primary="g", secondary="r", c0=0.00730066, c1=0.06508481, c2=-0.01510570),
    'r': Colorterm(primary="r", secondary="i", c0=0.00279757, c1=0.02093734, c2=-0.01877566),
    'r2': Colorterm(primary="r", secondary="i", c0=0.00117690, c1=0.00003996, c2=-0.01667794),
    'i': Colorterm(primary="i", secondary="z", c0=0.00166891, c1=-0.13944659, c2=-0.03034094),
    'i2': Colorterm(primary="i", secondary="z", c0=0.00180361, c1=-0.18483562, c2=-0.02675511),
    'z': Colorterm(primary="z", secondary="y", c0=-0.00907517, c1=-0.28840221, c2=-0.00316369),
    'y': Colorterm(primary="y", secondary="z", c0=-0.00156858, c1=0.14747401, c2=0.02880125),
})
# For the HSC r2 and i2 filters, use the r and i values from the catalog
for refObjLoader in (config.calibrate.astromRefObjLoader,
                     config.calibrate.photoRefObjLoader,
                     config.charImage.refObjLoader,
                     ):
    refObjLoader.filterMap['r2'] = 'r'
    refObjLoader.filterMap['i2'] = 'i'

Reducing Suprime-cam Data
Reducing Suprime-cam Data
How to properly setup astrometry refererence catalog?
(Paul Price) #2

Rick White (STScI) sent around this summary of the PS1 archive usage since the data release:

I thought you would be interested to hear about the usage of the PS1 public archive at STScI. It has been 1 month since the PS1 data release on December 19. In that time:

15.4 TB of images have been downloaded to 15,500 different IP addresses.

3.2 million image cutouts (“postage stamps”) have been extracted.

2.5 million Virtual Observatory catalog queries have been done by 70 different IP addresses, returning information on 580 million objects.

644,000 database queries have been executed through our CasJobs interface by 99 different users, returning more than 7.6 billion rows of data. The total data volume downloaded from CasJobs is 4.4 TB.

16,100 catalog searches have been done using the MAST web search form.

For comparison, the image data volume in the current public release is about 100 TB, and the catalog includes approximately 10 billion objects in a 20 TB database.

Despite the heavy usage, I believe there have been no negative impacts on other STScI archive usage or on our internet access. Our IT infrastructure has shown a lot of resiliency in handling the load with little difficulty.

This was only possible through the efforts of the PS1 team, and the huge popularity of the data is a testament to the value of the survey!