Creating custom instance catalogs?

ljones · September 25, 2015, 11:54pm

Are there any examples of creating custom instance catalogs with mixins?
I think this is what I need to do, anyway … I need to
(a) query the moving object database for the positions and velocities and vMag of the objects (I think this is all the moving object db contains as far as instantaneous values … maybe also phase/solar elongation).
(b) translate the vMag to the magnitude in the observed bandpass [this could be done using something like the stars photometry, but the moving object colors are even simpler, so probably will write something super short] – this is a SsmPhotometry mixin yes?
© translate the mag in bandpass + m5 of the observation to SNR and also to a probabilistic “detected” (yes/no) flag (another method on the photometry mixin?)
(d) incorporate the focal plane footprint to see if the object actually landed on active silicon (does this kind of mixin already exist?)
(e) translate velocity into length/position angle of the trailing vector (some other mixin - SsmAstrometry?)
(f) calculate an uncertainty on the object’s reported position, and output ra/dec values that include uncertainty (is this kind of thing already in some other astrometry mixin?)

Do I have the right idea here - that I’d use the base instance catalog, write a subclass for moving objects, then write moving object mixins as above, and then combine then into my custom instance catalog object?
If not, what should I be thinking of instead? And if so, can you point me to some (or ‘the best’) examples?

Do any of the tasks above already exist in mixins that I haven’t found yet?

danielsf · September 26, 2015, 12:08am

Yes. You will want to write mixins to accomplish this. To date, we do not yet have any mixins written for the SSM catalogs.

Probably the best examples for you to look at are in

sims_catUtils/python/lsst/sims/catUtils/mixins/AstrometryMixin.py

You can look at the classes AstrometryStars and CameraCoords.

For a generic example of how InstanceCatalogs mixins are meant to work, you can look at this notebook

github.com

lsst/sims_catUtils/blob/master/examples/tutorials/tutorial02.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will introduce how we use mixins to compartmentalize <b>InstanceCatalog</b> functionality.\n",
    "\n",
    "Not every <b>InstanceCatalog</b> will have the same outputs.  Even those that do have the same outputs may require different getters (one example that comes to mind is doing photometry on stars and galaxies; for galaxies, one must consider flux from the bulge, the disk, and the agn, all of which are stored separately in the database; for stars there is only one source of flux to consider). Therefore, we try as much as possible to define commonly used getters in mixins.  Mixins are classes that define methods explicitly for the purpose of being inherited by other classes (though, occasionally you will find a mixin with classmethods that are meant to be called independently).  Thus mixins should not have an `__init__()`.  They will rely on <b>InstanceCatalog</b>'s `__init__()`.  When writing an <b>InstanceCatalog</b> daughter class, the user can simple inherit from the required mixins and ignore all other mixins.\n",
    "\n",
    "Below, we show an example of a mixin and how it interacts with an example <b>InstanceCatalog</b>.\n",
    "The class `TutorialCatalog` calls for the columns `raPlusOneRadian`, `sum`, and `difference`.\n",
    "These columns do not exist in the database and `TutorialCatalog` does not contain getters defining\n",
    "them.  The class `ExampleMixin` does contain getters defining these columns.  By making\n",
    "`TutorialCatalog` inherit from `ExampleMixin`, we pass these getters on to `TutorialCatalog` and\n",
    "thus allow it to compute the desired columns.\n",
    "\n",
    "Two other bits of functionality are introduced below:\n",
    "\n",
    "1) Transformations: Sometimes you will want to define a unit transformation so the data in your written catalog is in different units than the data being manipulated by the code.  The most obvious example is that CatSim policy is to store all angles in radians while they are being manipulated.  However, RA and Dec are stored in degrees in most databases.  Therefore, RA and Dec are converted into radians when they are passed into the code (that will be discussed in tutorial04), manipulated as radians, and then converted back to degrees before being written out to a catalog.\n",

This file has been truncated. show original

or, my personal favorite (though this covers everything about CatSim)

github.com

uwssg/LSST-Tutorials/blob/master/CatSim/CatSimTutorial_SimulationsAHM_1503.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, some philosophy:\n",
    "\n",
    "The CatSim stack principally exists to create catalogs from simulated universes.  The default simulated universe that CatSim accesses lives on a machine at the University of Washington called \"fatboy\".  This simulated universe is really a <b>database</b> that consists of a distribution of galaxies drawn from the Millennium N-body simulation, and Milky Way stars generated with the GalFast software.  The most up-to-date documentation of what fatboy provides is here\n",
    "\n",
    "https://confluence.lsstcorp.org/display/SIM/Database+Contents+--+Catalog+Simulations\n",
    "\n",
    "https://confluence.lsstcorp.org/display/SIM/Database+Schema\n",
    "\n",
    "Whenever this notebook refers to \"querying the database,\" it is referring to this database on fatboy (it is possible, in principle, to connect CatSim to your own database containing your own simulated universe; that is beyond the scope of this demo).\n",
    "\n",
    "<b>See the link below for instructions on how to access the fatboy database</b>\n",
    "\n",
    "https://confluence.lsstcorp.org/display/SIM/Accessing+the+UW+CATSIM+Database\n",
    "\n",

This file has been truncated. show original

ljones · September 26, 2015, 12:17am

Great, thanks Scott.
Looking through the available mixins, it looks like I will have to write one for ssm photometry and add one for astrometric uncertainty and trailing vector.
On the other hand, it looks like I could use the CameraCoords mixin, if I make sure that the appropriate columns already exist in the right format in my instance catalog at that point, in order to add the focal plane footprint.
Okay!