Running latest pipeline tutorial with "current w_2022_16 w_latest setup"

Data Management
#1

I upgraded my Mac OS to Monterey 12.3.
I scuttled the prior pipeline software.
I installed the newest stack.
I successfully installed the latest distribution with: eups distrib install -t w_latest lsst_distrib
I downloaded the latest RC2 SUBSET (even though there is a middle directory named /gen2/ )
So far, so good.
I got to Part 2 of the tutorial and received this output from the butler query
butler query-data-ids SMALL_HSC --collections HSC/RC2/defaults --datasets ‘raw’
WARNING: version mismatch between CFITSIO header (v4.000999999999999) and linked library (v4.01).
WARNING: version mismatch between CFITSIO header (v4.000999999999999) and linked library (v4.01).
WARNING: version mismatch between CFITSIO header (v4.000999999999999) and linked library (v4.01).
No results. Try --help for more information.

#2

As an add-on, these other butler queries work just fine in the tutorial, part 2:

  • butler query-collections SMALL_HSC
  • butler query-dataset-types SMALL_HSC
  • butler query-dimension-records SMALL_HSC/ instrument
    yet, both of the “butler query-data-ids” respond with “No results…”
#3

If I compose the equivalent query from Jupyter, it seems to WORK!!!
for ref in registry.queryDatasets(‘raw’, collections=‘HSC/RC2/defaults’, instrument=‘HSC’):
** print(ref.dataId.full)**

{band: ‘y’, instrument: ‘HSC’, detector: 41, physical_filter: ‘HSC-Y’, exposure: 322}
{band: ‘y’, instrument: ‘HSC’, detector: 41, physical_filter: ‘HSC-Y’, exposure: 346}
{band: ‘y’, instrument: ‘HSC’, detector: 41, physical_filter: ‘HSC-Y’, exposure: 358}
{band: ‘z’, instrument: ‘HSC’, detector: 41, physical_filter: ‘HSC-Z’, exposure: 1178}
{band: ‘z’, instrument: ‘HSC’, detector: 41, physical_filter: ‘HSC-Z’, exposure: 1184}

#4

Apologies. The tutorial needs to be updated to use butler query-datasets (as you use later on when you report it working). There was a bug in query-data-ids that we fixed recently. The bottom line is that if you want to know what datasets are available you should be using query-datasets – we will try to fix the tutorial soon.

#5

okay, thanks, great. I’ve refabricated the butler command as:
butler query-datasets SMALL_HSC --collections HSC/RC2/defaults raw
Onward…

#6

My fresh install included: eups distrib install -t w_latest lsst_distrib
This yielded : gb444b43468+bf97d9bd2a current w_2022_16 w_latest setup
With a CONDA env : LSST_CONDA_ENV_NAME=lsst-scipipe-3.0.0
Located at : lsst_stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-3.0.0

With this software stack, I receive these messages with most/many commands:
WARNING: version mismatch between CFITSIO header (v4.000999999999999) and linked library (v4.01).

Might there be an off the shelf solution to sync the lib with the cfitsio header? , so I can continue with 3.0.0?

or, should I just ignore the warnings and continue forward.

#7

The versions are synced. The warning message is a bug coming from, we think, the healpy package because in v4 cfitsio changed the way it defined floating point versions. You can ignore it.

#8

I’m working to graph/plot/visualize a region, using the region column value in the visit table. After I decode it using sphgeom.Region.decode(), I get a ConvexPolygon with a long list of UnitVector3d values:
ConvexPolygon([UnitVector3d(-0.8637107211893635, 0.5034290256812234, 0.023727751773054258), etc…
With these values is there a convenient means to portray a 3d visualization,perhaps in terms of RA and DEC.
Just looking for some guidance. Matplotlib or sphgeom.ConvexPolygon methods?

#9

At a low level, you can convert the UnitVector3d values from poly.getVertices() to RA (longitude) and dec (latitude) using sphgeom.LonLat.latitudeOf(v).asDegrees() and sphgeom.LonLat.longitudeOf(v).asDegrees() or by constructing sphgeom.LonLat(v) and then using getLongitude().asDegrees()and getLatitude().asDegrees().

See LSST Applications: lsst::sphgeom Namespace Reference for the C++ documentation that is generally straightforwardly translated into Python.

#10

ktl:
okay, once I convert lat and long to degrees, what method would I use to satisfy my original question…about graph/plot/visualize. Or, might you have an example?
many thanks