Hi all! I am currently using Rubin’s API with ObsTAP to access DP0.2 Images on my local system, which is very convenient for retrieving & using the images. My team was wondering if we will still be able to do this once the images are no longer simulated.
ObsTAP is going to continue to be supported when LSST data start arriving so you will be able to use ObsTAP or the Butler to access data. The current ObsTAP deployment is a demonstration of intent, not something specific for DP0.2. We are committed to supporting the IVOA standards into survey operations.
2 Likes
Sorry there is some confusion about what @sierrajanson actually was asking. Tim’s answer is correct if the question was about the protocol/API for serving images.
If the question is whether Rubin intends to serve the Data Preview 0.2 images after actual Rubin data exists (
) the answer is that the plan was to take down the DP0.2 data once actual Rubin data is available, since the purpose of the Data Preview program was to collectively prepare us for on-sky data.
(Cc @gpdf as the actual source of truth here)
@frossie I presume that’s the same for DP0.3 Is there a time frame for that? start of operations? data preview 1? data release 1? It would be helpful to know if this is sometime in 2025 or later?
Thank you for the responses, sorry for the confusion! I was indeed asking about the protocol/API for serving images.
Absolutely; we will support ObsTAP throughout operations, using the same model in which the access_url for each image points to an IVOA DataLink “links endpoint”. The data from the links endpoint will include not only a URL for retrieving the image itself, but also the cutout service for that image, and a number of additional data items and queries associated with the image. These will be rolled out progressively between now and Data Release 1; however, what’s there already for DP0.2 - the image itself and the cutout link - will definitely continue to be there.
We also intend to support SIAv2 access to images, which is essentially just a simplified protocol offering a subset of what can be done via ObsTAP. SIAv2 queries will return exactly the same format of metadata table, including the DataLink “links endpoint” approach to access the images themselves and associated services. (This is unlikely to be deployed retroactively for DP0.2, however.)
1 Like
Since the question of the lifetime of the DP0 datasets was also raised in the thread:
This is somewhat of an evolving situation, as it is linked to the commissioning schedule for the observatory, the availability of staffing to provide support, and the continued availability of some of the temporary arrangements we made for computing resources for DP0 before the operations-era resources were in place. We have a partial answer to this now and will lay out a more complete plan at the Rubin Community Workshop in July:
In general we would like to keep the datasets available while they still have significant value to the community in preparing to do science with the survey data, but it is also important to recognize that they are snapshots in time of the data models and processings used to prepare the datasets, and will be increasingly out of date compared to the operations-era data. This is not only potentially misleading to users, but also requires additional staff effort to keep both old and new data and their associated services running at the same time.
For DP0.3 we intend to keep the data available at least until the first availability of Solar System data during operations. We recognize the value to the Solar System community in having a sample of longer-period simulated data available even then, and will evaluate options for this.
For DP0.2 the technical and support challenges are more substantial and we are still evaluating options. We will report on this at the Rubin Community Workshop.
1 Like
I don’t understand what this means. Technically Solar System data products are available in commissioning as prompt products. Who has access to that database varies. So is this saying that during commissioning of LSSTCam that DP0.3 will disappear? Does that mean at DP2?
Not at all. “During operations” meant after the start of the main survey, and “availability” meant to data rights holders as a whole. My apologies for not acknowledging all the nuances involved.