Will there be external TAP access to RSP DP0.2 tables?

For DP0.2 will it be possible to access the RSP through TAP from an external site. E.g., directly running a TAP client (e.g., pyvo.dal.TAPService) from my laptop or NERSC?

3 Likes

I am asking about the opposite direction of access from the following two posts, which were about accessing an external TAP service from the RSP:

For DP0.1, I was able to access the RSP TAP services using TOPCAT on my laptop; so I believe this will be the case. Let me revisit this question in full a little later today…

I get

DALServiceError: 401 Client Error: Unauthorized for url: https://data.lsst.cloud/api/tap/sync

When I try to do it from external to the RSP.

You will need to provide a token in some way (authorization header or as username/password) to access TAP from outside the RSP. See SQR-039: Discussion of authentication and authorization for Science Platform.

As Douglas says, this works today (via https://data.lsst.cloud/auth/tokens/), but I’m not sure it’s as “pretty” as we would like, so there isn’t full documentation for it yet.

Thanks for the pointer to that SQR document, @ktl, I’ll try it out.

@frossie Are there any performance limitations I should anticipate? Or, “How can I be a nice DP0 Delegate and not break things?”

Sorry for the delay. I had to dig into the code a bit…

Anyway, @MelissaGraham pointed me to this useful piece Rubin DP0 Delegate documenation:
https://data.lsst.cloud/api-aspect .

Once you have your data.lsst.cloud TAP service token, you can either use it other TAP access services. I will give two cases: (1) via TOPCAT and (2) via a NOIRLAB DataLab Jupyter Notebook.

TOPCAT

  1. Start up TOPCAT.
    (see http://www.star.bris.ac.uk/~mbt/topcat/ )

  2. Click on 'Table Access Protocal (TAP) Query under the “VO” menu.

  3. Fill in https://data.lsst.cloud/api/tap in the “TAP URL” window and click the “Use Service” button.

  4. Fill in your security token under “User” in the Authentication window that pops up. Leave the “Password” blank. Click OK.

  5. Now you have access to the RSP TAP service from TOPCAT.

NOIRLAB Data Lab

Here, basically you want to use the basic pyvo commands, not the specialized DataLab convenience functions.

  1. Go to the NOIRLAB DataLab ( https://datalab.noirlab.edu/ ) and launch a Jupyter Notebook.

  2. At the very minumun, import the pyvo and requests python modules:

import pyvo
import requests
  1. Define the data.lsst.cloud TAP server URL and your security token:
tap_url = 'https://data.lsst.cloud/api/tap'
token = 'yy-xxxxxx'   # insert your own RSP TAP server security token here.
  1. Set up appropriate authorization to access the RSP TAP server:
s = requests.Session()
s.headers["Authorization"] = "Bearer " + token
auth = pyvo.auth.authsession.AuthSession()
auth.credentials.set("lsst-token", s)
auth.add_security_method_for_url(tap_url, "lsst-token")
auth.add_security_method_for_url(tap_url + "/sync", "lsst-token")
auth.add_security_method_for_url(tap_url + "/async", "lsst-token")
auth.add_security_method_for_url(tap_url + "/tables", "lsst-token")
  1. Access the RSP TAP servics:
tap = pyvo.dal.TAPService(tap_url, auth)
  1. Run a query:
query = "SELECT * FROM tap_schema.schemas"
results = tap.run_sync(query)
results.to_table()

Below, I have screenshots of a full Jupyter notebook running the above in the NOIRLAB DataLab (well, with my security token changed to a generic yy-XXXXX):



I hope this helps! Apologies for the gory details, but I hope to be able to go back to this and remind myself how this works in the future(!).

Thanks!

Oh, it is “official”, then.

For notebook steps 4+5, I found it simpler to do:

cred = pyvo.auth.CredentialStore()
cred.set_password("x-oauth-basic", token)
tap = pyvo.dal.TAPService(tap_url, cred.get("ivo://ivoa.net/sso#BasicAA"))

(and no need to import requests)

2 Likes

Thanks, all! I tried @ktl 's solution and it works perfectly.

Thanks, @DouglasLTucker I appreciate the gory details for exactly the purpose of providing a good detailed answer for people searching this in the future (which will likely include us). I wasn’t familiar with using requests and appreciate learning about building the auth by specifying what to do for the different parts of the TAP service.

@mwv I would prefer @ktl’s more concise recipe be propagated. It’s the result of some recent work on improving Python interfaces to authenticated IVOA services, and it’ll be more robust than enumerating specific endpoints (which could very well change over time - they are not required by the standard to keep those names).

Yep. Here’s what I wrote up and circulated to DESC as a simple example test of NOIRLab and RSP TAP services.

test_rsp_tap_service.py (1.5 KB)

2 Likes