Local access pyvo DALServiceError

Support Data Preview 0
#1

Hi, I am trying to access the data previews from my local machine (Apple M1, Sonoma) using these instructions: Introduction to the RSP API Aspect — Vera C. Rubin Observatory Documentation for Data Preview 0.2

Unfortunately I keep getting the DALServiceError below. My credentials are correct, as I am able to follow the tutorial for topcat on my machine and I can run the pyvo code when on the RSP. I was wondering if there was a version issue with pyvo, so I updated to pyvo=1.5.1 (I also tried creating a fresh conda environment for pyvo) but the error persists.

I saw similar forum topics but the main advice appeared to be log in and out, which I have done. Any advice would be greatly appreciated!

Traceback (most recent call last):
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/pyvo/dal/query.py", line 244, in execute_votable
    return votableparse(self.execute_stream(post=post).read)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/astropy/io/votable/table.py", line 164, in parse
    return tree.VOTableFile(config=config, pos=(1, 1)).parse(iterator, config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/astropy/io/votable/tree.py", line 4167, in parse
    vo_raise(E19, (), config, pos)
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/astropy/io/votable/exceptions.py", line 124, in vo_raise
    raise exception_class(args, config, pos)
astropy.io.votable.exceptions.E19: None:1:0: E19: File does not appear to be a VOTABLE

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jrobinson/adler/notebooks/rsp_tap_pyvo.py", line 18, in <module>
    results = rsp_tap.run_sync(query)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/pyvo/dal/tap.py", line 278, in run_sync
    **keywords).execute()
                ^^^^^^^^^
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/pyvo/dal/tap.py", line 1115, in execute
    return TAPResults(self.execute_votable(), url=self.queryurl, session=self._session)
                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/pyvo/dal/query.py", line 246, in execute_votable
    self.raise_if_error()
  File "/Users/jrobinson/miniconda3/envs/pyvo/lib/python3.12/site-packages/pyvo/dal/query.py", line 255, in raise_if_error
    raise DALServiceError.from_except(e, self.queryurl)
pyvo.dal.exceptions.DALServiceError: 401 Client Error: Unauthorized for url: https://data.lsst.cloud/api/tap/sync/bgkfidcxvge019r9/run
#2

Since you are able to run RSP queries with TOPCAT and with pyvo on the RSP, I am wondering if there could be a problem with your ~/.rsp-tap.token file on your local machine. I don’t think it is a pyvo version issue, since I can run the query in Introduction to the RSP API Aspect — Vera C. Rubin Observatory Documentation for Data Preview 0.2 on my local machine using a pretty old version of pyvo (pyvo v1.4.2).

Maybe you could delete the current ~/.rsp-tap.token file on your local machine and re-create it using Step 1 of Introduction to the RSP API Aspect — Vera C. Rubin Observatory Documentation for Data Preview 0.2 ? You can use the same token as you used for TOPCAT (and, for that matter, what you used when you used pyvo on the RSP), as long as that token is not expired.

My big concern with the current ~/.rsp-tap.token file on your local machine is that perhaps the contents are corrupted in some way or that there is no “end of line” character after the token. In the latter case, listing the file might show something like this:

dtucker@mac-137051 ~ % cat .rsp-tap.token_fake
**-*************************************************%

instead of something like this:

dtucker@mac-137051 ~ % cat .rsp-tap.token_fake
**-*************************************************
dtucker@mac-137051 ~ %

(where “**-*************************************************” would be the token).

So maybe try recreating ~/.rsp-tap.token on your local machine.

I hope this helps!

#3

Hi Douglas, thanks for the advice. I had originally followed those instructions for creating the ~/.rsp-tap.token file but I made the file again just to be sure. I can confirm that I included the EOF and the output of cat ~/.rsp-tap.token looks like your second example. Unfortunately I’m getting the same error!

#4

I wonder if this is a mac thing (specifically a my mac thing…). I just tried the pyvo code (version 1.3) with my token on our department’s unix server (Ubuntu 22.04.4 LTS) and it works.

#5

Hi James, sorry for the problems you are having! Right now, I am grabbing at straws, but I am wondering if it could be a pyvo incompatibility with the M1 architecture of you Mac. I “googled” around and haven’t found anything, but I recall that some other software packages have yet to catch up with the newer Mac chip architecture. Let me ask around and see if I can find out…
Thanks!

1 Like
#6

Okay to recap, same user, same token, works from computer A (random linux box) but not computer B (the M1 mac). Yeah this is unlikely to be on our end however I am curious as to what is going on. @jrob93 - if you wouldn’t mind, can you please post a minimal code example that demonstrates this problem ?

1 Like
#7

Thanks Douglas, yeah I’ve encountered M1 related problems before so it wouldn’t surprise me if this could be the case!

#8

Hi, yes that recap is correct, what I am trying to do appears to work everywhere except on my M1 Macbook Pro. Here is the exact python script that I am running:

import pyvo
import os

RSP_TAP_SERVICE = 'https://data.lsst.cloud/api/tap'
homedir = os.path.expanduser('~')
token_file = os.path.join(homedir,'.rsp-tap.token')
with open(token_file, 'r') as f:
    token_str = f.readline()

cred = pyvo.auth.CredentialStore()
cred.set_password("x-oauth-basic", token_str)
credential = cred.get("ivo://ivoa.net/sso#BasicAA")
rsp_tap = pyvo.dal.TAPService(RSP_TAP_SERVICE, credential)

query = "SELECT * FROM tap_schema.schemas"
results = rsp_tap.run_sync(query)
print(results.to_table())

It’s just the same code given in the example documentation. Thanks for taking a look, please let me know if there is any other info I can send.

#9 
results = rsp_tap.run_sync(query)
print(results.to_table())

And the failure is on the run_sync(), right, not on the next line?

#10

Yup the failure is on this line:
results = rsp_tap.run_sync(query)

I think their error log in the first post confirms this.

FWIW all I can offer here is to confirm that code snippet works fine for me, from the NOIRLab Astro DataLab, with pyvo version 1.4.1.

#11

Sorry to dash hopes of an explanation, but the reproducible snippet does work fine on my M1.

James, sorry to ask again, but when you said it worked from a different machine, did you copy the exact token file you use on your M1 ? Or recreate it in some other way (cut/paste etc)

Also can you list the unix permissions of your token file
ls -al ~/.rsp-tap.token

#12

Hey, when I tried it on a different machine I repeated the steps in the tutorial to create the file securely and I copy-pasted the token into the new file at the prompt.

And the permissions are:

$ ls -al ~/.rsp-tap.token
-rw-------  1 jrobinson  staff  49 24 Apr 09:57 /Users/jrobinson/.rsp-tap.token

But I’m pretty sure that my token and file are fine, I’ve even experimented with temporarily pasting the token directly into the python script and the results are the same.

@frossie could you possibly share details on the conda environment (or similar) that works for you and I’ll try recreate it?

#13

Hi @frossie, just following up with you on @jrob93’s request for details on the conda environment you used to get things to work. Or @jrob93, perhaps you were able to figure out what the issue was?

#14

Hi - I am sorry I don’t use conda. My steps were

  • A fresh virtual env (python-venv, Python 3.11.5)
  • Get token and put in ~/.rsp-tap.token
  • Pip install pyvo
  • Run the code provided by James (which worked fine)
1 Like