Hi, thanks for your question! TAP queries should be able to handle millions of rows (I found this documentation that references a limit of 5 million). But, regarding speed, it is recommended that spatial constraints be included in the search if you are using DP0.2, because those catalog data are sharded by coordinate (RA, Dec). So excluding a spatial search will slow it down a lot. Wondering if you could provide further information on the query you are trying to perform so we can help further (or attach it here)? Thanks!
Thanks again Christina and apologies for not picking that up in the documentation.
Our use case here is mimicking uploading a table of 20 million objectIds and cross matching with a “science” table in qserv. The results here are just over 10 million rows so will need to split things up a little.
There’s also this information about current Qserv/TAP limitations:
As for the internal RSP enviroment, the database server will generally be unable to process queries that have an internal intermediate form that is larger than roughly 5GB, and the TAP service will have difficulties with final results that are over roughly 2GB in size. That said, this is by no means the final word on what limits might be in operations.