I was recently at the Gaia Data Release meeting at ESAC, and a demo of a catalog visualization tool called vaex (http://vaex.astro.rug.nl/) caught my eye there.
This is a fast visualizer of catalog data. The claim is it can visualize (e.g. footprints, CMDs) a ~billion rows in a second on a not too beefy machine. It’s written in pure Python, can output to bqplot or bokeh (so it integrates well with IPython notebooks), and has a native QT interface as well. It did a wonderful job on Gaia catalog data.
My initial thought was that this could be interesting for initially enabling some of our QA as well (and therefore a ping to @RHL, @connolly, @ivezic, @gpdf, et al.). The author – Maarten Breddels – seems very open and interested in feedback & collaboration with potential downstream users. It may be good to have him give a remote tech talk.
Hi @mjuric, vaex looks very nice, worth to try. I also have set up an example using datashader here https://github.com/lsst-sqre/bokeh-plots which could be easily adapted to load Gaia catalog.
Another web visualization product that just got announced is deck.gl from Uber. It’s a framework for WebGL:
Performance: Get high-performance rendering of large data sets (millions of points or vertices), including features like on-the-fly aggregation and visual exploration, based on latest WebGL technologies.
Accuracy: Achieve high-precision numerical computations on the GPU, thanks to our custom fp64 math library. To our knowledge, no WebGL-based library currently provides this functionality, which is crucial for full interactivity of geo data sets.
Extensibility: Use the latest coding standards, including ES2016, and a rich ecosystem of libraries and settings that enable easy debugging and profiling of WebGL applications
It may not be appropriate for DM’s stack, but @benepo might want to take a look at it for EPO web apps.
Thanks @jsick! I’ll take a look. I’ve kept my eye on WebGL for quite a while but it’s always had issues (like crashing the browser, polygon artifacts, lag, etc.) so hopefully Uber’s implementation is a bit more robust.