2017_05_22 visualization meeting

Data Management Camera-DM Visualization
#1

Bluejeans: 107541143
UIUC people meet in Loomis

Note nonstandard day and time:
Monday, May 22 at 11 AM PDT = 1 PM CDT.

Agenda:

  • UIUC code walk-through. Download the outline (pdf) here.

Notes:

  • Action: @jbpagliuco: Update install instructions.
  • Visualization SW is running at SLAC. No default image.
    load_image ffview /mnt/ramdisk/conn_flat_0100_20170504003759.fits will display one.
  • The “display latest” code is in the updateviewer branch for now. @jbpagliuco will move it to master.
  • Action: @tatianag will point to React and Redux docs (a talk/article by @roby).
  • Action: Stuart will update the SLAC installation and establish the connections to Python libraries (SciPy, etc), that are needed by the back end code. Some Firefly documentation is here.

Next meeting: Tuesday (May 30) at 11 AM (PDT) = 1 PM (CDT).

#2

@tatianag I’ve been trying to figure out how to use the Firefly interface for creating tables, but the docs are a bit confusing. Also, I can’t tell if Firefly supports using raw data for the table, instead of a file. Any clarifications would be great, thanks.

#3

@jbpagliuco where you table data are coming from?

If you have table as a file on the server, the best way is to use table API, described here.

If your table is produced by a python task, see Example 2 at this doc:

    var tblReq = firefly.util.table.makeTblRequest('TableFromExternalTask', 'Table from Python task',
        { launcher : 'python', task : 'TableTask', taskParams : {p1: 1, p2: 2} }, // search parameters
        { pageSize: 15} // table options
    );
    firefly.showTable('tableHere', tblReq);

Firefly can read comma-separated, FITS, or IPAC tables.

If the table is not on the server, the best way for now is to upload it. (We might need to expose file upload function for that.)

In theory, you can use dispatchTabelInsert on the client, but tableModel is not well documented and you loose table-chart connections built into Filrefly. Before we go too deep into it, let’s see what your use case is.

#4

@tatianag, how should I format my return value from the task for a CSV file? I tried to do something basic and all I get is
Fail to load table. Error: edu.caltech.ipac.firefly.server.query.DataAccessException: DataAccessException:Unexpected error from:unknown

#5

@jbpagliuco In CSV file, the first line should be the header, for example:

timeMs,posDiff,gyro,detectedLight
0,0,0,0.12218963831867058
5,0,0,0.12218963831867058

You can check if the format is valid by loading it as a catalog at http://localhost:8080/firefly/ Catalogs->Load Catalog

If you can not resolve it, please, send me your table.

#6

@tatianag I’ve got everything working, but there’s a bug.

I have to delete the temp directory contents, otherwise the table won’t display new information (when I change the python task code). The directory I’m deleting is $CATALINA_HOME/temp/workarea/firefly/temp_files.

#7

@jpagliuco Thank you for the report. Could you, please send me your table to tatianag at ipac.caltech.edu? It would help to debug why the last row is not displayed. (May be we have a bug when the last line is not new line terminated.)

As for caching, it’s probably not a good idea to disable cache for any table. Firefly considers two table search requests equal if all the parameters (excluding system parameters like “filters”, “sortInfo”, “pageSize”, “startIdx”, etc) are equal. Firefly thinks that if the requests are equal, the results should be the same. To create a distinct request consider adding a timestamp parameter. This way, the new request will be different for the purposes of getting the source table. However when getting second page or filtering the table from UI, Firefly would not have to call python to produce the same table again.