Ready for discussion (3/3/16)
Note (3/6/16): I (@jjt) have added @tatianag’s (tg): comments in-line to make the discussion easier to follow.
Architecture:
User console:
Two use cases:
Use cases are now in a Google doc.
- Select an image region and perform some function on it.
- Blink two displayed images.
Select an image region and calculate some statistics in it.
Assume that an image is being displayed in one box.
-
User selects a region:
-
Using FF tool.
-
By entering coordinates on the Javascript command line.
(tg): I think if coordinates are entered into command line, the region still needs to be displayed -
Send command to FF client side communication channel, by typing this:
<Command> <region> <Image_source_URL> <Result_box>
Nominal region format (if rectangle):xleft ,ytop, xright, ybottom. -
Do we want to be able to enter either image or display pixels?
-
The image URL is only needed if more that one image is being displayed. Is there another way for the server to figure it out?
(tg):: A user should be providing Image URL or file name only when choosing which image to load into image viewer. After that, the user should be referencing the displayed image by the name (for example, div name passed to makeImageViewer). The client code should keep track of the image, displayed by the given image viewer. In future, it should be able to query FF for this info. -
If
<Result_box>is not passed to the server, how does the client know what to do with a response message? Should the command interface block? I have in mind that some commands might take some time to process.
(tg)::<Result_box>should not be passed to the server. Javascript code should use Promises for asynchronous calls. -
Server receives the command.
-
How do we synchronize the list of valid commands? (need configuration files).
(tg):: Firefly Extension API makes it easy to define context actions when a selection is made. Perhaps your client code should start using this mechanism to define the actions available on a particular selection. -
Is any validity checking needed? If so, how are errors returned? We may need a separate communication channel, because the client is looking for a “result” in a particular format. (or, maybe, each return format has an error value - seems like a kludge).
(tg):: If the region is displayed on the image it helps with validation. -
(For the future) We may have several client and server instantiations. How is the linkage established and maintained? If one server deals with multiple clients, a more complicated communications protocol will be required.
(tg):: FF is designed to run in multi-server, multi-client environment. In fact we do run it this way in production. The interface should be designed in such a way that all necessary information is in the request. -
Server Python processes the command.
-
It needs to know what image to process (if URL is not part of the command).
(tg):: We should separate end-user interface from the internal interface (between Camera Javascript code and FF). Camera Javascript code will know or will be able to query FF for the path of the displayed image. The path to the displayed image file will be passed from Javascript client via FF API to Python server-side code. -
It may need auxiliary data (e.g., compare image with a template). How will it know where to find this?
-
After processing, server sends the response.
-
Response format is determined by
<Command>.
(tg):: Python server-side process can return any information in JSON format. You can also tell FF to display FITS image or table produced by the server-side Python. -
(For the future) If multiple clients, how does FF know where to send it?
(tg):: Firefly is using shared file system and is replicating cache between server nodes. For efficiency, it might be using sticky sessions to make sure the subsequent requests go to same server. -
Client receives the response and displays the result in <Result_box>
-
If the interface is non-blocking, we need to think about how to associate responses with commands.
(tg):: As I mentioned, this problem is solved with Javascript Promises, which allow to handle the response whenever it becomes available. -
Other issues:
-
It may be that the response is an update to an existing graph (e.g., a time history).
-
The user may want to manipulate (e.g., compare) results (perhaps in separate boxes. The data must be kept in retrievable form.
Blink two displayed images
I am choosing one scenario. There might be many ways to do this.
Assume that two images are being displayed in two boxes.
-
I have in mind two commands,
Start_blinkandStop_blink. That is, blinking is a persistent state. -
I am assuming that the user will want to see both images separately during blinking.
-
User selects a the two images, by naming the boxes.
-
Command:
Start_blink <image box 1> <image box 2> <display box> -
The image in
<box 1>will appear in<display box>. -
The client is now in a
blinkingstate, and some behaviors will be different. -
Each click in
<display box>will cause the other image to be displayed there - the display toggles back and forth under user control. -
While in the
blinkingstate, the user can manipulate the image display properties. -
Panning, zooming, and other changes of image display properties are automatically applied to both images.
-
The user might want to pan, zoom, etc., the
<display box>independently, in order to focus on a small region (e.g., change the contrast) without losing the big images. -
The user might want to obtain statistics from both images in the region displayed in the
<display box>. -
To leave the
blinkingstate, the user enters this command:
Stop_blink <display box>. -
Other issues:
-
Low latency is an important property of the
blinkingstate. -
Some commands will require two commands to the back end, to modify the display of both images.
-
This behavior is one reason one needs a
blinkingstate.