Tom Defanti (Calit2, EVL) and Bernard Pailthorpe (UQVislab) viewing an
unscaled 5760x4000 Google map of Brisbane
of the unscaled 5760x4000 Google map of Brisbane
|Using a 3D
wireless mouse to interact with a 5760x4000 genome browser
5760x4000 genome browser. Many of the genome features are themselves
links which can be followed by clicking with the 3D mouse.
technique is not restricted to web browsers. Here, an unscaled
3840x2400 instance of Paraview is running alongside the genome browser.
scaling, the large rendering windows result in quite small looking
menus and other features. Notice the yellow SAGE pointer for
The technique entails running the target application inside a large X virtual frame buffer, the contents of which are delivered to SAGE using the screamer (SCReen strEAMER) application. As well as streaming the contents of the frame buffer, screamer also synthesizes pointer events (click, drags) inside the frame buffer environment based on the activity of the SAGE pointer. Since an X virtual frame buffer has no event mechanism of its own, it is screamer's ability to synthesize events which provides interactivity with whatever application is running in the X virtual frame buffer environment.
The overall steps are as follows:
Of course, the screamer appplication needs to be available first.
To build it, first download the
latest screamer source tarball.
Unpack it somewhere and then install the included
DXT shared library before building screamer itself. Typically, the
following sequence of commands will do everything required (change version
numbers as necessary):
tar xvf screamer-0.2.3.tar.gz
Everything seems to run very slowly
The sluggishness is related to the size of the virtual frame buffer. Using a 1920x1080 frame buffer results in reponsiveness similar to a regular desktop. As larger frame buffer sizes are employed, the reponsiveness is correspondingly reduced and there are two main causes for this. Firstly, in applications like a web browser, the downloading of rich content e.g. a Google map, takes much longer for a large viewport than for a regular vewport. There's just a lot more data to download and render. We really have no control about this aspect of the problem. Secondly, the contents of the virtual frame buffer need to be regularly transferred to the SAGE buffer. The time to complete this transfer is also ralated to the buffer size. For a 1920x1080 buffer, we achieve about 30fps; for a 5760x4000 buffer, we achieve about 3fps with DXT compression enabled (about 1fps without). Since this data transfer is performed by the screamer application, it is an area ripe for further work. Some optimisations which immediately spring to mind are:
What about keyboard input? We can now click on links etc., but how do we input a new url into the browser?
Keyboard events aren't yet exposed to SAGE appplications - now that there is a use case, perhaps they will be soon.
My target application (paraview) doesn't support the "--display"
Set the environment DISPLAY variable e.g.
Please send any comments, corrections, advice etc., to Chris Willing <c.willing _at_ uq.edu.au>