Background

At the APAN29 conference, held in Sydney Australia in February 2010, QCIF/UQVislab and AARNet demonstrated transmission of high bandwidth realtime video data simultaneously to multiple OptIPortal display systems. This technique, known as Visualcasting, utilised  a SAGE  bridge provided by AARNet and was able to take advantage of the 10Gb/s connections at both the conference site and the University of Queensland to the AARNet backbone.

The demonstration showed how researchers at different sites could now collaborate effectively in situations requiring the high resolution display capabilities which OptIPortals provide. It built on the experience previously gained through UQVislab's participation (also with assistance from AARNet) in Visualcasting related events at past Supercomputing conferences (SC08 and SC09).


Method

Following the 2009 Supercomputing conference (SC09) it was decided to implement a visualcasting testbed in Australia. Earlier UQVislab testing had been hampered by local networking constraints. Since then, the University of Queensland had upgraded to a 10Gb/s connection to the AARNet national backbone, following which a 10Gb/s network was installed to the UQVislab - just in time for SC09.

The Asia Pacific Advanced Network  (APAN) "provides an advanced networking environment for the research and education community in the Asia Pacific region, and promotes global collaboration". Regular APAN conferences host a number of working groups and sessions including one for HDTV, which exists "to discuss the latest developments around the network based delivery of HD (high definition/4K video technology)". It was agreed that a number of institutions would participate in a series of visualcasting tests using a SAGE Bridge proposed to be built by AARNet. The results of this testing would be presented in the HDTV session at  the 29th APAN conference, held in Sydney, Australia, 8-11 February, 2010. The participating institutions were University of Queensland Vislab (UQVislab), Gwangju Institute of Science and Technology's Network Media Laboratory (GIST), University of Melbourne (UMelb) and Texas Advanced Computation Center (TACC).

The AARNet SAGE Bridge was implemented as a cluster of three machines, each with 10G connection to the network. The machines were initially installed in the UQVislab machine room for setup and testing, for later deployment on the AARNet backbone. Due to delivery delays of various components, the physical installation was not completed until January 2010 and then, after OS and software installation were complete, trials of visualcasting with local machines were not successful. The planned demonstration at the APAN29 conference was therefore recast to be a show of multiple video streams from each of the participating sites. In the days leading up to the conference, this scenario was tested extensively using multiple streams from UQVislab in Brisbane, leading to a new record for customer generated data flow across the AARNet backbone. Sustained (overnight) streams of 5Gb/s were achieved, as well as a brief period of 6Gb/s during testing the day before the demonstration.

It had originally been envisaged that visualcasting could be demonstrated at the conference by splitting AARNet's "OptIPortable" display into two separate units. During the demonstration, one would be designated the "local" display, the other would be the "remote" display. However since visualcasting could now not be demonstrated after all, the OptIPortable was set up as a single 9x3 display during Sunday 7th February. In the meantime, Byungil Jeong of TACC, who had authored the bridge component of SAGE during the time of his PhD study at the Electronic Visualisation Laboratory (EVL) of the University of Illinois, Chicago (UIC),  was able to work with us to resolve our earlier visualcasting problem. We could now demonstrate visualcasting as had been orginally intended. However the OptIPortable had already been set up as single 9x3 display, so how could we simulate "local" and "remote" displays?

Fortunately, the OptIPortal display at UQVislab in Brisbane was being monitored by a camera whose output could be streamed back to the APAN29 conference site and shown on the local OptIPortable display. Thus we would be able to demonstrate visualcasting with a truly remote display along with the local display at the APAN29 conference site.


Results

Immediately before the presentation, remote video streams from GIST, TACC, UMelbourne and UQVislab were commenced. The outputs were positioned on the local OptIPortable display, whose LCDs were then blanked for the duration of the slide presentation.

To open the session, a series of slides by Chris Willing (UQVislab) and Brett Rosolen (AARNet) was presented, giving a quick overview of tiled displays, some requirements for collaboration between such displays and the benefits of SAGE in these situations; in particular, due to the availability of the SAGE Bridge.

Following the slide presentation, the OptIPortable LCDs were unblanked, revealing the remote video streams and highlighting the overall stability of the system by having been running without problem during the half hour of the slide presentation. The stream types were quite diverse, showing a good range of SAGE's streaming capabilities.

From GIST:
    1x 1920x1080 video, compressed in-camera to MPEG-2 Transport Stream, captured via firewire port and streamed using SAGE's SVC application. Showing activities in the GIST lab in Korea, the stream was received by a local machine which also used SVC, this time to decompress the stream and render the result onto the OptIPortable.

From TACC:
    A single 4K animation was streamed directly to the OptIPortable display from TACC at the University of Texas (Austin, USA). The 4096x2160 animation showed the results of a high-resolution, large-scale simulation of jet-induced supernovae using the adaptive-mesh refinement hydrodynamics code Flash performed on TACC's Lonestar supercomputer. Prior to transmission, the animation was compressed to DXT format using built in SAGE tools. DXT provides "light" compression to about 20% of the original size, thereby maintaining excellent image quality while reducing the network bandwidth required to about 600Mb/s in this case.

From UMelbourne:
    A 1920x1080 live capture was streamed directly to the OptIPortable display. Using a Blackmagic Design IntensityPro HDMI capture card, SAGE's new decklinkcapture application is able to stream uncompressed (YUV422 encoded) 25fps video at about 830Mb/s, or DXT compressed 25fps video at about 230Mb/s. As an alternative to showing live camera video of the University of Melbourne machine room, prerecorded blueray video of the UMelbourne campus was input to the capture card.
 
From UQVislab:
    Two SAGE SVC streams were transmitted. However, this time (unlike the GIST case), the cameras' compressed streams were decompressed by SVC in the capturing computers themselves and streamed as uncompressed 1920x1080 (YUV422 encoded) video directly to the APAN29 OptIPortable display.
    Another 2x streams of uncompresssed live camera video using the Blackmagic Design IntensityPro HDMI capture card were also transmitted.
    An Intensity HDMI capture card was also used to capture the 1920x1080 display output of a workstation computer, demonstrating how any application able to be run in a desktop computer could also be simultaneously displayed on an OptIPortal display.
    Lastly, another of the Blackmagic Design cards, the Decklink SDI was used to capture the 720p output of a Tandberg C20 videoconferencing unit camera and stream it to the APAN29 OptIPortable display. This demonstrated the possibility of at least some level of compatibility between traditional VC systems and OptIPortal displays.

The total bandwidth of the incoming streams, which had by then been running for over half an hour, was a little under 5Gb/s.

Following a brief explanation and discussion of the various input streams being displayed, they were closed down and and the visualcasting demonstration commenced.

During the slide presentation, a minimum requirement for collaboration had been suggested and this was to be the target for the demonstration, namely the ability to simultaneously view dynamic content at different display sites. For the demonstration, the screen output of a local workstation running various applications was captured using a Blackmagic Design IntensityPro capture card. The configuration file for SAGE's decklinkcapture application was changed to add location details (bridgeIP & bridgePort) of the SAGE Bridge and setting bridgeOn true. Now the captured stream would reach the display via the SAGE Bridge, rather than stream directly to the display (as previously). Using the SAGE gui, the stream was positioned as desired on the local OptIPortable display. The SAGE gui was then used to connect to the remote OptIPortal display at UQVislab, enabling the gui to be switched (using gui tabs) between graphical representations of both local and remote displays. A camera showing the contents of the remote display was streamed back to the conference site and displayed on the local OptIPortable alongside the local stream. This would enable attendees to visually confirm that any changes at the workstation being captured were being synchronously displayed on both local and remote OptIPortals.

When the icon representing the stream in the local display was dragged onto the tab of the remote display, the stream immediately appeared on the remote display. Continued change of content and movement of windows around the workstation display were seen to be synchronously mirrored in both local and remote displays. The stream data was being replicated by the SAGE Bridge, which sent one copy to the local display and a second copy to the remote display. In fact, the content of the remote display appeared to consistently lag the local display by about half of one second. This was entirely due to the latency caused by the in-camera compression at the remote site and should be considered as just a measurement artifact.


Conclusion

AARNet's SAGE Bridge is able to provide a visualcasting service for suitably connected OptIPortal sites, providing a basis for collaboration between them. Additional confidence in such a service is provided through the knowledge that normal collaborative OptIPortal sessions are unlikely to be conducted in quite such stressful conditions as the APAN29 visualcasting demonstration - a first time event, a live audience, a temporary location with minimal testing time and the use of streams with the highest possible bandwidth.

It had been conceded at the outset of the APAN29 HDTV session that this demonstration would address only a minimal prerequisite for collaboration; the ability to simultaneously view dynamic content at different display sites. Having established the viability of multisite display through the SAGE bridge, the next obvious step toward true OptIPortal collaboration would be to provide multisite control of applications. In this APAN29 demonstration, the application being streamed to both sites' displays was controlled by the workstation whose output was being captured. In this type of situation, it is possible that VNC (or similar) could be used to provide some level of multisite control. However VNC may not always be an acceptable solution, particularly as the number of (possibly "unknown") participants increases. Some method of multisite control for applications running in collaborative OptIPortal sessions is definitely an area for further development.

Perhaps the most important next step toward useful collaboration between OptIPortals is in the area of audio. Currently, if audio was enabled during compilation of SAGE, then audio can be streamed in conjunction with a video stream. This is useful in cases where the audio is intimately associated with a particular video stream. However this may be insufficient when ongoing audio communication between sites is required irrespective of whether a particular video stream is running or not. In particular, collaborators at remote sites need to be able to speak with each other and it is not clear which, if any, video stream(s) this type of audio should be attached to. Just as SAGE's built in text chat tool runs independently of any video streams, an audio channel that is available independently of any other streams seems to be a critical requirement for effective collaboration between OptIPortal sites; another area for further development.

Although the last minute resolution of earlier visualcasting issues led to a successful demonstration, it should be remembered that one of the original intentions - that of conducting extensive organized testing and presentation of results - was not fulfilled. Hopefully this can now be addressed and some results published soon. Judging by the number of questions both during and after presentation and demonstration, it is an area of intense interest amongst existing and potential SAGE users.