As mentioned in my previous post, I was invited by PhD student Ben Smith to participate in a telematic musical session between the NCSA and Rensselaer Polytechnic Institute in New York. The idea of the session was to utilise a high-speed internet connection to enable musicians to collaborate in real-time using live audio and video.
| The telematic musical session |
The session was divided into two sections. In the first, the musicians were invited up individually to improvise freely with a musician on the other end. The second section was similar, except that the musicians were instructed to gradually introduce a beat / pulse into their collaborative improvistion.
The reason that the session was structured in this way is that there are significant challenges associated with latency, amounting to approximately 1/16th of a second. It was immediately apparent that although this degree of latency is largely unnoticeable and generally considered acceptable when videoconferencing, it is significantly noticeable when collaborating rhythmically. As such, the first section allowed us to familiarise ourselves with collaborating with the other musicians through free improvisation over the remote connection. The second section was aimed at testing the ability for the musicians to improvise and introduce a beat, while dealing with the latency evident in the audio connection.
We found that it was very difficult (impossible) to play in perfect synchronisation because of the latency evident in the connection. The noticeable delay would cause each musician to feel the urge to speed up to compensate, which would in turn result in the other musician speeding up, creating a perpetual accelerando. As such, we had to deal with the latency in other ways; by creating syncopated or abstract rhythms, or for one musician to create the rhythm and the other to focus on melodic passages.
We found that it was very difficult (impossible) to play in perfect synchronisation because of the latency evident in the connection. The noticeable delay would cause each musician to feel the urge to speed up to compensate, which would in turn result in the other musician speeding up, creating a perpetual accelerando. As such, we had to deal with the latency in other ways; by creating syncopated or abstract rhythms, or for one musician to create the rhythm and the other to focus on melodic passages.
After the session, we discussed how we felt the session progressed. We all agreed that very minimal latency in the audio transmission was vital to enable the performance of music where all instruments maintain the same tempo and rhythm. Low latency was also important for the video, but we felt that this was less critical. The main reason for this is that the musicians (in this session) used the video primarily to 'feel' the presence of the remote musicians - it was not used as much specifically to follow or lead the song itself. I would assert however, that as the audio latency improves to the point where it is unnoticeable, low video latency will become increasingly important to the collaborators.
Ben has held a number of remote concerts to date, and they all consisted of abstract, improvised music. A suggestion I made was that it was important for the bulk of the audience to 'understand' the music that is being played - the music needs to appeal to the audience. Abstract and fully improvised music may not be as easily intelligible to the audience and I recommended that we should experiment with other styles / rhythmic combinations with more defined and familiar chordal progressions, performed at a tempo appropriate for the amount of latency on the connection.
I thoroughly enjoyed participating in the session and it inspired me to conduct a similar session between Monash and NCSA. I raised this with Ben and he shared my enthusiasm about the idea. I called David Abramson from Monash straight away to discuss the idea and he indicated that he would also be keen to help me organise it. I will be putting a proposal together during the course of this week to get the ball rolling.
| Taken during telematic testing with Ben |
We started off with some rhythmic testing by clapping, then moved on to some gypsy jazz standards including Minor Swing and Honeysuckle Rose. We spent the remainder of the session improvising and experimenting with various rhythmic and melodic/harmonic combinations. The session was very successful overall and we will be catching up next week to conduct some more tests. Ben kindly offered use of one of his violins until I leave :)
As for my AR project, I have made some good progress over the last week. I arranged a meeting on Tuesday to discuss the progress of the project and Alan and Bob seemed happy with the work that I had done. They were happy with the layout/format of the website and we decided to proceed and move the site to a live server.
I also had a chance to show Alan and Bob some of the 3D models that I had found to consider porting to the AR platform. Bob took a liking to some of the models and has now started to port them across. I will update you later on the progress of this. During the meeting, we also agreed that it would be a good idea to carry out some research on the other AR platforms that are known to the market and provide a summary of each. I have partially completed this and will try to discuss it in more detail in a subsequent post.
On the way to the Fab Lab a couple of weeks ago, Bob showed me around the Krannert Center for the Performing Arts, a large concert hall affiliated with the university College of Fine and Applied Arts. While I was there, I noticed that a tango group (8 dancers, piano, bass, guitar, bandoneon, violin) was scheduled to perform on Thursday. I bought a couple of tickets and thoroughly enjoyed the show. Although it wasn't the first tango show I had been to, it definitely carried additional meaning after having completed a Tango dance course.
Friday also turned out to be a very nice evening. Firstly, I attended a musical/dance performance conducted by some people from the telematic musical session. The after-hours performance was held in the UIUC School of Music Library by The COLAB Ensemble, "a cooperative laboratory for collaborations between musicians, dancers and engineers". The performance was fairly abstract and avant-garde and the audience were invited to move throughout the dimly lit library to a number of scenes where the performance took place. The show was very unique and enjoyable overall.
I raced off to my friend's place after the show to help her prepare for a party in the evening. She had spent a great deal of time (two days!) preparing tamales for the party and I prepared some guacamole and and a salad (under her instruction). The night was very enjoyable - plenty of nice company, great food and dancing. I decided to spend the weekend in Chicago and I caught an early bus the following morning. I had been to Chicago a couple of times for salsa, but wanted to spend a couple of days to explore the city.
| The Chicago skyline from the bus |
I raced off to my friend's place after the show to help her prepare for a party in the evening. She had spent a great deal of time (two days!) preparing tamales for the party and I prepared some guacamole and and a salad (under her instruction). The night was very enjoyable - plenty of nice company, great food and dancing. I decided to spend the weekend in Chicago and I caught an early bus the following morning. I had been to Chicago a couple of times for salsa, but wanted to spend a couple of days to explore the city.
![]() |
| "The Bean" |
As for this week, I'm looking forward to attending TEDxUIUC on Saturday during the day and then leaving once again for Chicago on Saturday evening to spend my last weekend in Illinois at the Chicago Salsa Congress!










