ArtSciLab Paper by ATEC PhD Candidate Accepted by Union College Symposium on Engineering and Liberal Education

An ArtSciLab paper by Kathryn Evans, Senior Lecturer in Music and ATEC PhD Candidate at UT Dallas, has been accepted for the Union College “Engineering and the Liberal Education” symposium in Schenectedy, NY.
Below is an abstract:


 
“Does studying music enhance higher order learning skills in undergraduate non-music majors?”
Authors:
Kathryn Evans, Senior Lecturer in Music, School of Arts and Humanities, Frank Dufour, Associate Professor, Rosanna Guadagno, Associate Professor and Roger Malina, Professor, Arts and Technology, The University of Texas at Dallas
Abstract:
Many studies have looked at the correlation between music study and academic skills. A review of over 11,000 studies between 1950 and 1990 conducted by Harvard Project Zero tested the claim that studying the arts leads to some form of academic improvement. Only three areas were found that demonstrated a clear causal link between education in an art form and achievement in a non-arts, academic area. Two were in music: a medium-size causal relationship between listening to music and spatial-temporal reasoning and a large causal relationship between learning to make music and spatial-temporal reasoning. (Winner 2001). The majority of these studies have been conducted with students in primary and secondary education, but little research has been done on students at the undergraduate college level who study music, either as a minor or for general interest. Most pedagogical studies in music address the needs of music majors and not non-majors.
This pilot study looked at students at the University of Texas at Dallas who enrolled in music studies (either music performance, music theory or sound design) who are not majoring in music. Many are students in the STEM (Science, Technology, Engineering and Mathematics) areas. Through phenomenological research methods, we looked at the experiences of students who study music or sound design and how they perceived it affects their academic skills in other areas. Emails for students currently enrolled in music or sound design courses in AY2014-15 were obtained from registration records and they were solicited to take an on-line survey, with an option to volunteer for an in-depth interview. Over 800 students were solicited in February 2015 and a response rate of 20% has already been obtained. Additionally, over 30 students have volunteered for the interview. Initial data and a preliminary analysis will be presented.

DataRemix Paper Live on Leonardo Just Accepted Page

“DataRemix: Designing the Datamade,”is now on the Leonardo Just Accepted page, hosted by MIT Press. The paper was a part of the Special Section of Leonardo Transactions “Highlights from VISAP’13”, and was previously announced by the ArtSciLab as presented at the 2013 IEEE VIS Arts Program (VISAP) in Atlanta, Georgia. The authors of the paper were Ruth West, Roger Malina, John Lewis, Scot Gresham-Lancaster, Alejandro Borsani, Brian Merlo, and Lifan Wang.
The full paper can be downloaded here: DataRemix: Designing The Data made Through ArtScience Collaboration
The article is forthcoming in the Leonardo print publication, and can be cited with the DOI: 10.1162/LEON_a_01060.

ArtSciLab Paper Presented at IEEE VISAP Conference

Analogy and Conceptual Blending are Part of a Visualization Toolkit for Artists and Scientists: Introducing the Cognitive Space Transfer, an ArtSciLab paper by Jack Ox, was presented at the IEEE VIS 2014 Arts Program, VISAP’14: Art+Interpretation in Paris, France, held November 9th-14th 2014.
Below is an abstract:
Analogy and Conceptual Blending are Part of a Visualization Toolkit for Artists and Scientists: Introducing the Cognitive Space Transfer
Author:
Jack Ox, University of New Mexico
Abstract:
This paper demonstrates knowledge representation mapping techniques common in both the domains of art and science. Analogical mapping systems take information from a source domain and map that data to a target domain located in another perceptual mode. I also explain conceptual blending, in which information from different sources combine into a new emergent structure. The theories that describe these visualization processes are conceptual metaphor theory (CMT) and conceptual blending theory (BT), which were orginally created by George Lakoff, Mark Johnson [15], Gilles Fauconnier and Mark Turner [4] more than thirty years ago. My own work of visualizing music also began in the late seventies, coincidentally during the same period of time that CMT and BT were being conceptualized and written down. I will illustrate the use of analogy as a basic visualization tool through describing visualizations of extant music, including the twentieth-century, intermedia masterpiece––the Ursonate by Kurt Schwitters. The cognitive space transfer is an important part of this process; it is a type of conceptual blend. I developed this method while creating art works, but predict that it can also contribute a rich, qualitative dimension to scientific visualization that adds in a substantial way to the story told by the information.
Keywords:
Analogy, cognitive space, conceptual blend, metaphor, knowledge representation, model, visualization.
Introduction:
Even the most austerely ‘scientific’ models operate through analogy and metaphor. The Rutherford-Bohr model depicts a hydrogen atom as a miniature solar system. Darwin’s concept of ‘natural selection’ is analogous to the ‘artificial selection’ process practiced by animal breeders [2].
Beginning in the seventeenth-century and continuing through to the present, science has developed strong analogical processes in order to create new knowledge and make concrete, originally abstract concepts. Scientific models are analogies [5, 11]. The mode of re-expression, or representation, is usually other than linguistic, for example visual and/ or sonic. A model is always a partial mapping; part of creating a successful model is the knowledge of what to filter out from the mapping process. It must be limited because including all information would be an uninteresting duplication of the original [11]. By looking at data in a new mode or domain, researchers are able to see it in different ways, sometimes bringing about a conceptual change that is dramatic enough to cause a frame shift.


 
[2] T. L. Brown, Making Truth; metaphor in science, Urbana and Chicago: U. of Illinois Press, 2003.
[4] G. Fauconnier, and M. Turner, The Way We Think; Conceptual Blending and the Mind’s Hidden Complexities, NY, NY: Basic Books, 2003.
[5] D. Gentner, and M. Jeziorski, “The shift from metaphor to analogy in Western science,” Metaphor and Thought, A. Ortony, ed., pp. 447-480, Cambridge: Cambridge University Press, 1993.
[11] R. Harré, J. L. Aronson, and E. C. Way, “Apparatus as Models of Nature,” Metaphor and Analogy in the Sciences, F. Hallyn, ed., pp. 1-16, Dordrecht: Kluwer Academic Publishers, 2000.
[15] G. Lakoff, and M. Johnson, Metaphors We Live By, Chicago, Ill.: University of Chicago, 1980.

ArtSciLab Paper Accepted for Understanding Visual Music 2016, Brazil

An ArtSciLab paper by Andrew Blanton, Connectome Data Dramatisation: The Human Brain as Visual Music, has been accepted for Understanding Visual Music to be held June 10, 2015 in Brazil.
Below is an abstract:


 
Connectome Data Dramatisation: The human brain as visual music.
Authors: Andrew Blanton, MFA; Sruthi Ayloo, MS; Micaela Chan, MS; Scot David GreshamLancaster, MA, MFA; Roger Malina, PhD; Tim Perkis; Neil Savalia, BA; Maximilian Schich, PhD; Anvit Srivastav, MS; Gagan Wig, PhD
Abstract
We, as a collaboration of scientists and artists, have built a visual and sonic representation of highly connected areas in the human brain. This model was developed to not only be a tool of scientific research but also as a tool for art creation. In the process of developing the software, the tool was built to interface with musical instruments for real time visualization and sonification. Working conceptually with the idea that scientific data can be repurposed for art creation, the Connectome is performed as both a sonic and visual representation of fMRI data, manipulating the model in real time as a form of multimodal data dramatisation.
Introduction
Partnerships between artist and scientist allow for creative forms of collaboration that can push both scientific and artistic research. With the Connectome Data Dramatisation project, our principal interest was in the creation of a hybridized tool, one that could work as both scientific instrument as well as artistic work. Beginning with a dataset that consisted of 441 neural bundles or nodes systematically differentiated into 21 areas or systems of interest in the human brain based on fMRI data collected by one of us (Gagan Wing) as part of the work of the UTDallas Cognitive Neuroimaging Lab.[1]
Area Centers Coded by System Membership
Our team was able to extract visual and sonic representations of the connections between those areas using custom software. We then developed that representation further in the form of an interactive three dimensional node edge graph and sonification of the 421 highly connected areas of the brain (in the case of the visualization, the width of the edges).
This would form the basis of the representation. With the addition of the ability to activate nodes from external data feeds via Open Sound Control[2] different nodes could be excited at will creating a virtual, three dimensional instrument that could be used for visual and sonic performance. Using four small drums, the visual and sonic representation of connections between areas of the brain can be played in real time. Custom software receives input in the form of audio signal from each drum and excites specific areas of the brain. Each section of the brain that is played will present a unique visual and sonic representation.
Historical Perspective
Building on previous explorations in bridging art and science through the development of new technology, we were actively looking to understand how this project is situated within the history of visual music. In looking at the work done at Bell Labs in the 60’s and 70’s[3] and with the work of artist such as James Whitney[4], the question emerges, what are the components of a successful art and science collaboration? How do separate practitioners collaborate while furthering each of their own research? Phill Mortin and Dan Sandin’s image processing units[5] also played a role in both the conceptual development as well as the technical development of the work. How is information shared and disseminated after it’s creation? Other contemporary artist were looked at as well including the work of Noisefold[6] in their sound extraction techniques form visual information, Ryoji Ikeda[7] in his visual and sonic representation of data as well as Semiconductor[8] in their blending of art and science amongst others working with visual music as a contemporary practice.
Visual music has been historically tied to the development of technology. This holds true now as much as it has in the past. Current rendering technologies are evolving rapidly within the gaming community and practitioners of visual music are greatly benefiting from real time rendering advancements within the gaming communities. Robust community support and the indie gaming movement have provided new tools for interfacing with gaming environments[9]. Two areas that are underdeveloped with regard to these environments and practitioners of visual music can provide insight are in the development of procedural animation, and the assimilation of data into these environments. With this project we have begun to build a framework that can both provide a series of procedural animations with regard to node edge graphs as well as interface a gaming environment with a dataset of approximately 77,000 connections. In doing so we have tried to maintain the work as both a piece of art and a scientific instrument.
Future Work
In the process of building this project, we have worked with many technologies to find the right combination of frameworks and development to allow for extensive flexibility in artistic representation of the data set. We have worked with Max/MSP Jitter[10], Unity[11], Syphon[12], Three.js[13], node.js[14] midi.js[15], coffee collider[16] and D3.js[17] in a exploration to find what technology would serve the representation of this dataset best. Beginning with a representation using three.js hosted on a node.js server we were able to bring in live data via OSC to trigger the model. We found ultimately that building everything in the web browser provided great accessibility for global use of the tool, however, confining the project to the web browser also creates limitations with regard to power for rendering and audio synthesis. We have built a framework that now uses the Unity game development environment specifically for it’s strength with regard to real time rendering and are working on integration of Pure Data[18] via the Kilimba Unity extension[19]. This process will allow us to build a platform addressing the two primary areas of dataset integration into gaming environments and procedural manipulation as well as sonification and visualization of said dataset.
Summation of Findings
The creation of the Connectome project has led to some interesting further work in collaborations between artist and scientist. Beginning with the fundamental question can scientific instruments be used as tools for art creation and can artist tools produce scientifically valid results, our team was working to further a dialogue between artist and scientist while creating real value for each party involved. In doing so we have opened up another path of exploration in the form of using game development platforms for data visualization and sonification as well as the reappropriation of these platforms for use in real time audio visual work. By creating a core representation, we were able to build a model that could be manipulated in real time using incoming Open Sound Control data and provide a scientifically accurate representation of the underlying dataset.


[1] Area of interest in this case were areas of concentration of neurons in the brain as identified by researchers at of the Center of Vital Longevity Cognitive Neuroimaging Lab at the University of Texas at Dallas. http://vitallongevity.utdallas.edu/cnl/ accessed march 7 2015.
[2] http://opensoundcontrol.org/ accessed March 7 2015
[3]http://www.ieeeghn.org/wiki/index.php/Archives:Bell_Labs_%26_The_Origins_of_the_Multimedia_Artist accessed March 7 2015
[4] William Moritz on James Whitney’s Yantra and Lapis http://www.centerforvisualmusic.org/WMyantra.htm accessed March 7 2015
[5] Museum of Modern Art https://www.moma.org/momaorg/shared/pdfs/docs/press_archives/5958/releases/MOMA_1982_0014_14.pdf?2010 accessed March 7 2015
[6] http://noisefold.com/
[7] http://press.web.cern.ch/press-releases/2014/01/japanese-artist-ryoji-ikeda-wins-third-prix-ars-electronica-collide-cern
[8] http://semiconductorfilms.com/
[9] http://pjim.newschool.edu/issues/2011/01/pdfs/ParsonsJournalForInformationMapping_Medler-Ben+Magerko-Brian.pdf
[10] https://cycling74.com/ accessed March 7 2015
[11] http://unity3d.com/5 accessed March 7 2015
[12] http://syphon.v002.info/ accessed March 7 2015
[13] http://threejs.org/ accessed March 7 2015
[14] https://nodejs.org/ accessed March 7 2015
[15] http://mudcu.be/midi-js/ accessed March 7 2015
[16] https://github.com/mohayonao/CoffeeCollider/wiki accessed March 7 2015
[17] http://d3js.org/ accessed March 7 2015
[18] http://puredata.info/ accessed March 7 2015
[19] https://github.com/hagish/kalimba accessed March 7 2015

ArtSciLab Students Move Onto New Ventures

Two students in the ArtSciLab, Sruthi Ayloo and Stephanie Brisendine, will be graduating this spring 2015 semester and going into new jobs and opportunities.
Sruthi will be graduating with her Masters in Computer Science from UT Dallas. She says:
Working at the ArtSciLab as a graduate research assistant was the best thing that happened to me during my graduate studies. It was an amazing opportunity, as I got a chance to play and experiment with new technologies and was also introduced to the arts and music. I learned a lot during my stay here. Interacting with diverse people within the lab gave me a better understanding of various cultures and made my work fun. There were days when I used to get more work done for my lab than my coursework as I truly relished my job here and I really loved what I did.
I have always wanted to work on products that help with advancing computation across fields. In line with this goal, I have taken up a full-time offer as an Application Support Engineer at Mathworks (Massachusetts) and will be joining their Engineering Development Group. I will be solving different types of MATLAB problems and at the same time work with different teams within the company to build better features for MATLAB and its various toolboxes.
Stephanie will be graduating with her undergraduate degree in Emerging Media and Communications from UTD. She says:
My post-graduation plans evolved as I gained confidence in my ability to actualize opportunities. My two years at UTD, and more specifically, my year and a half with the ArtSciLab has given me confidence and curiosity in the world again. I’ve been able to learn, fail, experiment and create things that I never thought possible before. Being surrounded by an infinitely interesting international and academic community has also played a crucial role in my long-term goals. I remember thinking that finding a job after finishing my undergraduate would be sufficient, because going to grad school just equaled more debt. However, my current long-term goals include graduate research as well as becoming internationally established. The beautiful thing is there are multiple paths that I can take – job offers, grad school, abroad opportunities – which is extremely exciting. And as of late March, I’m happy to say that I was accepted into the JET Program in Japan and I will be living there for at least a year. I plan to use this time to become fluent in Japanese as well as continue investigating some of the questions I present in my capstone (senior project) this semester.
Congrats and good luck to Sruthi and Stephanie!