New Mexico Supercomputing Challenge

Synesthesia Visualization

Team: 1

School: Academy For Tech & Classics

Area of Science: Neuroscience


Problem Definition:

Synesthesia has rarely been modeled. Synesthesia is a rare condition in which the
stimulation of a perceptual sense produces a sensation in a different modality. This condition was documented nearly a century ago; but until recently, there was no way of testing for this condition and there exist few simulations. Over twenty different versions of synesthesia have been documented; examples include tasting colors, and smelling sounds. The most common and documented form of it is “seeing” audio- be it a simple tone or a complex symphony.
For our project, our team has decided to take actual data and personal accounts of synesthetes who see audio, and create a visualizer in Python that allows users to see the same as a synesthete might. Our most pressing issues are finding and optimizing human optical functions, such as motion detection; and acquiring accurate descriptions of synesthesia

Problem Solution:

With the OpenCV library for python we will, hopefully, model the synesthesia condition of
seeing audio with a minimal margin of error and disinformation. Our data will be derived from personal descriptions online and, with luck, interviews with synesthetes who experience this extraordinary condition. Synesthesia can vary in intensity and our model will include settings for this. We wish to keep our project open to many people. As such, the model will require only a somewhat rapid computer, a microphone and a simple camera.

Progress to Date:

We are expanding our knowledge of primary experiences via online personal accounts,
published scientific reports, and an anonymous man in North Korea who shared some of his research with us concerning the topic and his experiences with the condition– he requests his research remain unpublished. We're also experimenting with the OpenCV computer vision library in Python, and we are researching computer vision fundamentals. These include facial recognition haar cascades, good feature detection, human retina modeling, and motion tracking. Dropbox is used for project synchronization and data transfer. OpenCV and subsequently, our model are running on the Linux Mint operating system and Windows 7– which is run on as a virtual machine on Mac OSX. We had trouble installing OpenCV on the latest version of OSX.
Expected Results:
We expect to be one of the first, if not the first, to accurately simulate the synesthesia of
seeing audio according to first person accounts. With this new model our knowledge of the human mind's perceptual capabilities will increase, as will synesthesia awareness, and perhaps usher more synesthesia research.
opencv libraries for Python

Pre-compiled ideas on the sort

Synesthesia and the human psyche

David Eagleman's work in the subject

One of the many personal accounts

Team Members:

  Noah Caulfield
  Hugo Rivera

Sponsoring Teacher: NA

Mail the entire Team

For questions about the Supercomputing Challenge, a 501(c)3 organization, contact us at: consult @

New Mexico Supercomputing Challenge, Inc.
Post Office Box 30102
Albuquerque, New Mexico 87190
(505) 667-2864

Supercomputing Challenge Board of Directors
Board page listing meetings and agendas
If you have volunteered for the Challenge, please fill out our In Kind form.