2020/05 Graphics and Sound Programming

Sound Programming part of the course (19-21 May 2020)

Brief Description of course content:

  • Basic concepts in acoustics and music theory.
  • Digital signal processing.
  • Digital filtering.
  • Fast Fourier transform.
  • Sound synthesis.
  • Audio engines.
  • Procedural sound generation in p5.js.
  • Integration of procedural audio in games.

Calendar:

  • May 19, 16:15-19:45
    Audio engines. Procedural sound generation in p5.js.
    Presentations 1 and 2; Exercises 1 and 2 (see below)
  • May 20, 16:15-19:45
    Digital signal processing. Digital filtering. Fast Fourier transform.
    Presentation 3; Exercises 3, 4 and 5 (see below)
  • May 21, 16:15-19:45
    Integration of procedural audio in games.
    Presentation 4; Exercises 6, 7 and 8 (see below)
  • May 29, 23:59
    Assignment submission deadline

Resources:

Presentations:

Exercises:

  • Ex1-Oscillator: Code / Video
  • Ex2-SoundFile: Code / Video
  • Ex3-AmplitudeFFT: Code / Video
  • Ex4-SineFilter: Code / Video
  • Ex5-RandomLoop: Code / Video
  • Ex6-DelayMap: Code / Video
  • Ex7-ParticlesReverb: Code / Video
  • Ex8-SpaceInvaders (revisions): original code / version with oscillator & noise
    Instructions:

    • Copy the original code.
    • Add foreground sounds (sound files) for a) shooting and for b) destroying an enemy.
    • Combine each sound with a sound processing method: with rate, filter, delay or reverb.
    • Manipulate one of the 2 sounds mapping it to a x or y value in the game (e.g. map the x position of player to filter frequency).
    • Manipulate the other sound based on a random parameter (e.g. make the explosion playback rate a random value).

Assignment:

Create the sound design for a game, using programming techniques in p5.js. Use the game Asteroids as a starting point. Code: https://editor.p5js.org/codingtrain/sketches/VtYr6E4_M (*) The code is spread out in different files, press the “>“ next to Index.html to see all files.

Instructions:

  1. Copy the Asteroids code. You can simply go to File > Duplicate to edit it and save, when logged in.
  2. Add foreground and background sounds (at least 2 of each).
  3. Combine each sound with a sound processing method – for example: rate, filter, delay or reverb.
  4. Manipulate at least one of the sounds by mapping it to a parameter in the game.
  5. Manipulate at least another sound based on a random parameter.
  6. Manipulate at least one graphical element based on sound analysis methods, such as FFT or Amplitude (if needed, add graphical elements).
  7. Explore further techniques in p5.Sound library similar to the ones studied in class.
  8. Comment the code to explain the additions made.
  9. In the first line of the code, add a comment with your name and the URL of the project (for example: https://editor.p5js.org/nunocorreia/sketches/1x2y3z
  10. Download the project files (File > Download). Rename folder with your name. Zip (compress) the folder (.zip extension) and upload to: https://www.dropbox.com/request/dBP2UGqxg2ENUbDzn9DH (deadline: 29/May/2020)

Evaluation criteria:

  1. Quality and appropriateness of foreground and background sounds
  2. Quality, diversity and appropriateness of sound manipulation techniques
  3. Use of mappings of sound manipulation to game parameters
  4. Use of randomization techniques
  5. Use of mappings of sound to graphical elements
  6. Exploration of additional techniques
  7. Quality of the comments to the code
  8. Overall quality of the sound design

(*) From Coding Train website. More information:

2019/12 Multitouch Interaction @ Aalto Media Lab

From 9 to 13 December 2019, I taught the Multitouch Interaction course at Aalto University, Media Lab. More information on the course: http://mlab.taik.fi/mediacode/archives/13506

Made with: openFrameworks
Links: http://mlab.taik.fi/mediacodehttp://medialab.aalto.fi
Projects code available from: https://github.com/aalto-multitouch-interaction/

2019/05 Generative Media Coding @ Aalto Media Lab

From 13 to 17 May 2019, I taught the Generative Media Coding course at Aalto University, Media Lab Helsinki. More information on the course:
http://mlab.taik.fi/mediacode/archives/13532

Made with:
www.openframeworks.cc
Links:
mlab.taik.fi/mediacode
github.com/aalto-mediacode
medialab.aalto.fi

2018/12 Multitouch Interaction @ Aalto Media Lab

From 10 to 14 December 2018, I taught the Multitouch Interaction course at Aalto University, Media Lab. More information on the course: http://mlab.taik.fi/mediacode/archives/13506

Made with: openFrameworks
Links: http://mlab.taik.fi/mediacodehttp://medialab.aalto.fi
Projects code available from: https://github.com/aalto-multitouch-interaction/

2018/10/04 SIIDS Doctoral Symposium @ Madeira-ITI

SIIDS – Sound, Image and Interaction Doctoral Symposium (4/10/2018) was hosted by M-ITI – Madeira Interactive Technologies Institute (Funchal, Portugal), and had two tracks: poster (with oral presentation) and performance. There was also a keynote presentation (and performance) by Prof. Atau Tanaka (Goldsmiths, U. London). Participation was open to all international PhD students conducting research on computer-based systems for interactive sound, in particular combined with image. We had 9 participants for the poster track, and 2 for the performance track. The Symposium was preceded by a talk (3/10/2018) by Fabio Morreale (Queen Mary U. London).

2018/05 Generative Media Coding @ Media Lab Helsinki

From 14 to 18 May 2018, I taught the Generative Media Coding course at Aalto University, Media Lab Helsinki. More information on the course:
http://mlab.taik.fi/mediacode/archives/13487

Made with:
www.openframeworks.cc
Links:
mlab.taik.fi/mediacode
github.com/aalto-mediacode
medialab.aalto.fi

2017/12 Multitouch Interaction @ Media Lab Helsinki

From 10 to 15 December 2017, I taught the Multitouch Interaction course at Aalto University, Media Lab Helsinki. More information on the course: http://mlab.taik.fi/mediacode/archives/13467

Made with: openFrameworks (9 projects) and Unity 3D (1 project)
Links: http://mlab.taik.fi/mediacodehttp://medialab.aalto.fi
Projects code available from: https://github.com/aalto-multitouch-interaction/

2017/11/26 Workshop on Generative Audiovisuals with openFrameworks @ Fine Arts Faculty, University of Lisbon

20171125_111255

The one-day workshop was an introduction to audiovisual composition with openFrameworks and C++

Place: Faculdade de Belas Artes, Universidade de Lisboa
Time: 7 hours duration (2017/11/26, from 11am)
Target: Participants with basic knowledge of programming
Objective: to gain knowledge on combining generative sound and image

Continue reading…

2017/06 INTERACTIVE SYSTEMS @ Digital Media M.A., Hochschule Rhein-Waal

Between 26 June and 1 July 2017, I taught the Interactive Systems course at the Digital Media M.A., Hochschule Rhein-Waal. Below are the support materials we used for the course.

The course is focused on interaction design for media generation and manipulation. Students will experiment with a broad range of media elements with an emphasis on controlling media behavior through code. Special emphasis will be given to crossings between interaction design, sound and image. The main development environment for the course will be openFrameworks (http://www.openframeworks.cc/), although other environments can be explored. Students are required to plan, develop, document and present a project at the end of the course.

Program:

Continue reading…

2017/04 Multitouch Interaction @ Media Lab Helsinki

From 10 to 14 April 2017, I taught the Multitouch Interaction course at Aalto University, Media Lab Helsinki. More information on the course: http://mlab.taik.fi/mediacode/archives/13403

Made with:
www.openframeworks.cc
Links:
mlab.taik.fi/mediacode
medialab.aalto.fi