How to visualize sound on the web: a selection of thematic materials and video lectures with theory and practice

Last time we prepared a digest for you with open libraries for visualizing audio content and decided to find materials on how to work with all this. The result is a selection of guides on the topic for developers of web applications or games.

^{Photo s_keller / Pixabay}

In our blog on Habré -> Reading for the weekend: 65 materials about streaming, the history of the old "musical iron", audio technologies and the history of acoustic manufacturers

Theory and History

Theory: methods for highlighting the fundamental tone . A member of the Ableton development team, a studio-based sound program, parses a few common algorithms for isolating the pitch (repeating sound wave cycle) in music recordings. Examples are: the zero crossing method and the method based on the identification of the fundamental harmonic. The author also cites several additional reading materials that discuss advanced approaches to highlighting the fundamental tone. The article may be of interest to those who plan to write a music application that displays graphs of sound waves.

A brief history of synthesizers . This is a video of one of the speeches at the Scotland JS conference. Chris Lowis, editor of Web Audio Weekly, talks about how synthesizers have evolved and how they work. All this with examples on JS.

Web Audio API Basics . Mozilla developers talk about Web Audio API features, specifications for managing audio content in browsers. It describes both basic things like interacting with audio buffers, as well as more advanced ones (for example, visualization of sound). Many graphs, diagrams, and code examples. The material is regularly updated - it can be used as a “desktop” reference when developing applications.

8-bit Music Theory . A YouTube channel dedicated to parsing video game audio from a musical theory perspective. For example, in this video, the author shows by examples how music helps to create an atmosphere in the Hollow Knight game, and here we are talking about one of the final compositions in Dark Souls. The channel can be useful to those who write their own game. Here you can pick up a few ideas for her musical accompaniment.

Practice

Explore the possibilities of the Web Audio API with the D3.js library . D3.js is a set of tools for visualizing data. It includes modules for constructing geometric shapes, managing the loading, formatting and scaling of data, as well as mathematical functions. The authors of the material tell how to use it to build graphs of sound waves. An example of the application from the article can be found on GitHub Pages , and its code is in the official repository .

How to create music systems in JavaScript . Global Web Audio Guide. With his help, the author recreates the works of composers Stephen Reich and Brian Eno - " It's Gonna Rain " and " Discreet Music ", respectively. Following the guide is pretty simple (there are examples, diagrams, and screenshots) even if you are unfamiliar with the Web Audio API.

Musical instrument with Web Audio API . A record with JSConf, during which Steve Kinney, founder of the Turing School, shows how to build a synthesizer in a browser on the getUserMedia Web API and WebSockets. He also talks about using the Web Audio API to build a musical instrument on Arduino.

Introducing p5.js. p5.js is a visualization library that allows you to "draw using code." In the video, its developer, Lauren McCarthy, demonstrates with examples how to use this tool to create artistic elements and animations. If you want to evaluate the capabilities of p5.js yourself, then there is a special editor on the official website of the project.

Music from 8-bit games on the Web Audio API . This article is about how to recreate and visualize 8-bit music using the Web Audio API and Tone.js framework from Pac-Man, Super Mario Bros, Metroid, Kirby's Adventure and The Legend of Zelda. Listen to the resulting composition, see the code and, if desired, you can modify it on CodePen . To make it easier to understand the source, the author recommends that you first watch this video on YouTube - it explains how the sound "worked" in old computers.

How to create a dance track using the Web Audio API . Recording performance by Paul Adenot, an engineer at Mozilla who works on the Firefox browser and helps W3C complete the Web Audio API specification. Paul takes apart the main components of the techno track and shows how to recreate it on JS. It can be said that this is a live music programming session.

Additional reading - from our "World of Hi-Fi":