Category Archives: JavaScript

the beauty of numbers..

I just did a little revival of my old WebGL fractal stuff. The two below screens use the latest version of my Commodore C64 emulator to play two fitting music creations by Markus Klein.

ricemilk

clubhouse

vu meters?

Just a little experiment for how to synchronize visualization of additional data streams with the the playback of WebAudio music: The music samples are generated on the fly using a ScriptProcessor emulating some legacy AtariST. In addition to the stereo output the respective ScriptProcessor also generates three streams containing “playback volume” for the AtariST’s internal soundchip voices:

vu2

just for fun a more psychedelic WebGL based rendering of the same data (the WebGL here combines an orbit trap fractal with an inverse distortion, and the “music volume” is used to parameterize the rendering):

vu1

 

unaligned “packed structs”…

are certainly not a good idea if a program is supposed to be portable. Unfortunately that is exactly what ZXTune is using to parse the different binary music files.

“One of the rules of packed structs is that you have to access them directly through the packed struct type. In other words, you can’t, in general, take the address of a packed field and then access the field through that pointer, because the pointer type won’t reflect the packed attribute.” (sunfishcode)

Unfortunately ZXTune used boost::array instances within the various packed structs.. Problem: when methods are invoked on boost::array (or std::array, etc). The ‘this’ argument to the boost::array functions may be misaligned, but the boost::array functions themselves don’t know this.

On CPUs which don’t mind unaligned memory access you may get away within without realizing that there is a problem.. and in this case it was my attempt to cross-compile the program using Emscripten that revealed the issue. Not wanting to rewrite too much of the foreign code I opted for a quickfix: replacing the boost::array with a built-in array fixed the immediate problem…

Naturally a clean implementation should better refrain from depending on unaligned memory access at all… not all the CPUs are as forgiving as Emscripten.

(click on the below image for a live demo).

screenshot

AdLibido – the wonders of early PC music ;-)

It was back “in the old days” and I remember my relief when some day I found out that all PCs were not necessarily mute: Thanks to some “walking frame” called “AdLib” they could actually make sounds… and a bit later things became pretty neat with the rise of Sound Blaster…

AdPlug plays sound data, originally created for the AdLib (OPL2) and Sound Blaster (Dual OPL2/OPL3) audio boards, directly from its original format on top of an emulator.

My latest bit of Xmas tinkering is a HTML5/WebAudio version of AdPlug (Thanks to Simon Peter and the other authors of AdPlug.). For a live demo click on the below image..

HTML5/WebAudio version of AdPlug

68000, Paula & Co.

screenYet another bit of home computer music nostalgia. This time it is the Commodore Amiga that is emulated using UADE. The original Unix Amiga Delitracker Emulator is based on a two process design where the core Amiga emulator engine and the music player’s frontend are separate processes that communicate via IPC. Using a regular Amiga emulator then may be tricky because depending on the song it expects to synchonously load additional data files. Obviously this is on a collision course with the concepts available for an HTML5/JavaScript page.

Bringing this one to the web therefore required some redesign of the original UADE code base. Once that had been done Emscripten again did a splendid job translating the C code into JavaScript and linking it to the manually written JavaScript callbacks (see live demo here).

This experiment once again confirmed my ealier observations that the debugger support built into today’s web browsers is utterly useless (but for the most trivial scenarios). So this not only was a travel back in time with regard to home computer music but also with regard to the modern developement tools that I had gotten used to: bye bye IDE – welcome back debug/trace output.

some real JavaScript IDE – anyone?

Debugging JavaScript can really be so much fun.. first there are Firefox’s “developer tools”. These are so slow that they bring back memories of the early Eclipse Java IDE: By the time the bloody tool shows the breakpoint you have long forgotten what you wanted to look for in the first place. From there it is then a steep downhill trip: When stepping through a function in the debugger the focus will suddenly disappear and the only way to reanimate the tool is to actually remove the previous breakpoint.. LOL it’s version 31 and Firefox still has no usable developer tools.
For all those thinking ‘ahhh but Chrome is so much better’ – think again: While it is true that Chrome’s developer tools are real quick there is a catch. Or as a colleague once said: “if the result does not have to be correct my code can be real quick!”. Here is an example of a situation that you may encounter while debugging in Chrome (this is actual – non tampered with – debugger output):

WhyJavaScriptSucks

The debugger here has just stepped over a line of code that sets some variable to 0. On the very next line the debugger tells you that this very variable is NOT 0.. Another fun Chrome surprise is when you find that Chrome’s garbage collector will actually trash ‘event handler’ functions that are still in use – unless you separately anchor them to some dedicated variable. 

I am so glad that I never make any mistakes as a matter of principle 🙂 So my latest bit of music playback programming could not be stopped by foul developer tools:

screenshot 
And what was this experiment about? xmp is a module player that plays over 90 mainstream and obscure module formats from Amiga, Atari, Acorn, Apple IIgs and PC, including Protracker (MOD), Scream Tracker 3 (S3M), Fast Tracker II (XM) and Impulse Tracker (IT) files. This functionality is now available on my little web page (click on the image above) 🙂

 

(You’ll find the source code here: http://sourceforge.net/projects/webxmp/)

Fractal Flames

I just completed my latest experiment 🙂 It is a revamped Web version of Orion Sky Lawlor’s “GPU-Accelerated Rendering of Unbounded Nonlinear Iterated Function System Fixed Points” (see here).
screenshot
For good measure I have added a bit of music interaction 🙂 I suggest you have a look at the live demo here (Chrome recommended). A Youtube recording can be found here.

The music I selected above is by Dexter Britain and licensed under a Creative Commons license. (see www.dexterbritain.co.uk)


Programming background information:

  • Starting from Dr. Lawlors opengl/C++/Windows program the main step was to get rid of any opengl legacy fixed pipeline rendering and replace it with a programmable pipeline WEBGL equivalent. Then some WEBGL limitations still had to be dealt with, e.g. adding missing WEBGL shader functions, replacement of unsupported for-loops in shaders or unsupported texture modes, etc. Luckily this process proved easier than expected. The resulting WEBGL/JavaScipt program works quite fluidly in Chome (on my GXT460). Firefox’s WEBGL implementation also works for the most part – but it occasionally it crashes for no good reason and it is rather slow in compiling the shaders.
  • Some surprise obstacles came with the attempt to implement the music playback and hook it up WebAudio’s AnalizerNode infrastructure: The first attempt to use some XMLHttpRequest based loading with AudioContext.decodeAudioData() processing proved a total failure, because Firefox’s implementation (versions 31 -33) just fails to decode the (supposedly “invalid”) mp3 data. Also it became obvious that the upfront decoding of the complete mp3 file (in Chrome) adds an unacceptable delay to the music startup. Interestingly playback of the same mp3 file works flawlessly in Chrome as well as in Firefox when using the HTMLAudioElement. So that’s what I am using now. Unfortunately that API is not implemented consistently in the different browsers and particularily the event handling in Firefox leaves a lot to be desired. But even Chrome fails to properly “loop” a song but instead goes into some corrupt state at the end of some (especially longer) songs. Once again browser specific tinkering is called for…
  • The page uses multiple AnaylzerNodes with low-/band- and high-pass filters to detect certain music patterns that are then reflected in the graphics. Unfortunately Firefox again proved a difficult customer: Connecting FilterNodes to the HTMLAudioElement here sporadically leads to corrupted music playback. Also the Firefox FilterNodes produce completely different results than the Chrome ones.

WEBGL and fractals

Did I mention that I love fractals? In fact I do and that’s why I created the below intro:


turtles

I am using orbit trap logic that I implemented in a WEBGL fragment shader.   The add-on text scrollers/messages sport an exploding-pixels effect which is done using plain Canvas pixel operations. For good measure I added my WebAudio based SID chiptune playback (see my previous posts).

I guess that next time I’d better leave all the graphics to the GPU .. because my CPU is just getting too old. In case you are using a recent browser and if you have some sort of GPU in your computer, you might try to see the online version by clicking on the above image.

 

“hello world 3D” version 2..

On my second trip to the world of THREE.js and WEBGL I tried out yet another bunch of “new” things:

  • use of THREE.js EffectComposers and Shaders to create “glow” effects, etc
  • creating lightweight 3D text rendering by rendering a 2D font into a texture and by creating a geometry that then represents each character as two simple textured triangles
  • create THREE.ShaderMaterials (i.e. WEBGL vertex and fragement shaders) to then apply visual effects to the above 3D text (e.g. rolling the characters or exploding the text-page, etc)

 

Then I came across Felix Woitzel’s fascinating “Travelling Wavefronts” demo and I had to understand how its WEBGL shader logic is done 🙂 After some tinkering I ended up with a slightly cleaned up version of my own with some added visual effect features:

egaux2

After throwing all of the above together and and adding a bit of WebAudio based SID chiptune playback (see my previous experiment), I ended up with a little Web intro 🙂 Those with a reasonably fast GPU may see it in action here  (due to my old CPU I really like to use as much of the GPU’s power as I can..).

egaux3

WebAudio ahoy..

 

At last Firefox and Chrome seem to be making steps forward with regard to audio 🙂 .. giving me an opportunity to migrate my Flash based music player (see previous experiment) to an HTML5-only implementation.

jsid

Using Emscripten I compiled my C code into JavaScript and hooked it up to the new experimental WebAudio APIs (ScriptProcessor, etc)… what can I say.. it works (see here).

PS: Unfortunately Chrome and Firefox still don’t seem to be on the same page with regard to correct chaining of Nodes.. still some work to be done.