This site was down for a while due to a bizarre WordPress design flaw, which coincided with a change in servers. This made it fairly difficult to diagnose the problem, but I figured it out and it’s back in its full unbridled glory!     Of the two months I’ve spent at UC Santa Barbara, I’ve spent a good chunk of it writing proposals. This one was for the NSF Graduate Research Fellowship. Instead of boring you with the details of a fellowship writing process, I’d like to point out the moiré pattern I inadvertently captured when I took a photo of a computer screen with a phone camera. I first stumbled upon the concept of a moiré pattern when i saw this beautiful moiré pattern wall clock on the internet about 6-7 years ago. It’s one of those things you keep noticing once you’re cognizant of it… whether it’s from layered screen doors or compressed youtube videos, the effect is caused by similar constraints [1]. However, when photographing a screen like this, the constraints of the moiré are governed by the resolving power of the camera sensor. This is what causes the red, green, and blue stripes – the camera has just the wrong gaps in the sensels (the individual sensors that record each pixel of a digital image) such that the RGB phases in and out of the white it’s all supposed to be. If we had an extremely high-resolution camera, this wouldn’t happen – the effect emerges due to the larger pixels in the low-res image. Interestingly enough, the idea of “sampling error” is one that is pronounced in a lot of different fields. Music files (MP3, etc) are an interesting “goldilocks” scenario. You want enough resolution that the music’s pretty, but small enough files so you can fit more songs onto your iPod. To accomplish this, some very clever people came up with a standard of digitizing music [2].

(wikipedia commons)

The data is converted from an analog (continuous) signal to a digital (discrete) signal using PCM, by means of an Analog Digital Converter (ADC). Since the data can only be stored into a computer as ones and zeroes, the amplitude of the wave is sampled thousands of times a second at a high resolution and these plotted points are recorded in binary. To turn them back into sound, the binary is converted back to amplitudes (DAC!), smoothed out a bit(capacitors!), and sent out to your headphones or speaker amplifiers(…cables!). This is the way music is recorded onto CDs (both .WAV and .AIFF are the same data, just with all the binary in a different order in the file).Well, how do we determine how often we sample the amplitude, and how many bits we allocate to the height? The first question is a lot more interesting, so let’s focus on that…

The answer is presented to us by Swiss mathematician Harry Nyquist in the form of the Nyquist Frequency. Simply put, the highest frequencies that can be encoded as PCM are (0.5  x  sampling frequency). We can set the highest frequencies to the upper range of human hearing – any higher of a sampling frequency would be out of hearing range and therefore unnecessary resolution. The upper range of human hearing is roughly 20kHz, so we must sample at 40kHz. CDs are standardized at 41kHz. There is very little excess resolution – CDs are designed to be (more or less) as precise in audio recreation as we can tell [3].

So how does this sound, and why do we care? Well, in the first photo, you saw the color white being separated out into its component parts (as displayed in RGB colorspace). In music, we hear a similar phenomenon. as you lower the resolution, you start to hear higher, abrasive notes that are amplified out of the music. This is more generally known of as a compression artifact. To hear this equivalent of moiré, I put together a short demo using a clip for a song I am currently writing. I gradually lower the resolution starting at about halfway through. You should start to hear some… distortion, some abrasive robotic crunchiness with odd ringing notes. This is what you DON’T wan’t to have on your music track! (unless you want to, of course, and its actually one of my favorite electronic music effects to use ).

As you can hear, it gets pretty bad near the end. Remember this is only changing the sampling resolution of the x axis, but not the y axis. Changing the resolution of the y axis is far less interesting, and just sounds a bit fuzzier. Just know that every 41,000th of a second, 16 bits of data are played from a CD. We’re not going to bother exploring this further(See [1]).

However, there are ways to offset this effect, to some extent. Turns out that by adding a layer of noise to a music file, we can drown out these patterns(?).[5] This process is called dithering, and can take out compression artifacts in both music and photos. In addition, even more clever people have thought of ways to compress images and audio further – using less memory while retaining resolution. MP3’s and JPGs are both great examples of this, and there are even formats that call themselves lossless (.alac and .flac for music files, .zip can even be thought of one as well!), which let you store the same data as smaller files! Now, as long as the data isn’t Kolmogorov-random, it can by definition be stored losslessly, but these algorithms that have been standardized to certain data types are far superior. Making a .flac version of a music file can compress it far more than a .zip, because the algorithm is designed to find patterns in music and exploit them for compression [6]. If you got this far, congratulations! I hope the above was somewhat coherent, or at least interesting. To tie this all back to my research [5]… part of my lab’s work focuses on doing high-frequency pH samples to understand the finer-scale dynamics of ocean conditions. By sampling every 20 minutes (or 0.0008333 Hz, to keep things consistent), we’re trying to avoid the same kinds of things – compression artifacts. If the compression artifacts are just noise, it’s no big deal, but when they form artificial patterns… it’s a lot more problematic. It’s advisable to be on the safe side and sample as often as possible, because there’s really no way to regain resolution posthoc.

-U

PS I know some of the footnotes got unruly and out of hand. [5] They have been drinking some bourbon.[2]

PPS: addendums and corrections added. 

—————–

[1] We normally think of low-res photos as just being “fuzzy”, “unresolved”, “grainy”. However, this is particularly interesting because by imparting a pattern of fuzzy onto a pattern of lines, we get a superimposed pattern. That’s more or less the focus of this whole thing. Wibbly, wobbly, timey wimey… stuff. 

[2] I hope they aren’t too confusing {4].

[3]In reality, this thought process is flawed. Nyquist frequency in the digital domain can only be expressed as a square wave (think 1-0-1-0-1-0…). As you start to get to higher frequencies, you start losing the resolution in a wavelength (since higher frequency = shorter wavelength = fewer samples). I guess the idea is that it really doesn’t matter if you play a square wave or a sine wave at 20.5 kHz… or does it? If you do a Fourier transform of said square wave you’ll find that it’s made up of members of the odd harmonic series in an infinite sequence. Presumably, all of these other than the fundamental would be out of hearing range, but if they didn’t quite line up with the sampling rate, it could very easily cause distortion if you continued converting between analog and digital. But with 8 samples per wavelength at 5 kHz, which is easily getting you into higher normal hearing range, I’d expect a pretty defined difference. Some CDs have been rereleased at higher bit rates and sample rates (24bit, 48kHz compared to 16 bit, 41kHz) [5] This is where my understanding clearly fades out.

[4] I’m also having a fun time doing these footnotes and see why people like David Foster Wallace and Terry Pratchett insist on it [2].

[5] Just roll with it, alright?

[6] There are some audiophiles that consider .aiff far superior to lossless formats, even though the bits are fundamentally the same. I think it’s the same magic that governs gold-plated HDMI cables and velostat hats.