Interesting and deep analysis, folks! I had problems with noise in littleBits audio circuits before. Turns out, it was the switching power supply (adapter) that was causing the noise. I started using a non-switching supply (an old Yamaha PA-3b) and the noise went away. I use the P1 power bit.
Even with clean power, my design is noisy. I tried using the same design with voice samples and never could tune the buzz/noise away.
So, I've got a better design using a Microchips MCP4921 DAC via the SPI pins on the Arduino module. The good news is that the design is less sensitive to noise. The bad news is that the MCP4921 solution requires a few more low-cost components and you must solder six header pins to the Arduino module.
Here are links at my web site that should help you out:
I eventually completed the SPI DAC project, but haven't had a chance to post the project to the littleBits site.
All the best -- pj