I think there might be two solutions for your nice puppet project:
The main cause of trouble is the little servo, it generates a lot of noise, mainly on the 5 Volt power supply. So it works when you separate it from other noise sensitive bits such as the MP3 player and synth speaker. (originally an idea from @chris101)
So first you could try to use two separate power supplies:
Battery + P1 Power + i6dimmer +011 servo
Battery + P1 Power + Synth speaker + MP3 player.
Then if this is not what you want you could build a noise filter with the protobit and place it just in front of the synth speaker.
This filter is a so called low pass filter with a resistor and a capacitor, it eliminates the noises on the 5 Volt connection.
The capacitor is connected on the output side between GND and VCC, the resitor sits between input VCC and output VCC. The most right connector on the front must be removed.
It works well also to eliminate the noise...
Hope this helps,
See this original post from @Ben:
You can find out more background details about the noisy servo here: