So, I decided to give it one last try and searched for more information about how Arduino writes the PWM signal. (Leonardo's datasheet, page 133: http://www.atmel.com/images/atmel-7766-8-bit-avr-atmega16u4-32u4_datasheet.pdf)
If you look in Arduino's source code (file "wiring.c") - by default the D5 and D9 ports use "Phase Correct PWM" (counting from 0x00 to 0xFF), but there are more PWM options there.
Most of them produced very distorted sounds, but "Fast PWM" (from 0x00 to 0xFF) seems to do the trick! So before any calls to analogWrite(), we need to change from "Phase Correct PWM" to "Fast PWM" like this:
#define OSCILLATOR_PORT 5
bitSet(TCCR3B, WGM32); // <---- Switch the PWM mode used by Arduino.
After switching to that mode the buzzing stopped immediately. I don't know exactly why it works, but... it works!