The yaw angle is most dependent on the magnetometer inputs, since the accelerometers don't change with orientation around this axis.
How fast is the angle drifting? Does it stop, slow-down or speed up? Can you post a data file that contains the accelerometer, gyroscope, magnetometer and angle outputs during the drift? Also, can you post a register dump for all configurable values?
NOTE: I am not going to be able to do much more with this for another week, as I must focus on some important family matters, but I will see if any colleagues are available to take a look at this & offer insights.