Hi hmani,
I discovered my problem with the WGAIN computation and now most of the calibration seems to be working. I was doing some manipulation to keep things in Integer format in the code (bit shifts and the like) and I had introduced a problem there. I converted most of the code to double precision for the time being, just to make it easier to read and debug. I don't convert to integer until I actually need to read or write a register.
Thank you for your offer of help. I found the problem when I went back to answer your questions.
I do have one remaining measurement that I can't seem to get right - VRMS. I am not getting a linear response, so i don't believe that VRMSOS or the LSB factor is the answer here. I am just keeping VRMSOS "0" for the time being - please tell me if that is not a good choice.
Also kindly tell me if I should post this under another heading since the topic has changed a bit.
After calibration, my VRMS readings are not much different than with no calibration.
At 200mA and 120.4V measured on an accurate meter, I am getting 553000 from the VRMS register which I am converting into 120.9V
At 9.8A and 117.3V measured on an accurate meter, I am getting 553550 from the VRMS register which I am converting into 120.9V.
The all of the values above are subject to constant slight fluctuations because the line in is not perfect, but they represent averages. As you can see, I could use offsets to correct one measurement, but the other end would become that much worse.
Where would you suspect I still have work to do with my calibration procedure in order to correct this problem? The Watts, VA and Current are all reading pretty accurately now - VRMS is the only problem.
Thanks!