So, lately, I've been working on developing the I2C messaging protocol library for the PIC. I got a development board with a 32 bit PIC and an EEPROM from Atmel.

As a habit, I developed a minimal amount of framework - a skeleton, perfectly in line with the protocol and the hardware requirements of the bus. To verify, that my imagination and reality were in sync, I probed the bus with the saleae logic analyzer. But to the disappointment, I saw something weird going on the bus:

This was just terrible. There weren't sufficient clock pulses (9, to be precise) on SCL, the data wasn't authentic on SDA, the frequency had an error of 17%, and the duty cycle was not 50%. And the worst part - this wasn't consistent. The pictures were changing with each run - an unmistakable signature of noisy environment.

Now, before plunging into my own development, I had tried the firmware from Microchip - and that was working perfectly okay at 8Mhz frequency. Although, I was trying to develop with PLL on, at 80 MHz, it didn't seem to be a problem - since the datasheet mentioned the usage of frequencies above 40Mhz, which was not possible, without using PLL.

So, the problem seemed like the PLL - with it being ON, there seemed to be interference; and without it; at raw crystal speed of 8Mhz, the data throughput was almost 100%.

With this I spent full 2 weeks, trying to figure out the workaround - contacted Microchip support, asked for sample sequencing code, did lot of signal analyzing to identify patterns, if any. The random spikes, on SCL lines, every now and then was just annoying and painful.

Nonetheless, at the end of the week, I realized, I had spent way too much of time on this - I decided to pull myself out, and look at the picture from the distance. I thought of isolating the PIC from the bus and see what's going on, on the pins - just to eliminate the possible causes of interference from other devices on the bus - pull-ups and the EEPROM. Since this development board was multi-layered, and traces hidden below a laminate layer, the only option I was left was to was to physically force-out and lift the pins off the board. SMD components strain my eyes, especially when the pins are in middle of the array of 25 other pins. But, fortunately, a microscope came in handy at this point and the operation was successful. I then re-soldered two thin wires at the pulled-up pins, and fetched them to a safer distance. I connected two external pull-ups on these lines and hooked them to the logic analyzer.

The result was pleasurably unexpected . The protocol was perfect! The noise source was hiding somewhere on that bus on the board! The first thing that came to my mind was to see the capacitance between the lines, and it was of a fatal value of more than 100 microFarads! I did not know what was causing this high capacitance (poor board layout? foreign particle somewhere in between the tracks? something else?), but I immediately isolated the whole bus from the board. The protocol seemed to be riding bus perfectly now.

But, the thing that bothered me was, the fact that the protocol was able to maintain viability with 8Mhz clock and PLL on?; you see, the output on the I2C bus clock line remains 100kHz irrespective of its source input. Somehow, in this situation, it seemed apparent that, the switching on PLL caused some kind of signal degeneration, which in-turn reacted adversely with the bus capacitance to produce those bus lags and waveform deterioration. I tried to see the difference between the waveforms under both scenarios; but couldn't catch anything significant.

I hope it doesn't remain a mystery unsolved.

Update: It indeed turned out to be a manufacturing defect on the board. Few months down the line, I purchased another piece of the same development board and it worked perfectly for the above use case. Out of curiosity, I pushed down the pins and resoldered them on the board exhibiting the problem. The noise returned back. I sent this observation back to Microchip who gladly refunded and replaced the board.

Related but unrelated:

• You need to be careful on road if the problems of electrical circuits grab too much of your attention: