Hmm, I think I should have used another word.

I was riffing off of "guilty pleasure" -- I'm torn between enjoying the series, and being frustrated by issues I have with it; lately the balance has tilted more towards frustration than enjoyment, but "guilt" on its own implies that there's something wrong entirely with watching/enjoying the show, which was not what I meant at all!