It's not just you, and it's not just the show.
In the comic books, red kryptonite has a different effect on Clark every time he is exposed to it. He never knows what is going to happen to him, but he knows that it will almost inevitably be something bad.
The show was just following the comic books' lead.
Joy,
Lynn
