Nah wouldn't have fried it. I didn't feel my new ones were much better till I had both switched out, and played with the radios equalizer. Then they came alive.
I did have one wired backwards as well though, and it didn't blow it.
Or maybe the speakers you got aren't much better than stock...