In earlier times rules were called rules, but just because we don't call them rules now doesn't mean they don't exist. Each songwriter/player in pop music now develops their own set of variances from the prevailing rules, which is exactly what Haydn, Mozart and Beethoven did with the rules of their time.
The great paradox of art is that the more you engage with the rules instead of ignoring them, the better the odds of creating something worthwhile. One of the most striking examples is The Art of Fugue by J. S. Bach that is a compendium of the techniques of counterpoint--techniques are just rules viewed from a different angle. All art is about the imposition of order on chaos. Another paradox is that creation demands you somehow find a new way of handling the rules.
Just after writing this post I was delighted to run across the following passage in Knud Jeppesen's classic book on counterpoint:
In the course of the fifteenth century the polyphonic art develops and becomes established, and as early as the middle of the century one can speak of a music that is in the modern sense intellectually mastered, and in which the non-essential--the most dangerous, but not always equally well-known enemy of all art--is forcibly put on the defensive. [Counterpoint: The Polyphonic Vocal Style of the Sixteenth Century, p. 8, my emphasis]It is the existence of rules, in the following or bending, that gives coherence to music. The purpose of rules is to exclude the non-essential, as Jeppesen says. Music without coherence is just bad music.
This isn't really bad, of course. It is just sort of a mis-judgment. Toru Takemitsu is a very fine Japanese composer but his jazzy arrangements of popular songs do fall rather flat, I think. I chose this because of the constant added-note harmonies that just seem to add harmonic flab...
For comparison, have a listen to this piece in pure Takemitsu style. No mashing together of different musics. I don't know if I have found the right examples--it was surprisingly difficult. But I hope you hear the point.