Modern art is really quite broad. Personally I like to shit on a lot of abstract art but I have to admit there's some beautiful and mighty interesting abstract art as well. I never got what makes Victor boogie woogie by Mondriaan so great though, and I'm not a fan of Picasso either. So yes, it is really mostly subjective.
About the 'he could paint' argument: that's how I looked at modern abstract and impressionistic art as a kid: shouldn't there be a line of improvement in paintings skills over the centuries? Why paint like a retarded kid and place a meaning onto it nobody else can see without it getting explained? Well, the thing is the art of painting realistically has been pretty much done to perfection. Most modern art is made with different intent. It isn't ALL about skill. I don't like the majority of modern art neither, but I acknowledge that it is often due to lack of understanding. I found when the meaning of an at first shitty abstract piece is explained I can sometimes even dig it!
There's also a large chance the pretentious artists that know how to get their work under the attention of galleries and the general public may be affecting OP's view on modern art as a whole (without taking notice of it all).