Stealth is an obvious one. Especially for chasing spooky fish on the flats. Minimalist (eventually), think most start off carying too much stuff to begin. But you can pare that down over time to the bare minimum to have a succesful trip. Cost.....no gas, no insurance, no maintenance. Improved fishing ability, you have to focus on an area, you cannot just pull up anchor run to a new spot. You have to fish the area you are in, which allows you to learn more than you ever can in a boat. And chicks dig them.
Freedom. Its awesome to be free to take your time and paddle back into a hidden jewel. Stealth is not that important for Redfish in our area, since they swim up to the boat and have been known to run into the side of the boat. However, Gator trout are a lot spookier and making long cast in a stealthy boat is a huge plus in the pursuit of the elusive 30" prize.
As a flats fishing girl, I'd have to say stealth. I can creep up on super spooky bonefish and permit better than from a skiff.
Also, I think you begin to think more like your target species when fishing from a kayak. Your paddling energy is a limited resource so just like the fish, you get much more in tune with the direction of the current, speed of the current, tides, the moon, water temperature and bait fish location.
I notice and enjoy nature a lot more from my kayak than from a flats boat too. It's just more zen :)