In my experience, I find people abuse the concept of MVP - especially those in the marketing / sales department. Slapping together a product and shoving it out the door is not a good approach IMO.
With that said, you don't want to be trapped in Scope Creep Hell either, but that is pretty easy to avoid if you are disciplined.
I consider a Minimal Viable Product to be one that can hold its own in the market segment (has comparable features and **is stable**), but has at least one feature that separates it from its competition (otherwise, why would they choose your product?)
Once you have accomplished this, then you are ready to ship. All the other great ideas for your product can be developed in parallel with your growth strategy.
I'm a bit of a perfectionist which is bad for a startup. But when I stop making everything perfect, I ship faster and do more things. I find that failing fast, acting fast and fixing fast is better than being a perfectionist.
I think at least build a stable MVP with some core features. Then, you can slowly build other stuff and adding more features.