If you did that you would become extremely unhealthy. Veganism has the caveat "as far as is practicable", and a fruitarian diet is not practicable for extended periods of time as you will become malnourished.
Fair enough. Swap "only fruit@ for "as much fruit as possible without becoming malnourished" if you would value your own wellbeing above another animals' (which I would as well).
I know a guy who was promoting frutarian lifestyle. He was a previous world champion kayaker and works in the outdoor industry. I saw him recently and he was still in the top 15-20 in the world in his even. I need to check with him if he's still doing it. He looks like a model, superfit and full of energy. He had an interesting blog on the amount of fruit he was having to eat, most of which he picked up for free or cheap at markets as it was approaching use by date.