This is the best and hardest part about going vegan or vegetarian. I used to depend on convenience foods a lot. I didn't spend more than 15 to 20 minutes cooking a meal. Now, I cook. The food taste better and it's healthier and more filling, but it also takes more time. Sometimes, it's hard to resist falling back into my old habits.
Are you buying what is in season? Maybe the problem is your idea of what is "whole food." Fruits and vegetables are not more expensive, but if you're not used to using them in a wide enough variety, you may be limiting yourself. There are several fruits and vegetables like carrots that are extremely cheap and some that are expensive like pineapples. Check for sales and shop at local farmer's markets. It's not just about fruits and vegetables either; beans, nuts and everything that isn't animal based has to be included.
It's proven that vegetarian diets are more expensive, but if you look up different products and stuff it's still manageable. The thing is that meat or fish need to be replaced by something like tofu which often is more expensive then just a cheap cut of meat.
I'm a vegan and I've never spent less on food lol. It's FAR cheaper in my opinion. I think it's because you're looking at "products" and not at whole, real foods. Or maybe you're looking at a whole lot of both which of course will make it expensive.. but it's the processed foods that are doing it. They aren't necessary at all, though I know the appeal. Anyone can go vegan and not be much healthier at all for it, because the ingredients are awful and it's highly processed junk. I eat a plant based diet. Either way is fine, don't get me wrong, veganism is mostly about the animals.. just saying.