IFR vs. VFR has nothing to do with it. IFR doesn't change the mode C requirement.
You need mode C:
In class A, B, C airspace.
Inside the mode C veil (up to 10,000 feet).
Under class B and C tiers and above the ceiling
Above 10,000 (and more than 2500 AGL) in the lower 48
Nowhere in 91.215 does it give you an out for VFR-only. You get a break on SOME (but not all) of these requirements if you have no engine-driven electrical system.