Thanks for the info & replies so far! I'm still researching and this is all helpful.
I do have a state driver's license, and haven't done the medical yet (so no failures logged, or SI process started), so I guess it comes down to the reading of 14 CFR Part 61.303:
It's the subparagraph 4 that worries me -- I can say I believe I'm safe to fly under sport pilot rules, but do I run a risk of the FAA deciding otherwise, and what sort of enforcement action is possible? Eg, if they just ground me until I can pass a medical with a special issuance, that's fine -- even if it ends up being impossible to get the SI, I'd feel like I went through proper procedure. But if they fine me for every flight I've ever made that would be un-fun!A person using a U.S. driver's license to meet the requirements of this paragraph must -
(1) Comply with each restriction and limitation imposed by that person's U.S. driver's license and any judicial or administrative order applying to the operation of a motor vehicle;
(2) Have been found eligible for the issuance of at least a third-class airman medical certificate at the time of his or her most recent application (if the person has applied for a medical certificate);
(3) Not have had his or her most recently issued medical certificate (if the person has held a medical certificate) suspended or revoked or most recent Authorization for a Special Issuance of a Medical Certificate withdrawn; and
(4) Not know or have reason to know of any medical condition that would make that person unable to operate a light-sport aircraft in a safe manner.
I do know that since the LSA and Sport Pilot spec limitations and operational rules are more limited they're a bit laxer on the medical stuff, so I understand if it literally is a more lax entry point to flight, because there's less danger from someone's health taking a surprise turn for the worse. It's also good to figure out just what the rules are.
In a pinch, ultralights are also definitely a possibility for me to explore to get my 'fly a bit' experience, and remain a fallback position -- as well as something I might due to just "build my own" even if I can also fly big-kid planes!
Thanks again for advice so far; I'm going to do more research on the sport pilot + driver's license situation specifically to see if I can get a firmer picture of things!