It's the pilot in command's job to operate the aircraft - not the controller's (see FAR 91.3(a) below).
It is the policy of the FAA to track aborted take offs and they often will ask why as part of that reporting. As others have already stated, there's nothing wrong with telling a controller to "Standby" and answer the question later. Can a controller be assumed to be acting as the Administrator (see 91.3(c))? If so, then that reply would be "a written report."
IMHO, the AOPA attorney who said you deviated from a clearance doesn't understand the proper and legal role of the pilot in command (I have personal experience with ATC regarding this authority - I was victorious).
Sec. 91.3 Responsibility and authority of the pilot in command.
(a) The pilot in command of an aircraft is directly responsible for, and is the final authority as to, the operation of that aircraft.
(b) In an in-flight emergency requiring immediate action, the pilot in command may deviate from any rule of this part to the extent required to meet that emergency.
(c) Each pilot in command who deviates from a rule under paragraph (b) of this section shall, upon the request of the Administrator, send a written report of that deviation to the Administrator.
para (b) and (c) of 91.3 only apply to aircraft in-flight. I would think definition of aborted takeoff plane never becomes airborne.
Regarding "administrator" very first definition in FAR 1.1
I've had ATC ask why I'm changing my destination. At first I thought they were just hinky about destinations around the DC Area but they seem to want to know why in general now.
Two years ago, DC area to OSH. IFR. Needles on the fuel gauges were going down faster than I expected. Asked to amend flight plan to land at a closer airport than I had filed. ATC DEMANDED to know the reason for the amendment. I thought an attitude like that from ATC would be enough to discourage someone from making a wise in-flight decision. Certainly it is enough to make someone reconsider filing IFR. Like user fees, anything that discourages pilots from taking advantage of the system will decrease overall safety.
I usually just tell them my wife has to use the bathroom. Never gets questioned.
I wouldn't let one controller having a bad day change my opinion. It's not like the pilot has to ask for permission to change his destination and provide a valid reason which may be rejected by ATC. I've changed destination dozens of times. Never had to answer to any demands. I got my job to do, controller has his job to do, there is some overlap, no need to make it an agonizing process.
I didn't say it would discourage ME. Going back to Bill's original post, I just found it annoying. After all, I had not deviated from a clearance, I was just asking to change my flight plan. If I did not need an IFR let down to VMC I would have just cancelled and left it at that.
I dunno. Maybe they are worried about hijacking. Those Skypigs are WMD waiting to be set loose, you know.
Almost all of the controllers I have encountered are professional and helpful. Only rarely is there one who encourages you to avoid participating in the system. For someone new to the system, however, one may be enough.