For a planning enforcement team, judging when a "change of use" is required for a home-related business or "operation" is not an exact science and can vary depending on what the operation is and how large the house/plot is. With things like cats and dogs and "home boarding", it is considered that for your average house the magic number is six. Above that and it's likely a change of use is required to accommodate the home boarding business.
I have to say, I've never seen it used for cars before in a change of use scenario. The suggestion here is that your turnover of cars (x per week/month/year?) combined with the number on the driveway at any given time (three or four?) means that you are effectively running a car dealership rather than enjoying a personal hobby, and that a change of use to cover this operation is required.
It's an incredibly risky precedent. Committing to a set number and "frequency of turnover" sets a precedent that could affect a lot of people, especially if that set number is low, or the turnover considered to be "dealership standard" is not that intense. I'd say that if they served an enforcement notice on you, and you appealed, it wouldn't stand much of a chance of being upheld. Especially if you can show that you are not turning cars over frequently or making a profit! The decision to uphold the notice would effectively ban anyone in your district from owning more than two cars, or from buying/selling them within a certain time frame.
The only other time I've seen cars involved in a planning complaint was where there were a number of cars seemingly abandoned or "not in use" on a driveway/front garden. In this situation the council can serve a Section 215 notice which is effectively saying to you that you have untidy land and you need to tidy it up. Notwithstanding the condition or appearance of the chod, it wouldn't seem appropriate in your instance.
It does sound like in this case you've fallen foul of the ultimate nosey neighbours from hell, who don't like your taste in motors or the fact you dabble in them as a hobby. Look out the window. They're the ones with the A1 TFSi and the 316d on the driveway.
I'm afraid I cannot help in terms of ridding you of your motors, but I thought my work-hat may provide some extra helpful info in some way!