Definitely a concern/worry.
So far I have been using the mall. The part of the lot near the old Sears is also deserted, so we aren't getting in the way much.
Gymkhana is actually a lot like practice drills. Enough G1 drivers training there, and since we don't do wheelies or burnouts, the security guards have actually waved as they walk by.
Things start to change when it becomes a group larger than 3-4 people especially in a public space.
Most often than not, if they see 1 or 2 people practicing in a safe manner, security guards (malls especially) let you do what you want. Once your numbers go up, it becomes an 'event' and you're usually told to move on.
GO lots are usually my go-to on the weekends for MotoGK (which is when I would normally go for a ride anyway). No pedestrian traffic, and typically very few parked cars.
FWIW, 99% of the interactions I've had with people were positive. As long as you're respectful, know you're technically not allowed to be there, and leave when requested, you should be fine.
The 1% was a janitor going in on the weekend that wanted to flex his muscles in very broken English. I didn't argue and left shortly after we spoke, but the guy felt the need to yell in my face because "the truck yard was not my property" as I was packing up.
I have had run-ins with police twice. One was an elementary school parking lot, and one was a high school parking lot on the weekend in Summer. They simply asked what I was doing, I explained, and they left with a "Ride safe..."