According to the Canadian Aviation Regulations (CARs), the minimum altitude for VFR flight over a built-up area depends on the type of aircraft you're operating.
For Aeroplanes:
You must operate at an altitude from which, in the event of an emergency necessitating an immediate landing, it...
Great question! This is something that catches a lot of pilots off guard, especially when operating to/from remote stands or cargo areas.
The Short Answer:
You're right that centerline lighting isn't always the definitive answer. The definitive source is the airport's Low Visibility Operations...