A threat to your property perhaps, a threat to your safety, not necessarily. That threshold would be breached if the intruder sees any of the occupants and directs any action towards them, other than talking.
If a drunk guy stumbles into your home, sees you and then asks where Joe is, then shooting him would be murder. If a drunk guy stumbles into your home, sees you and then walks up to you thinking you're Joe, then shooting him would be self-defense IMO.
And defending your property with lethal force is not about justice, it's about catharsis. So if it lands you in jail then so much the better IMO.