Slight means of little consequence, barely perceptible or scanty.

If a person is described as slight they’ll be slender or frail looking. If you refer to someone slightingly you’ll be treating them as though they’re insignificant. I think that’s more than slightly rude.

Historically castles were slighted to make them useless for military purposes. This kind of damage can be seen in this photo of Kenilworth.