Safety for Whom? Refusing the Right Subset of a Topic, Not the Whole Topic