This restriction reinforces the notion of the imperial family as living descendents and representatives of the gods despite the emperor's renunciation of divinity announced as one of the terms of surrender at the end of World War II (when some people literally fainted upon hearing the emperor's voice on the radio for the first time).
387133
Some people fainted when they heard the emperor's voice.
387134
The emperor was hated by all.