It takes 5 seconds to turn down the volume because pressing the button just opens the volume slider—in a different position— and doesn't actually turn down the volume.

Just as a heads up – you don't need to do this. You can just rest your finger on the volume icon and, without lifting it, move left or right to adjust the volume. Same with the brightness icon.