ということを今まで知らずに使ってた．User Interface - OpenCV v2.4.2 documentation#waitKey によると：

This function is the only method in HighGUI that can fetch and handle events, so it needs to be called periodically for normal event processing ...