How to re-join a router (cc2531) to a coordinator (cc2531)

I used the 20190619 version of Koenkk’s souce-routing coordinator binary on a cc2531 usb dongle, and the router binary on another cc2531 USB dongle. After I first flashed it, the router cc2531 joined the coordinator just fine.

Now, I’m trying to get the router to join again, and it seems like I’m just suppose to press the button farthest away from usb port. But it doesn’t join. When I press the button, the green light goes from green to flashing red (after a pause), and I see no join messages on Zigbee2MQTT. I’m able to join my other devices, however, so I know the coordinator is really accepting joins (and also permit_join is true). Pushing that button again make it go back to green always on.

Am I am not putting the router into join mode correctly? Or am I suppose to reflash the router every time I need to re-join?