Collision and simultaneity in special relativity

An almost rigid rectangle with length $D$ can rotate around its center. Assume that rectangle is horizontal and does not rotate before collision. According to observer $S$ two balls collide at the same time at two ends points of rectangle $R$ & $L$, in $t=0$ and before collision two balls have same speed $u$ in $y$ axis therefore there is no rotation for rectangle at all after clash. For these collisions we can assign two events in spacetime, that is $E_1=(-ct,r)=(0,-D/2,0,0)$ and $E_2=(0,+D/2,0,0)$. Note that $S$ is at rest relative to rectangle.

Now let's consider another observer $S'$ who moves at speed $-v$ relative to rectangle in $x$ axis. By using Lorentz transformation we can see that $E_1'=(-\gamma v D/2c,\gamma (-D/2+ vt),0,0)$ and $E_2'=(\gamma v D/2c,\gamma (+D/2+ vt),0,0)$. Because two events $E_1'$ and $E_2'$ are not simultaneous in this frame ($\Delta t'=\gamma vD/c^2$) it's logical to deduce that the rectangle will rotate after first collision. While it's not possible for obvious reasons (one can put a bomb under rectangle for example). Where am i mistaken? Does system wait for a signal that is coming from two end points to the center?

PS: I am fully aware that 100% rigid bodies are not possible in SR, by almost rigid i meant that we have an elastic collision and rectangle tends to rotates than bends around it's center.