newtonian mechanics - Ball flying towards me - Newton's third law is violated?

I was trying to answer the question of the flying ball here on the basis of Newton's third law and momentum conservation. Here is what I have tried.

Lets take a ball mass of $m_1$ (index 1 is the ball) hits the man with mass $m_2$ (index 2 is the man). The velocity of the ball is $u_1$. Then the force on the man is $$ m_1(v_1-u_1)/t = -m_2(v_2-u_2)/t$$

Now the second case, man hitting ball at rest at the same speed of that of the ball in case 1. i.e, $u_1$. The force on the ball is $$ m_2(v_2-u_2)/t = -m_1(v_1-u_1)/t$$

Since $u_1 = u_2$, the equations do not give equal forces on an object.

So that the pain should be different, shouldn't it ?