It is strange that it works fine without the encoder but there are problems when the encoder is used.
What encoder is it?
No support for Jumbo frames. UDP will pass fine (although it is encapsulated in TCP across the tunnel).
You can adjust for jitter - see the latency cut off in this post: