My microphone only works on speaker

I sim swoped from two different service providers and then my microphone stop working and I had to used speaker when I recieve a call. I restarted my device an it worked again for a while an went back to not working. I restored my factory settings and restored my data and still it gives the same problem. Any one have an idee how to fix this as this is clearly a software problem?

Thanks in advance for the help!