Haven't had that experience because I haven't done much freelance, but I'm sure it's very annoying.

In freelance you are supposed to have freedom. That's pretty obvious.