It's director James Burrows. That man has directed so many legendary sitcoms over the decades but I think Will and Grace is the only show where he directed every episode. I think I read that he enjoyed working with that cast so much that he stayed instead of moving on like he normally does.
You know that they are close friends and very comfortable with each other. As Karen went inside the top and Grace was like whatever.... until they laughed