We seem to be leaking mach ports over time (see bug 1755701) and it's not clear what's causing it.
As a tractable sub-investigation, we should check whether opening and closing a window leaks things that we would expect to not be leaked. I'm especially thinking of Objective C objects related to NSWindow / NSEvent / CALayer that we might not be cleaning up properly. Those objects then in turn could be holding on to mach ports.

Is there an easy way to check how many open ports we have, and if so, would it make sense to check it before and after every automated tests to assert the number is the same?

There used to be an easy way, but it seems it was broken on macOS 12: sudo lsmp -p <pid>. In the past, this command printed a nice list of open ports for that process. But on macOS 12 this command now prints gigabytes of useless output. (I've filed this as FB9957317.)
So we might only be able to debug this reliably on macOS 11.

As for Objective C object leaks, I think Instruments has a tool for that but I have never used it successfully.

I don't think we'll have something we can reliably check between tests.

The severity field is not set for this bug.
:spohl, could you have a look please?

For more information, please visit auto_nag documentation.