Adding some loggin in PropertiesView and TreeRow, I was able to see that we are doing 2 renders on PropertiesView and (nodes.length ✕ 2) renders on TreeRow.
This is slowing down the time it takes to get to a given item in the response (my example was a JSON response with an array of ~400 items).

Here's a profile recording of this: https://profiler.firefox.com/public/81224a09e1b813d47a0cb6e2675ba85ea7536255/stack-chart/?globalTrackOrder=0-1-2-3&hiddenGlobalTracks=1-2-3&implementation=js&range=3.9440_9.5215&thread=0&v=3

I also found out that this is an interaction that isn't tracked in DAMP, despite it being very common (my guess, but when I was working with APIs, this was my main interaction with DevTools).