Andreas, do you mind share what you plan to implement for telemetry? You said relative length that we spend in the validator compared to the OpaqueResponseBlocker, is this what you plan to add for telemetry?

So it turns out that that doesn't really give us something in the end.

Instead I'm adding only ORB_JAVSCRIPT_VALIDATION_OVERSHOOT_MS so far. This measures the from when we JSValidatorParent::OnStopRequest is called until the callback passed to JSValidatorParent::IsOpaqueResponseAllowed .

Sounds good!

Comment on attachment 9308015 [details]
Bug 1804638 - Add telemetry probes to measure ORB JavaScript performance. r=sefeng!

1. What questions will you answer with this data?
   What the distribution of responses needing JavaScript validation in the Opaque Response Blocking (ORB) safelist check.
   What the overshoot is for JavaScript validation.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:
   Verify the performance level of ORB JavaScript validation to ensure that it doesn't degrade performance. This is essential to remove certain performance degrading measure we've had to take in the past.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?
   Profile markers will also be introduced, but they will only be able to be used for investigation not full scope gauging of the validation.

4. Can current instrumentation answer these questions?
   No.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.
   ORB_JAVSCRIPT_VALIDATION_OVERSHOOT_MS will measure the overshoot introduced when validating than an opaque response is JavaScript, but not JSON. It will be keyed on what it validates as: "javascript", "json", "other", and "failure".

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
   This collection is documented in its definitions files Histograms.json

7. How long will this data be collected? Choose one of the following:
   I want to permanently monitor this data, until the ORB feature has shipped. (farre)

8. What populations will you measure?
   All

9. If this data collection is default on, what is the opt-out mechanism for users?
   Ordinary telemetry opt-in/out.

10. Please provide a general description of how you will analyze this data.
    Measurement Dashboard

11. Where do you intend to share the results of your analysis?
    DOM Core team internal generally.

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection? No

It fits in better with Data Steward process if the review requests are text attachments.

Comment on attachment 9308584 [details]
data collection review request

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :afarre is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

Result: datareview+

The planned changes are:

• ORB_JAVSCRIPT_VALIDATION_OVERSHOOT_MS gets renamed to ORB_JAVASCRIPT_VALIDATION_MS
• Add ORB_RECEIVE_DATA_FOR_VALIDATION_MS to measure the time to get data for validation
• Add ORB_JAVASCRIPT_VALIDATION_COUNT and ORB_TOTAL_COUNT to make it possible to determine the frequency of needed validations.

:chutten, sorry for re-requesting a data review, but I added some probes and changed some names around. Hope that it's not too big a hassle.

Comment on attachment 9311762 [details]
data-review-2.txt

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :farre is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

Result: datareview+

Backed out for causing build bustage in netwerk/protocol/http/OpaqueResponseUtils.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/c881577e2dc9c274994b214d8a3bf11a2df338b2

Push with failures

Failure log: /builds/worker/checkouts/gecko/netwerk/protocol/http/OpaqueResponseUtils.cpp:385:3: error: control reaches end of non-void function [-Werror=return-type]

https://hg.mozilla.org/mozilla-central/rev/9625c1f40f00