Closed Bug 1807029 Opened 2 years ago Closed 2 years ago

Migrate remaining miscellaneous hooks from AWS -> GCP

Categories

(Release Engineering :: Firefox-CI Administration, task)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ahal, Assigned: ahal)

References

Details

(Keywords: leave-open)

Attachments

(5 files)

There are still several hooks defined in ci-config that run in AWS, like the code-review / code-coverage bots.

Keywords: leave-open

This also switches the "testing" hooks to use it.

Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/ci-configuration/rev/f026d3316360 Create GCP pools for code-review / code-coverage bots, r=marco,releng-reviewers,jcristau
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/ci-configuration/rev/84a9832e70bd Add missing scope for new code-coverage GCP pool, r=releng-reviewers,jcristau

Looks like the hook is working, but the task itself is failing, e.g:
https://firefox-ci-tc.services.mozilla.com/tasks/EEzX2LnrS4SIhhIbZOmeww

That doesn't immediately look like a GCP problem, but it was passing consistently prior to the switch. Maybe we need to update DNS in order to resolve api.coverage.testing.moz.tools ? Marco, what do you think?

Flags: needinfo?(mcastelluccio)

It looks like the task is actually able to resolve the URL, but the server is returning 404 for this revision (https://api.coverage.testing.moz.tools/v2/path?repository=mozilla-central&changeset=616a6f1689dc99810e5d7e6465b781a49b6430c8&platform=all&suite=all doesn't work, while the non-testing https://api.testing.moz.tools/v2/path?repository=mozilla-central&changeset=616a6f1689dc99810e5d7e6465b781a49b6430c8&platform=all&suite=all works).
Looking at https://api.coverage.testing.moz.tools/v2/history it looks like the coverage for this revision was not uploaded (616a6f1689dc99810e5d7e6465b781a49b6430c8 is missing).

Let's check the next run of the hook.

Flags: needinfo?(mcastelluccio)
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/ci-configuration/rev/dd1e1eb93658 Revert code-coverage/review testing hooks back to AWS, r=marco

Looks like the testing hooks are still failing after reverting to AWS:
https://firefox-ci-tc.services.mozilla.com/tasks/BF2TNDKlToqpJtycdQKjYw

I wonder if the initial switch got them into some kind of bad state that we need to manually get them out of again? Marco, do you think we should try switching production to GCP? Or do you think the same thing will happen there?

Flags: needinfo?(mcastelluccio)

It looks like the Redis testing instance was full, I tried to upgrade it to a larger instance, let's see if it starts working again.
If it starts working again, we can move testing to gcp again and see if it works.

Flags: needinfo?(mcastelluccio)

Reverting these back to AWS did not resolve the failures, then Marco noticed
the DB was full. After clearing that out they started working again. Let's
see if they work on GCP now.

Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/ci-configuration/rev/d131a217a425 Switch code-coverage/review testing hooks back to GCP, r=marco

Woo, here's a green gcp task:
https://firefox-ci-tc.services.mozilla.com/tasks/LTTfkApyTDWnRawZTvL0rA

I guess I was just massively unlucky that this hit right in-between me switching the hook over to GCP. I'll put up a patch to migrate the non-testing hooks as well.

Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/ci-configuration/rev/9432f1650479 Switch code-coverage/review production hooks to GCP, r=marco

Marco, do you know what the plan is for project-relman/bugzilla-dashboard-backend-* hooks?

Flags: needinfo?(mcastelluccio)
Depends on: 1816884

We can drop those hooks, I filed bug 1816884.

Flags: needinfo?(mcastelluccio)
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: