Closed Bug 1419343 Opened 7 years ago Closed 7 years ago

heroku-release-notifications app/worker.1: Exception 'build_props' caught by generic exception trap

Categories

(Release Engineering :: Release Automation: Other, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jlorenzo, Assigned: garbas)

References

Details

Attachments

(2 files, 1 obsolete file)

Over email, nthomas said: These seem to appear when we start a Fennec build on beta, and in particular bouncer submission jobs like https://tools.taskcluster.net/groups/NnPn1IvtQqq9ur84LyqhWg/tasks/eB3IRBmhQ56zu1G_dKpxhg/details What's the right fix ? The full traceback is: > Nov 20 18:36:22 heroku-release-notifications app/worker.1: 2017-11-21 02:36:21,545 - pulsenotify.util - INFO - Fetching task eB3IRBmhQ56zu1G_dKpxhg from Taskcluster > Nov 20 18:36:24 heroku-release-notifications app/worker.1: 2017-11-21 02:36:24,016 - aiohttp.client - WARNING - Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8 > Nov 20 18:36:24 heroku-release-notifications app/worker.1: 2017-11-21 02:36:24,067 - pulsenotify.consumer - ERROR - Exception 'build_props' caught by generic exception trap > Nov 20 18:36:24 heroku-release-notifications app/worker.1: Traceback (most recent call last): > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 113, in dispatch > Nov 20 18:36:24 heroku-release-notifications app/worker.1: await self.notifiers[plugin_name].notify(task_data, id_section) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/util.py", line 57, in timed > Nov 20 18:36:24 heroku-release-notifications app/worker.1: result = await f(*args, **kw) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/plugins/ses.py", line 45, in notify > Nov 20 18:36:24 heroku-release-notifications app/worker.1: inspector_url=task_data.inspector_url) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/environment.py", line 1008, in render > Nov 20 18:36:24 heroku-release-notifications app/worker.1: return self.environment.handle_exception(exc_info, True) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/environment.py", line 780, in handle_exception > Nov 20 18:36:24 heroku-release-notifications app/worker.1: reraise(exc_type, exc_value, tb) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise > Nov 20 18:36:24 heroku-release-notifications app/worker.1: raise value.with_traceback(tb) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/templates/email_template.html", line 13, in top-level template code > Nov 20 18:36:24 heroku-release-notifications app/worker.1: {% for log in logs %} > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/plugins/ses.py", line 40, in <genexpr> > Nov 20 18:36:24 heroku-release-notifications app/worker.1: log_destinations = (l['destination_url'] for l in task_data.log_data()) > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 240, in log_data > Nov 20 18:36:24 heroku-release-notifications app/worker.1: 's3_key': self.make_s3_key(run_number), > Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 246, in make_s3_key > Nov 20 18:36:24 heroku-release-notifications app/worker.1: build_properties = self.definition['extra']['build_props'] > Nov 20 18:36:24 heroku-release-notifications app/worker.1: KeyError: 'build_props'
Attached file pulse-notify PR (obsolete) (deleted) —
`build_props` is used to define the S3 key[1] to upload logs. We can modify pulse-notify to fetch data from `payload.properties` instead of `extra.build_props`. Here's a patch to do so. [1] https://github.com/mozilla-releng/pulse-notify/blob/5db9b372d2fd2a0ca349b9bc96de15b4be6296ef/pulsenotify/consumer.py#L246
Attachment #8930431 - Flags: review?(rail)
Assignee: nobody → jlorenzo
Attachment #8930431 - Flags: review?(rail)
Assignee: jlorenzo → rgarbas
Status: NEW → ASSIGNED
Attached file pulse-notify PR v2 (deleted) —
Attachment #8930431 - Attachment is obsolete: true
:aki this patch fixes the subject/message of the emails send by release-notify-* tasks to use shipping-product and not project (branch). Additionally I also move more common things to job-defaults and get it ready to add other products.
Attachment #8934163 - Flags: review?(aki)
:aki The previous patch was created against maple. I will create a new patch for m-c since i realized we also need to fix it there.
Attachment #8934163 - Attachment description: fix-in-tree-notification-subject.patch → maple-fix-in-tree-notification-subject.patch
Attachment #8934163 - Attachment filename: fix-in-tree-notification-subject.patch → maple-fix-in-tree-notification-subject.patch
:aki i looked how to solve this in m-c, but maybe easiest would be just to wait until we merge maple into m-c, since you are already working towards that.
Comment on attachment 8934163 [details] [diff] [review] maple-fix-in-tree-notification-subject.patch I've been thinking the default: of release-drivers and the maple: release-drivers-staging is backwards. If we ever start a staging release on a different branch than maple, we'll spam the production list. This should probably be: by-project: mozilla-beta: release-drivers mozilla-release: release-drivers default: release-drivers-staging Other than that, lgtm.
Attachment #8934163 - Flags: review?(aki) → review+
:aki Right, that makes much more sense.
(In reply to Rok Garbas [:garbas] from comment #3) > Created attachment 8934163 [details] [diff] [review] > maple-fix-in-tree-notification-subject.patch > > :aki this patch fixes the subject/message of the emails send by > release-notify-* tasks to use shipping-product and not project (branch). > Additionally I also move more common things to job-defaults and get it ready > to add other products. now we're going to get "fennec shipped .." emails for staging, rather than "maple shipped". This may be confusing to people on both the prod and staging lists?
:aki tnx, i will include you suggestion in the other fixes i'm working on regarding notification (follow up from dustin's review).
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: