Closed
Bug 1419343
Opened 7 years ago
Closed 7 years ago
heroku-release-notifications app/worker.1: Exception 'build_props' caught by generic exception trap
Categories
(Release Engineering :: Release Automation: Other, enhancement)
Release Engineering
Release Automation: Other
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jlorenzo, Assigned: garbas)
References
Details
Attachments
(2 files, 1 obsolete file)
(deleted),
text/x-github-pull-request
|
Details | |
(deleted),
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
Over email, nthomas said:
These seem to appear when we start a Fennec build on beta, and in particular bouncer submission jobs like
https://tools.taskcluster.net/groups/NnPn1IvtQqq9ur84LyqhWg/tasks/eB3IRBmhQ56zu1G_dKpxhg/details
What's the right fix ?
The full traceback is:
> Nov 20 18:36:22 heroku-release-notifications app/worker.1: 2017-11-21 02:36:21,545 - pulsenotify.util - INFO - Fetching task eB3IRBmhQ56zu1G_dKpxhg from Taskcluster
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: 2017-11-21 02:36:24,016 - aiohttp.client - WARNING - Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: 2017-11-21 02:36:24,067 - pulsenotify.consumer - ERROR - Exception 'build_props' caught by generic exception trap
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: Traceback (most recent call last):
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 113, in dispatch
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: await self.notifiers[plugin_name].notify(task_data, id_section)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/util.py", line 57, in timed
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: result = await f(*args, **kw)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/plugins/ses.py", line 45, in notify
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: inspector_url=task_data.inspector_url)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/environment.py", line 1008, in render
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: return self.environment.handle_exception(exc_info, True)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/environment.py", line 780, in handle_exception
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: reraise(exc_type, exc_value, tb)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/.heroku/python/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: raise value.with_traceback(tb)
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/templates/email_template.html", line 13, in top-level template code
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: {% for log in logs %}
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/plugins/ses.py", line 40, in <genexpr>
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: log_destinations = (l['destination_url'] for l in task_data.log_data())
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 240, in log_data
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: 's3_key': self.make_s3_key(run_number),
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: File "/app/pulsenotify/consumer.py", line 246, in make_s3_key
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: build_properties = self.definition['extra']['build_props']
> Nov 20 18:36:24 heroku-release-notifications app/worker.1: KeyError: 'build_props'
Reporter | ||
Comment 1•7 years ago
|
||
`build_props` is used to define the S3 key[1] to upload logs. We can modify pulse-notify to fetch data from `payload.properties` instead of `extra.build_props`. Here's a patch to do so.
[1] https://github.com/mozilla-releng/pulse-notify/blob/5db9b372d2fd2a0ca349b9bc96de15b4be6296ef/pulsenotify/consumer.py#L246
Attachment #8930431 -
Flags: review?(rail)
Reporter | ||
Updated•7 years ago
|
Assignee: nobody → jlorenzo
Updated•7 years ago
|
Attachment #8930431 -
Flags: review?(rail)
Assignee | ||
Updated•7 years ago
|
Assignee: jlorenzo → rgarbas
Status: NEW → ASSIGNED
Assignee | ||
Comment 2•7 years ago
|
||
Attachment #8930431 -
Attachment is obsolete: true
Assignee | ||
Comment 3•7 years ago
|
||
:aki this patch fixes the subject/message of the emails send by release-notify-* tasks to use shipping-product and not project (branch). Additionally I also move more common things to job-defaults and get it ready to add other products.
Attachment #8934163 -
Flags: review?(aki)
Assignee | ||
Comment 4•7 years ago
|
||
:aki The previous patch was created against maple. I will create a new patch for m-c since i realized we also need to fix it there.
Assignee | ||
Updated•7 years ago
|
Attachment #8934163 -
Attachment description: fix-in-tree-notification-subject.patch → maple-fix-in-tree-notification-subject.patch
Attachment #8934163 -
Attachment filename: fix-in-tree-notification-subject.patch → maple-fix-in-tree-notification-subject.patch
Assignee | ||
Comment 5•7 years ago
|
||
:aki i looked how to solve this in m-c, but maybe easiest would be just to wait until we merge maple into m-c, since you are already working towards that.
Comment 6•7 years ago
|
||
Comment on attachment 8934163 [details] [diff] [review]
maple-fix-in-tree-notification-subject.patch
I've been thinking the default: of release-drivers and the maple: release-drivers-staging is backwards. If we ever start a staging release on a different branch than maple, we'll spam the production list.
This should probably be:
by-project:
mozilla-beta: release-drivers
mozilla-release: release-drivers
default: release-drivers-staging
Other than that, lgtm.
Attachment #8934163 -
Flags: review?(aki) → review+
Assignee | ||
Comment 7•7 years ago
|
||
:aki Right, that makes much more sense.
Comment 8•7 years ago
|
||
Landed on maple with fixes: https://hg.mozilla.org/projects/maple/rev/e60ed99c0c3e654792885ee0f74fe96d9bdf434d
Comment 9•7 years ago
|
||
(In reply to Rok Garbas [:garbas] from comment #3)
> Created attachment 8934163 [details] [diff] [review]
> maple-fix-in-tree-notification-subject.patch
>
> :aki this patch fixes the subject/message of the emails send by
> release-notify-* tasks to use shipping-product and not project (branch).
> Additionally I also move more common things to job-defaults and get it ready
> to add other products.
now we're going to get "fennec shipped .." emails for staging, rather than "maple shipped". This may be confusing to people on both the prod and staging lists?
Assignee | ||
Comment 10•7 years ago
|
||
:aki tnx, i will include you suggestion in the other fixes i'm working on regarding notification (follow up from dustin's review).
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•