eg: Running in test mode. Getting top bugs... Getting bug data from /bybug?tree=trunk&startday=2015-07-31&endday=2015-08-06. Getting JSON from http://localhost/orangefactor/api/bybug?tree=trunk&startday=2015-07-31&endday=2015-08-06. Results: <snip> ...skipping... File "woo_mailer.py", line 70, in top_bugs jdata = self.get_bybug() File "woo_mailer.py", line 100, in get_bybug return self.get_bugdata('/bybug?tree=trunk&startday=%s&endday=%s' % (self.startday, self.endday)) File "woo_mailer.py", line 95, in get_bugdata jdata = self.get_json(url_path) File "woo_mailer.py", line 86, in get_json data = urllib2.urlopen(url, timeout=30).read() File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 391, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 409, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open raise URLError(err) urllib2.URLError: <urlopen error timed out> Running in production mode. Getting top bugs... Getting bug data from /bybug?tree=trunk&startday=2015-08-04&endday=2015-08-10. Getting JSON from http://localhost/orangefactor/api/bybug?tree=trunk&startday=2015-08-04&endday=2015-08-10. Traceback (most recent call last): File "woo_mailer.py", line 282, in <module> main() File "woo_mailer.py", line 195, in main top = tb.top_bugs()[:10] File "woo_mailer.py", line 70, in top_bugs jdata = self.get_bybug() File "woo_mailer.py", line 100, in get_bybug return self.get_bugdata('/bybug?tree=trunk&startday=%s&endday=%s' % (self.startday, self.endday)) File "woo_mailer.py", line 95, in get_bugdata jdata = self.get_json(url_path) File "woo_mailer.py", line 86, in get_json data = urllib2.urlopen(url, timeout=30).read() File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 391, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 409, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open raise URLError(err) urllib2.URLError: <urlopen error timed out> We should: 1) Actually check the wrapper shell scripts into the repo 2) Disable test mode for the prod script 3) Stop dumping so much spam to the log 4) Increase the urllib2 timeout from 30s to something larger

* Stops running the mailer script twice (once in test mode, once in normal mode) * Raises the get json timeout from 30s to 120s * Switches to using requests over urllib2 * Retries up to 3 times if there was an error * Now prints the get json response body only if there was an error * Now prints the URL rather than entire cache contents when there is a cache hit * Sorts the imports in woo_mailer.py to follow PEP8 recommendations I'll be using parts of this mailer script for bug 1179310, so extra resilience here will improve the reliability of that script too.

Comment on attachment 8646861 [details] [diff] [review] Make the weekly mailer script less susecptible to timeouts Review of attachment 8646861 [details] [diff] [review]: ----------------------------------------------------------------- thanks!

Thank you for the review :-) [webtools@brasstacks1.dmz.scl3 root]$ cd ~/apps/orangefactor/src/orangefactor/ [webtools@brasstacks1.dmz.scl3 orangefactor]$ hg pull -u -v pulling from http://hg.mozilla.org/automation/orangefactor/ searching for changes all local heads known remotely adding changesets adding manifests adding file changes added 1 changesets with 3 changes to 3 files resolving manifests getting requirements/prod.txt getting woo_mailer.py getting woo_mailer.sh 3 files updated, 0 files merged, 0 files removed, 0 files unresolved [webtools@brasstacks1.dmz.scl3 orangefactor]$ ~/apps/orangefactor/bin/pip install -r requirements/prod.txt ... Successfully installed requests-2.7.0 [webtools@brasstacks1.dmz.scl3 orangefactor]$ crontab -l ... 0 8 * * tue (/home/webtools/apps/orangefactor/src/orangefactor/woo_mailer.sh 2>&1) >> /home/webtools/apps/orangefactor/woo_mailer.log [webtools@brasstacks1.dmz.scl3 orangefactor]$ (/home/webtools/apps/orangefactor/src/orangefactor/woo_mailer.sh 2>&1) >> /home/webtools/apps/orangefactor/woo_mailer.log [webtools@brasstacks1.dmz.scl3 orangefactor]$ tail ~/apps/orangefactor/woo_mailer.log -n 3 Getting JSON from http://localhost/orangefactor/api/bybug?tree=trunk&startday=2015-07-30&endday=2015-08-05. Sending email... Done!