`manage.py check` runs a number of Django self-tests: https://docs.djangoproject.com/en/1.8/ref/django-admin/#check https://docs.djangoproject.com/en/1.8/ref/checks/#builtin-checks Using the --deploy option enables a few more deployment-specific checks: https://docs.djangoproject.com/en/1.8/ref/checks/#security We should run these both during the Travis job and likely also deployment too (since some of the checks are environment-specific - eg like making sure SECRET_KEY is long enough).

There are no errors whilst running the plain --check in the vagrant environment, but when also using the --deploy option, there unsurprisingly some warnings from features we cannot turn on when running locally (since they rely on using HTTPS, and using self-signed certs is probably not worth the effort for now). vagrant ~/treeherder $ ./manage.py check System check identified no issues (0 silenced). vagrant ~/treeherder $ ./manage.py check --deploy System check identified some issues: WARNINGS: ?: (security.W001) You do not have 'django.middleware.security.SecurityMiddleware' in your MIDDLEWARE_CLASSES so the SECURE_HSTS_SECONDS, SECURE_CONTENT_TYPE_NOSNIFF, SECURE_BROWSER_XSS_FILTER, and SECURE_SSL_REDIRECT settings will have no effect. ?: (security.W002) You do not have 'django.middleware.clickjacking.XFrameOptionsMiddleware' in your MIDDLEWARE_CLASSES, so your pages will not be served with an 'x-frame-options' header. Unless there is a good reason for your site to be served in a frame, you should consider enabling this header to help prevent clickjacking attacks. ?: (security.W009) Your SECRET_KEY has less than 50 characters or less than 5 unique characters. Please generate a long and random SECRET_KEY, otherwise many of Django's security-critical features will be vulnerable to attack. ?: (security.W012) SESSION_COOKIE_SECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions. ?: (security.W016) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE_CLASSES, but you have not set CSRF_COOKIE_SECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token. ?: (security.W017) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE_CLASSES, but you have not set CSRF_COOKIE_HTTPONLY to True. Using an HttpOnly CSRF cookie makes it more difficult for cross-site scripting attacks to steal the CSRF token. ?: (security.W018) You should not have DEBUG set to True in deployment. System check identified 7 issues (0 silenced).

Bug 1258700 makes this easier, since all requests (even the ones served by WhiteNoise) can make use of the Django security middleware. As such, we both can (and will need to) enable the various Django security options, whereas had we done so before, it would have resulted in double HSTS headers etc (due to https://github.com/jacobian/wsgi-sslify/issues/3).

Comment on attachment 8748150 [details] [treeherder] mozilla:django-check-deploy > mozilla:master lgtm! (assuming it works on stage)

The three heroku instances' secret keys are long enough, just waiting on bug 1269753 :-)

I've had to make a slight change from the original PR. Until Django 1.10, the check command only exits non-zero if the failures are of ERROR level and above. However many of the deployment security checks are WARNING level. As such, until Django 1.10 (where we can use a newly added `--fail-level` option) we have to use awk to override the exit code (works better than grep since we want to print the original output and not suppress it).

Commits pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/5f54f6a363b987b08db7801d58a341b297efe676 Bug 1247344 - Set the X-Content-Type-Options header to 'nosniff' This instructs compatible browsers to not guess the content type of files, thereby avoiding potential security issues with user-uploaded files: https://docs.djangoproject.com/en/1.8/ref/middleware/#x-content-type-options https://github.com/mozilla/treeherder/commit/f15384f7f8b35b29508f42f22da0c462b387a41d Bug 1247344 - Mark session and CSRF cookies as being HTTPS-only This prevents them from being leaked if the client inadvertently later connects over HTTP: https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SESSION_COOKIE_SECURE https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-CSRF_COOKIE_SECURE https://github.com/mozilla/treeherder/commit/d74610fca398fd6aad8c80f8883eeb417a7000b2 Bug 1247344 - Set the X-XSS-Protection header This instructs compatible browsers to enable their XSS filters: https://docs.djangoproject.com/en/1.8/ref/middleware/#x-xss-protection https://github.com/mozilla/treeherder/commit/e726cb5b57bbcf1f43a5adf6ff5b4d03deba3f96 Bug 1247344 - Set the X-Frame-Options header to DENY This instructs compatible browsers to disallow the embedding of site pages in frames, preventing click-jacking attacks: https://docs.djangoproject.com/en/1.8/ref/clickjacking/ The /embed/ endpoint has been exempted, since its sole purpose is to be embedded (and it also not not contain any login forms). https://github.com/mozilla/treeherder/commit/bf1696eec9497418ddaf2dc4a6711f97d176db09 Bug 1247344 - Make the Django secret key longer for testing/development The Django deploy checks output errors if the secret key is shorter than 50 characters, and we're about to enable it on Travis and in runtest.sh: https://docs.djangoproject.com/en/1.8/ref/django-admin/#cmdoption-check--deploy https://github.com/mozilla/treeherder/commit/71ce9eef92b921f30225e75bcfb9b35ec6492770 Bug 1247344 - Move CommonMiddleware after SessionMiddleware To be more consistent with the Django default/recommended ordering: https://docs.djangoproject.com/en/1.8/ref/middleware/#middleware-ordering https://github.com/mozilla/treeherder/commit/073108842ab6480ef3c78568915dacbc4baffa55 Bug 1247344 - Silence the HSTS subdomains check Since we cannot set `SECURE_HSTS_INCLUDE_SUBDOMAINS` to True whilst the Vagrant and Travis SITE_URLs are fake subdomains of the stage/prod URLs, and can't use HTTPS. https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SILENCED_SYSTEM_CHECKS https://github.com/mozilla/treeherder/commit/7ed94e8eff39f72723d722e582e9bf8812480583 Bug 1247344 - Silence the CSRF_COOKIE_HTTPONLY check Since access to the cookie is legitimately required via JS: https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-CSRF_COOKIE_HTTPONLY https://github.com/mozilla/treeherder/commit/9e20b7934d34c75bf2b15b1afda8c44be1d5ec88 Bug 1247344 - Enable Django system checks during testing and deployment `manage.py check --deploy` is now run during Travis testing and as part of stage/prod/Heroku deployment. It checks for a number of common configuration mistakes & ensures security best practices are being followed: https://docs.djangoproject.com/en/1.8/ref/checks/

The deploy to stage worked :-)