Closed
Bug 1421073
Opened 7 years ago
Closed 7 years ago
Connections from host to Vagrant instance loopback adapter broken on non-Windows
Categories
(Tree Management :: Treeherder, defect, P1)
Tree Management
Treeherder
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: armenzg, Assigned: emorley)
References
Details
Attachments
(1 file)
(deleted),
text/x-github-pull-request
|
Details |
At various times today I managed to run `yarn start:local` and in all occassions the command will stop showing any output or saying that is ready [1].
I tried loading these just in case:
http://127.0.0.1:5000/
http://127.0.0.1:8000/
Maybe the nfs trick is not sufficient [2]. I don't even know if it is enough.
[1]
vagrant ~/treeherder $ yarn start:local
yarn run v1.3.2
$ node ./node_modules/neutrino/bin/neutrino start --presets ./neutrino-custom/local-watch.js
clean-webpack-plugin: /home/vagrant/treeherder/dist has been removed.
(node:20799) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
[2]
diff --git a/Vagrantfile b/Vagrantfile
index 37932015..e99e1a70 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -20,7 +20,7 @@ Vagrant.configure("2") do |config|
config.vm.network "private_network", type: "dhcp"
end
- config.vm.synced_folder ".", "/home/vagrant/treeherder", type: "nfs"
+ config.vm.synced_folder ".", "/home/vagrant/treeherder", nfs: false
config.vm.provider "virtualbox" do |vb, override|
# The Bento boxes (https://github.com/chef/bento) are recommended over the
Assignee | ||
Comment 1•7 years ago
|
||
The `yarn start:local` command does a full build (which can take 2-3 minutes) then remains in watch mode. The assets have to be served by running `./manage.py runserver` (or gunicorn) at the same time. This workflow is something that will be improved after bug 1363722.
If you are wanting to just work on backend/APIs, then you can skip this and just run `./manage.py runserver` and hit the API on the port 8000 URLs directly. If you haven't already seen this I'd check out:
https://treeherder.readthedocs.io/common_tasks.html#profiling-api-endpoint-performance
Assignee | ||
Comment 2•7 years ago
|
||
Could you confirm how long you waited for the command to complete, and also if you were running runserver in parallel (as mentioned here: https://treeherder.readthedocs.io/installation.html#starting-a-local-treeherder-instance)?
Component: Treeherder: Frontend → Treeherder: Docs & Development
Flags: needinfo?(armenzg)
Reporter | ||
Comment 3•7 years ago
|
||
I have been running `./manage.py runserver` and `yarn start:local` and I don't see anything in here:
http://localhost:8000/api/project/mozilla-inbound/runnable_jobs/
I've verified one more time this morning.
---------------
vagrant ~/treeherder $ ./manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
November 28, 2017 - 14:59:19
Django version 1.11.7, using settings 'treeherder.config.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
----------------
vagrant ~/treeherder $ ./manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
November 28, 2017 - 14:59:19
Django version 1.11.7, using settings 'treeherder.config.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Flags: needinfo?(armenzg)
Assignee | ||
Comment 4•7 years ago
|
||
Could you be more specific about "I don't see anything in here"?
What HTTP status code, what response body?
If you mean there's just zero runnable jobs returned, has data ingestion been run?
https://treeherder.readthedocs.io/seta.html#local-set-up
Note that `yarn start:local` just builds the static UI, and isn't needed for when just working on the API only.
Reporter | ||
Comment 5•7 years ago
|
||
Assignee | ||
Comment 6•7 years ago
|
||
Does replacing the standard runserver step with this help?
./manage.py runserver 0.0.0.0:8000
Reporter | ||
Comment 7•7 years ago
|
||
That did help. Thanks Ed!
What did I do wrong?
Assignee | ||
Comment 8•7 years ago
|
||
Ah great! In this case it sounds like you've not done anything wrong, but perhaps the fix for bug 1417099 didn't work as well as I hoped? (Sorry for the hassle!)
Could you confirm you're on latest master, Vagrant provision was run on it, and then you've disconnected and SSHed in again after that?
Failing that, could you pastebin the output of `ip -o -0 addr` and `ifconfig -a` run inside the Vagrant instance, and then also the output from `vagrant box list` run on the host?
Reporter | ||
Comment 9•7 years ago
|
||
I took the steps mentioned and I could still not make it work without `0.0.0.0:8000` appended to the command.
vagrant ~/treeherder $ ip -o -0 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1\ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\ link/ether 08:00:27:67:d9:b9 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\ link/ether 08:00:27:1c:c0:80 brd ff:ff:ff:ff:ff:ff
vagrant ~/treeherder $ ifconfig -a
eth0 Link encap:Ethernet HWaddr 08:00:27:67:d9:b9
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe67:d9b9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:329716 errors:0 dropped:0 overruns:0 frame:0
TX packets:57290 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:413689957 (413.6 MB) TX bytes:6717052 (6.7 MB)
eth1 Link encap:Ethernet HWaddr 08:00:27:1c:c0:80
inet addr:172.28.128.3 Bcast:172.28.128.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe1c:c080/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:696 errors:0 dropped:0 overruns:0 frame:0
TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:88170 (88.1 KB) TX bytes:12510 (12.5 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:2594 errors:0 dropped:0 overruns:0 frame:0
TX packets:2594 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:169037 (169.0 KB) TX bytes:169037 (169.0 KB)
armenzg@armenzg-mbp treeherder$ vagrant box list
bento/ubuntu-16.04 (virtualbox, 2.3.7)
bento/ubuntu-16.04 (virtualbox, 201710.25.0)
Assignee | ||
Comment 10•7 years ago
|
||
So to explain a bit what's happening here:
* various dev servers (eg Django's runserver, webpack-dev-server, ...) only listen on 127.0.0.1 by default, which isn't normally accessible from outside the VM. One has to instead tell them to listen on 0.0.0.0 instead, which in Django's case is a pain to do unless the user remembers to type `./manage.py runserver 0.0.0.0` each time.
* to avoid that, we instead use iptables rules in the VM to redirect all traffic to the loopback adapter so things "just work" without users needing to type `0.0.0.0` every time.
* recently the Bento Virtualbox images changed the network adapter name breaking the iptables rules, which I landed a fix for in bug 1417099.
* however that fix expected only one non-loopback network adapter (which is the case on Windows, where I tested it), but on other platforms there are actually two, since an additional adapter exists (via a conditional in Vagrantfile) to support NFS.
For now I have a fix to the iptables rules, though longer term I'd like to stop using them, since they are a bit hacky - and if/when we switch to Docker, we can use Docker Compose `command`s to streamline workflows such as starting runserver, so the user never has to type `./manage.py runserver 0.0.0.0`.
Assignee: nobody → emorley
Blocks: 1417099
Status: NEW → ASSIGNED
Priority: -- → P1
Summary: Cannot start UI → Connections from host to Vagrant instance loopback adapter broken on non-Windows
Comment 11•7 years ago
|
||
Comment 12•7 years ago
|
||
Commit pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/12a7c357cf98087cbcecba424ea17b8196025448
Bug 1421073 - Vagrant: Fix networking on non-Windows platforms (#3015)
On platforms that are not Windows, the Vagrantfile enables a second
virtual network adapter to allow NFS to work. However this breaks
the fix in bug 1417099, since it only expected one interface to be
returned by the grepped `ip` output. We could have adjusted that fix
to instead iterate through the results, however it's easier to just
use a wildcard with `iptables` instead, since we know all ethernet
adapters start with the letter `e`.
Assignee | ||
Comment 13•7 years ago
|
||
Cameron confirmed this fixed the issue for him; Armen, let me know if this doesn't for you :-)
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 14•7 years ago
|
||
It works without 0.0.0.0:8000
yarn start:local also works.
\o/
Updated•3 years ago
|
Component: Treeherder: Docs & Development → TreeHerder
You need to log in
before you can comment on or make changes to this bug.
Description
•