Closed Bug 1421073 Opened 7 years ago Closed 7 years ago

Connections from host to Vagrant instance loopback adapter broken on non-Windows

Categories

(Tree Management :: Treeherder, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Assigned: emorley)

References

Details

Attachments

(1 file)

At various times today I managed to run `yarn start:local` and in all occassions the command will stop showing any output or saying that is ready [1]. I tried loading these just in case: http://127.0.0.1:5000/ http://127.0.0.1:8000/ Maybe the nfs trick is not sufficient [2]. I don't even know if it is enough. [1] vagrant ~/treeherder $ yarn start:local yarn run v1.3.2 $ node ./node_modules/neutrino/bin/neutrino start --presets ./neutrino-custom/local-watch.js clean-webpack-plugin: /home/vagrant/treeherder/dist has been removed. (node:20799) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56 parseQuery() will be replaced with getOptions() in the next major version of loader-utils. [2] diff --git a/Vagrantfile b/Vagrantfile index 37932015..e99e1a70 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -20,7 +20,7 @@ Vagrant.configure("2") do |config| config.vm.network "private_network", type: "dhcp" end - config.vm.synced_folder ".", "/home/vagrant/treeherder", type: "nfs" + config.vm.synced_folder ".", "/home/vagrant/treeherder", nfs: false config.vm.provider "virtualbox" do |vb, override| # The Bento boxes (https://github.com/chef/bento) are recommended over the
The `yarn start:local` command does a full build (which can take 2-3 minutes) then remains in watch mode. The assets have to be served by running `./manage.py runserver` (or gunicorn) at the same time. This workflow is something that will be improved after bug 1363722. If you are wanting to just work on backend/APIs, then you can skip this and just run `./manage.py runserver` and hit the API on the port 8000 URLs directly. If you haven't already seen this I'd check out: https://treeherder.readthedocs.io/common_tasks.html#profiling-api-endpoint-performance
Could you confirm how long you waited for the command to complete, and also if you were running runserver in parallel (as mentioned here: https://treeherder.readthedocs.io/installation.html#starting-a-local-treeherder-instance)?
Component: Treeherder: Frontend → Treeherder: Docs & Development
Flags: needinfo?(armenzg)
I have been running `./manage.py runserver` and `yarn start:local` and I don't see anything in here: http://localhost:8000/api/project/mozilla-inbound/runnable_jobs/ I've verified one more time this morning. --------------- vagrant ~/treeherder $ ./manage.py runserver Performing system checks... System check identified no issues (0 silenced). November 28, 2017 - 14:59:19 Django version 1.11.7, using settings 'treeherder.config.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C. ---------------- vagrant ~/treeherder $ ./manage.py runserver Performing system checks... System check identified no issues (0 silenced). November 28, 2017 - 14:59:19 Django version 1.11.7, using settings 'treeherder.config.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C.
Flags: needinfo?(armenzg)
Could you be more specific about "I don't see anything in here"? What HTTP status code, what response body? If you mean there's just zero runnable jobs returned, has data ingestion been run? https://treeherder.readthedocs.io/seta.html#local-set-up Note that `yarn start:local` just builds the static UI, and isn't needed for when just working on the API only.
Does replacing the standard runserver step with this help? ./manage.py runserver 0.0.0.0:8000
That did help. Thanks Ed! What did I do wrong?
Ah great! In this case it sounds like you've not done anything wrong, but perhaps the fix for bug 1417099 didn't work as well as I hoped? (Sorry for the hassle!) Could you confirm you're on latest master, Vagrant provision was run on it, and then you've disconnected and SSHed in again after that? Failing that, could you pastebin the output of `ip -o -0 addr` and `ifconfig -a` run inside the Vagrant instance, and then also the output from `vagrant box list` run on the host?
I took the steps mentioned and I could still not make it work without `0.0.0.0:8000` appended to the command. vagrant ~/treeherder $ ip -o -0 addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1\ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\ link/ether 08:00:27:67:d9:b9 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\ link/ether 08:00:27:1c:c0:80 brd ff:ff:ff:ff:ff:ff vagrant ~/treeherder $ ifconfig -a eth0 Link encap:Ethernet HWaddr 08:00:27:67:d9:b9 inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe67:d9b9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:329716 errors:0 dropped:0 overruns:0 frame:0 TX packets:57290 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:413689957 (413.6 MB) TX bytes:6717052 (6.7 MB) eth1 Link encap:Ethernet HWaddr 08:00:27:1c:c0:80 inet addr:172.28.128.3 Bcast:172.28.128.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe1c:c080/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:696 errors:0 dropped:0 overruns:0 frame:0 TX packets:60 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:88170 (88.1 KB) TX bytes:12510 (12.5 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:2594 errors:0 dropped:0 overruns:0 frame:0 TX packets:2594 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:169037 (169.0 KB) TX bytes:169037 (169.0 KB) armenzg@armenzg-mbp treeherder$ vagrant box list bento/ubuntu-16.04 (virtualbox, 2.3.7) bento/ubuntu-16.04 (virtualbox, 201710.25.0)
So to explain a bit what's happening here: * various dev servers (eg Django's runserver, webpack-dev-server, ...) only listen on 127.0.0.1 by default, which isn't normally accessible from outside the VM. One has to instead tell them to listen on 0.0.0.0 instead, which in Django's case is a pain to do unless the user remembers to type `./manage.py runserver 0.0.0.0` each time. * to avoid that, we instead use iptables rules in the VM to redirect all traffic to the loopback adapter so things "just work" without users needing to type `0.0.0.0` every time. * recently the Bento Virtualbox images changed the network adapter name breaking the iptables rules, which I landed a fix for in bug 1417099. * however that fix expected only one non-loopback network adapter (which is the case on Windows, where I tested it), but on other platforms there are actually two, since an additional adapter exists (via a conditional in Vagrantfile) to support NFS. For now I have a fix to the iptables rules, though longer term I'd like to stop using them, since they are a bit hacky - and if/when we switch to Docker, we can use Docker Compose `command`s to streamline workflows such as starting runserver, so the user never has to type `./manage.py runserver 0.0.0.0`.
Assignee: nobody → emorley
Blocks: 1417099
Status: NEW → ASSIGNED
Priority: -- → P1
Summary: Cannot start UI → Connections from host to Vagrant instance loopback adapter broken on non-Windows
Commit pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/12a7c357cf98087cbcecba424ea17b8196025448 Bug 1421073 - Vagrant: Fix networking on non-Windows platforms (#3015) On platforms that are not Windows, the Vagrantfile enables a second virtual network adapter to allow NFS to work. However this breaks the fix in bug 1417099, since it only expected one interface to be returned by the grepped `ip` output. We could have adjusted that fix to instead iterate through the results, however it's easier to just use a wildcard with `iptables` instead, since we know all ethernet adapters start with the letter `e`.
Cameron confirmed this fixed the issue for him; Armen, let me know if this doesn't for you :-)
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
It works without 0.0.0.0:8000 yarn start:local also works. \o/
Component: Treeherder: Docs & Development → TreeHerder
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: