Something that's been coming up a lot recently is that it's very difficult to understand the state of Rules in the UI. Because there's so many axis' of complexity (at least one for each part of the update URL), it can be difficult to figure out what any given update request will receive. We should do our best to keep the Rules simple (both through human effort and improving rule-related Balrog features), but we're always going to be dealing with some level of complexity, and I'm certain that we can find better ways to represent the Rules to make it easier for humans to understand. There may not be a one-size-fits-all approach here, and there's no reason we can't have multiple views or visualizations of the Rules.

A good start might be a flowchart to visualize all possible paths for a given product+channel combination.

I'm mulling over this a lot and I've been talking to people about various *potential* solutions. Instead of a flowchart, I'm inclined towards a different paradigm. Just like there's tooling at the top of the Rules page to do some searching and filtering; it'd be nice to be able to input one or more "client" simulations. By that I mean you type in (or some fancy drop-downs) things like "Firefox, 53, win 64, locale fr, etc." and it uses this to calculate which rules it'd match. Once it knows which rule(s) this matches, it displays that/those card(s). The ideal use case is to type in a complete client request simulation, find thee one rule that it matches and from there manually inspect its mappings. I don't know if it's safe to do this all in JavaScript. Ideally it should be pushed down to the same Python business logic we use in auslib when a client request comes in. But broken up and instead of returning the XML, it returns a set of rules. Also, additional parameters to this should be things like "Assume 'relmanadmin' have signed off all pending changes" or "Assume today is tomorrow". That way you should be able to see what rules are matched not only depending on the client but also on the state of balrog. The above is a loose brain dump of ideas from chatting to various Balrog users. Something needs to be done but we really need to dig into the weeds before we start coding on something.

Some of this overlaps (or maybe supercedes) a bit with https://bugzilla.mozilla.org/show_bug.cgi?id=1343899, but I think these are all fantastic ideas. (In reply to Peter Bengtsson [:peterbe] from comment #2) > I'm mulling over this a lot and I've been talking to people about various > *potential* solutions. > > Instead of a flowchart, I'm inclined towards a different paradigm. Just like > there's tooling at the top of the Rules page to do some searching and > filtering; it'd be nice to be able to input one or more "client" > simulations. By that I mean you type in (or some fancy drop-downs) things > like "Firefox, 53, win 64, locale fr, etc." and it uses this to calculate > which rules it'd match. Once it knows which rule(s) this matches, it > displays that/those card(s). > > The ideal use case is to type in a complete client request simulation, find > thee one rule that it matches and from there manually inspect its mappings. > > I don't know if it's safe to do this all in JavaScript. Ideally it should be > pushed down to the same Python business logic we use in auslib when a client > request comes in. But broken up and instead of returning the XML, it returns > a set of rules. I think this is really important -- otherwise we have to reimplement the entire rule matching logic in the frontend. Something like https://bugzilla.mozilla.org/show_bug.cgi?id=1316948 might help with this. I think someone also suggested that being able to append a timestamp to update queries might be useful. Eg: A request to https://aus5.mozilla.org/update/3/Firefox/33.0/20141202185629/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/default/default/default/update.xml?at=1510043156 would rewind (or fast forward the Rules to the given time before making a response. There might be security issues with doing that on public domains, though. > Also, additional parameters to this should be things like "Assume > 'relmanadmin' have signed off all pending changes" or "Assume today is > tomorrow". That way you should be able to see what rules are matched not > only depending on the client but also on the state of balrog. > > The above is a loose brain dump of ideas from chatting to various Balrog > users. Something needs to be done but we really need to dig into the weeds > before we start coding on something.