Overview: Need to create a content server for the start page. Project Scope: - input validation for incoming variables (GET) - logic to retrieve data based on provided variables - schema for storing content (go with key/val store for now) - caching policies, use of redis or memcache, and aggressive http headers to work properly with proxy cache/load balancers/cdn - panic button to turn all pages static for high load situations Dependencies: Legal: Any agreement with openx would have to be considered, should we go that route. Don't think there are any other concerns since the data is provided in public env vars. Security: Normal sec review. Finance: Any billing to openx, should we go that route. Assumptions: - that we'll work something out with openx as a content distribution service and take advantage of their reporting Deliverables: Need to flesh this out.

https://wiki.mozilla.org/Firefox/Projects/Firefox_Start#Design_Specification The snippet service has more details.

I drew a picture, in case it helps: http://people.mozilla.org/~morgamic/snippet-service.png My questions are: * metrics: how would we track conversion and click through rates (use an existing system) * enterprise solutions: how would we integrate a pre-existing solution without degrading service level * what kind of admin functions are crucial * disaster recovery - how do we tell firefox we are down, or enter a static only mode * cheap checks - can we set it up so client only pulls if content-length header is changed for json feed (and skip downloading entire document)

the about:home related patches have landed, it will gather and inject an html content, this content can have <script/> tags with code, all the js code should go into these. It is currently looking at "http://snippets.mozilla.com/" + STARTPAGE_VERSION + "/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/ this can be changed if needed. For now it will see a 404 error and use default inlined content.

Starting to take a look at putting together a service for this, and thinking I might have a couple of bugs to file: * "http://snippets.mozilla.com" is hard-coded into the browser. Is there any way we could get this as a preference in about:config? Changing the domain and port would be especially nice for development / QA * bug 594989 added a hack to move <script> tags around to ensure execution, but it seems to be not quite working. If I serve up content like this: <div class="snippet"><script type="text/javascript;version=1.8">alert("HELLO")</script><p>Sample content #1</p></div> <div class="snippet"><script type="text/javascript;version=1.8">alert("HELLO AGAIN")</script><p>Sample content #2</p></div> It seems to break with an error in my console, with the snippets never shown and JS never executed: aboutHome.js (line 174) "Node was not found" code: "8 snippetsElt.replaceChild(relocatedScript, elt); If I remove the wrapper <div>s, things seem to work. Seems like it requires bare <script> tags, but not sure why it would make a difference after looking at the code

(In reply to comment #5) > * "http://snippets.mozilla.com" is hard-coded into the browser. Is there any > way we could get this as a preference in about:config? Changing the domain and > port would be especially nice for development / QA we can change that code without any problem. I did not put the url in a pref to avoid surprises (like adwares/extensions/users changing the pref when we really don't want that). But could be I'm just too paranoid?

(In reply to comment #6) > (In reply to comment #5) > > * "http://snippets.mozilla.com" is hard-coded into the browser. Is there any > > way we could get this as a preference in about:config? Changing the domain and > > port would be especially nice for development / QA > > we can change that code without any problem. I did not put the url in a pref to > avoid surprises (like adwares/extensions/users changing the pref when we really > don't want that). But could be I'm just too paranoid? Not sure who has the final say on this, but addons/extensions/users can already change the home page altogether. And, it looks like there's a localStorage key named "snippets-update-url" that can be changed after the first hit, anyway. I think a malicious extension or a user with Firebug can change that as well. An about:config pref would just make it easier for an advanced user (eg. me and QA)

(In reply to comment #7) > (In reply to comment #6) > > (In reply to comment #5) > > > * "http://snippets.mozilla.com" is hard-coded into the browser. Is there any > > > way we could get this as a preference in about:config? Changing the domain and > > > port would be especially nice for development / QA > > > > we can change that code without any problem. I did not put the url in a pref to > > avoid surprises (like adwares/extensions/users changing the pref when we really > > don't want that). But could be I'm just too paranoid? > > Not sure who has the final say on this, but addons/extensions/users can already > change the home page altogether. And, it looks like there's a localStorage key > named "snippets-update-url" that can be changed after the first hit, anyway. I > think a malicious extension or a user with Firebug can change that as well. > > An about:config pref would just make it easier for an advanced user (eg. me and > QA) Oh, and speaking of that localStorage["snippets-update-url"] key: If there is an about:config pref, you'll probably need to replace the localStorage value on any changes, as well

could you please file a bug on the pref addition? So we can move the discussion/implementation there.

(In reply to comment #9) > could you please file a bug on the pref addition? So we can move the > discussion/implementation there. filed bug 603674

This will probably result in another bug filed, but having some trouble getting this to pull snippets from a server. I've got this as a baby step toward a snippet service: http://snippets.decafbad.com/ It doesn't do anything fancy yet, just serves up a couple of snippets. If you set up an /etc/hosts entry for snippets.mozilla.com, this service should respond for that as well. I copied over aboutHome.{css,js} and captured the HTML to set up a quick-and-dirty about:home simulator at that above URL. That seems to work. But, when I go to about:home in a nightly, I see a hit to my server - but it never shows up in the browser. Firebug doesn't seem to want to work with about:home, so I'm not sure what's going on. By typing some javascript: URLs into the address bar, I can see what's in localStorage. The URL looks okay, but nothing ever shows up in localStorage['snippets']

did you enable cross-site requests on your webserver?

and lol for the pie :)

(In reply to comment #12) > did you enable cross-site requests on your webserver? Looks like adding an Access-Control-Allow-Origin: * worked. Next problem is that <style> tags in snippet content don't seem to work, not even if they're at the root of the document fragment. Should I file a bug for that?

I think could be the same issue we had for scripts... does inline styling work?

also, try to put the <style> after all contents, as the last thing in the file.

(In reply to comment #15) > I think could be the same issue we had for scripts... does inline styling work? Inline styles like this work: <p id="pie" style="font-weight: bold; color: blue">I like pie. Do you like pie?</p> But <style> tags like this don't work, no matter where they appear in the response: <style type="text/css">#pie { font-weight: bold; color: blue }</style> (In reply to comment #16) > also, try to put the <style> after all contents, as the last thing in the file. Well, there isn't a file, per se. It's a collection of snippets selected by client attributes, appended together in one response. Like I said with the <script> tags, I can try to pool all the styling together from all selected snippets. But, it would be much better if we can just serve up a block of arbitrary HTML and have it work.

(In reply to comment #17) > (In reply to comment #16) > > also, try to put the <style> after all contents, as the last thing in the file. > > Well, there isn't a file, per se. It's a collection of snippets selected by > client attributes, appended together in one response. I know, just append the <style> tag at the end, not at the start of your response. From what I know it could be anough.

I mean, put any <style> tag after the element they are styling, not before. so: snippet <style> snippet <style>

(In reply to comment #18) > (In reply to comment #17) > > (In reply to comment #16) > > > also, try to put the <style> after all contents, as the last thing in the file. > > > > Well, there isn't a file, per se. It's a collection of snippets selected by > > client attributes, appended together in one response. > > I know, just append the <style> tag at the end, not at the start of your > response. From what I know it could be anough. No matter where <style> tags appear, the rules are not applied. Do you have an example you've tried where they're working?

Hm, so if I create a test page with this code it works fine, the start page is not doing anything more than this, so doesn't looks like a bug of innerHTML like the <script> one. Some issue with the encoding of the chunk? <div id="test"/> <script> var content = '<p id="pie">I like pie. Do you like pie?</p><style type="text/css">#pie { font-weight: bold; color: blue }</style>'; document.getElementById("test").innerHTML = content; </script>

(In reply to comment #21) > Hm, so if I create a test page with this code it works fine, the start page is > not doing anything more than this, so doesn't looks like a bug of innerHTML > like the <script> one. Some issue with the encoding of the chunk? > > <div id="test"/> > <script> > var content = '<p id="pie">I like pie. Do you like pie?</p><style > type="text/css">#pie { font-weight: bold; color: blue }</style>'; > document.getElementById("test").innerHTML = content; > </script> Yes, <style> tags injected with innerHTML work fine in an HTML page that's not about:home. Did you try that snippet above with about:home? I don't know what the difference is, but I'm not all that familiar with how the about:home page itself works

yes I can confirm it does not work in about:home, I'm trying to find what's causing that.

the problem is due to the fact this is an xhtml document. first xhtml does not allow <style> to be used out of <head>, so the document is not valid xhtml, but that's a secondary issue (Also innerHTML is somehow an extension so...) innerHTML in xhtml had/has a bunch of issues in xhtml docs, Boris Zbarsky worked on it and could know if it has issues with the <style> tag. Replacing innerHTML with proper DOM methods is feasible but adds a lot of code, if that's the only way I'd prefer going for inlined style.

(In reply to comment #24) > the problem is due to the fact this is an xhtml document. first xhtml does not > allow <style> to be used out of <head>, so the document is not valid xhtml, but > that's a secondary issue (Also innerHTML is somehow an extension so...) > innerHTML in xhtml had/has a bunch of issues in xhtml docs, Boris Zbarsky > worked on it and could know if it has issues with the <style> tag. Replacing > innerHTML with proper DOM methods is feasible but adds a lot of code, if that's > the only way I'd prefer going for inlined style. I think we'd really like to not litter HTML with inline styles, and we'll probably want to develop common reusable CSS stylesheets for various kinds of snippets. Seems like using innerHTML leaves us with a lot of hacks & restrictions. Any chance you could try using importNode/adoptNode from the XHR response document? It's been awhile since I did that, but it didn't seem like much more code than innerHTML

Basically, what I'm saying is: If there's a cleaner way to do it, whether it takes more code or not, now's the time to do it before we ship a major product release and have to live with it going forward

Here's a quick & dirty tweak to use DOMParser / importNode. It works in my testing page on my server, but I don't have an Fx build env set up to try it in about:home. Maybe it works?

(Maybe as a patch this time)

Actually, it looks like the getElementsByTagName("HEAD") causes a problem. Got an Fx build env working, and this patch seems to work better

Comment on attachment 483755 [details] [diff] [review] Switch aboutHome.js to use DOM methods I think before going this way we should figure out if th innerHTML issue is a bug or a wanted behavior, work-arounding a bug would be generally bad. Will try to ping bz today to get an answer. >diff --git a/browser/base/content/aboutHome.js b/browser/base/content/aboutHome.js >+ // Attempt to parse the snippets as XML source >+ let parser = new DOMParser(); >+ let dom = parser.parseFromString(snippets, 'text/xml'); >+ if ('parsererror' == dom.documentElement.nodeName) { >+ snippets = null; >+ } else { >+ snippets = dom.documentElement; >+ } >+ } the injecting code should go to its own function, something like importNodesFromStringToElement(aString, aDestElt) >- // Scripts injected by innerHTML are inactive, so we have to relocate them >- // through DOM manipulation to activate their contents. >+ snippets = document.importNode(snippets, true); >+ snippetsElt.appendChild(snippets); and this should be inside the function as well. I'm not sure doing an importNode of another documentElement is sane, imo it would be better to loop through its children and insert them. >+ > Array.forEach(snippetsElt.getElementsByTagName("script"), function(elt) { > let relocatedScript = document.createElement("script"); > relocatedScript.type = "text/javascript;version=1.8"; > relocatedScript.text = elt.text; > elt.parentNode.replaceChild(relocatedScript, elt); > }); Please check if with the importChild stuff we still have to do this. >+ let doc_head = document.getElementsByTagName("head")[0]; >+ Array.forEach(snippetsElt.getElementsByTagName("style"), function(elt) { >+ elt.parentNode.removeChild(elt); >+ doc_head.appendChild(elt); >+ }); is this needed for functionality or just for doc validation? I'd probably ignore the latter here

Also I guess you should move the patch to a new bug, this bug is not about fixing innerHTML or styling.

(In reply to comment #30) > Comment on attachment 483755 [details] [diff] [review] > Switch aboutHome.js to use DOM methods > the injecting code should go to its own function, something like > importNodesFromStringToElement(aString, aDestElt) This was quick and dirty code I did in around 5 minutes, just to see if it worked. 1) The feedback I'm really looking for is: Did you try it? Do you have a test page with script and style tags to run it against? It seemed to work for me, does it work for you? 2) I don't know the Fx codebase very well. If it goes in its own function, where does that go and what else might use it? > I'm not sure doing an importNode of another documentElement is sane, imo it > would be better to loop through its children and insert them. That's what importNode does, at least according to the docs. The 'deep' parameter set to true causes a recursive cloning of children: https://developer.mozilla.org/en/DOM/document.importNode > Please check if with the importChild stuff we still have to do this. Yes, <script> tags in snippets fail to work unless that code is there. > is this needed for functionality or just for doc validation? I'd probably > ignore the latter here Yes, moving the <style> tags to <head> is required. Adding it is what allowed styling in my test HTML to work.

(In reply to comment #31) > Also I guess you should move the patch to a new bug, this bug is not about > fixing innerHTML or styling. Filed bug 605157

(In reply to comment #32) > 1) The feedback I'm really looking for is: Did you try it? Do you have a test > page with script and style tags to run it against? It seemed to work for me, > does it work for you? not yet tried, but from what I read around it is a proper (just more verbose) replacement for innerHTML in xhtml, so it should just work fine. > 2) I don't know the Fx codebase very well. If it goes in its own function, > where does that go and what else might use it? in the same file, it's just that it looks like a util, thus it should be a util function in that file. > > I'm not sure doing an importNode of another documentElement is sane, imo it > > would be better to loop through its children and insert them. > > That's what importNode does, at least according to the docs. The 'deep' > parameter set to true causes a recursive cloning of children: I think it's importing the root node and other nodes inside it, while we probably want to skip the root node. thus said, let's move to the other bug.

> the problem is due to the fact this is an xhtml document. That's totally a bug. I'll have a fix in a few minutes here.

Comment on attachment 483755 [details] [diff] [review] Switch aboutHome.js to use DOM methods clearing request, looks like we'll get a proper bug fix to innerHTML :)

Filed bug 605356 to get a staging server set up for the snippet service as implemented so far. FWIW, the repo is here on github: http://github.com/lmorchard/home-snippets-server

A staging server has been set up here: http://snippets.stage.mozilla.com/ If anyone wants access to the admin pages, I can create accounts. Looking into hooking this up to LDAP, but it has its own auth for now.

(In reply to comment #14) > (In reply to comment #12) > > did you enable cross-site requests on your webserver? > > Looks like adding an Access-Control-Allow-Origin: * worked. Not to jump the gun on the security review, but, can we use something other than Access-Control-Allow-Origin: *? The problem is that this creates an information leakage issue where any site can issue an xmlHTTP request and retrieve the info from /1/Firefox/4.0b8pre/20101010030401/Darwin_Universal-gcc3/en-US/nightly/Darwin%2010.4.0/default/default/ Right now that just includes the timestamp of the last generated snippet; however, there is always the possibility that future enhancements could add more meaningful data here that would also be exposed.

Just to clarify, this is only a concern if the information is user specific. If this page always returns publicly accessible, non-user-specific data, then its really not a big deal.

(In reply to comment #39) > Not to jump the gun on the security review, but, can we use something other > than Access-Control-Allow-Origin: *? The requesting page is about:home - that didn't seem to work in the ACAO header. So, I don't think we can use another value there. > The problem is that this creates an > information leakage issue where any site can issue an xmlHTTP request and > retrieve the info from > /1/Firefox/4.0b8pre/20101010030401/Darwin_Universal-gcc3/en-US/nightly/Darwin%2010.4.0/default/default/ > > Right now that just includes the timestamp of the last generated snippet; > however, there is always the possibility that future enhancements could add > more meaningful data here that would also be exposed. As far as I'm aware, this will only ever serve up content specific to that combination of client information. There are no users, no logins, and no personalization

(In reply to comment #41) > > > > Right now that just includes the timestamp of the last generated snippet; > > however, there is always the possibility that future enhancements could add > > more meaningful data here that would also be exposed. > > As far as I'm aware, this will only ever serve up content specific to that > combination of client information. There are no users, no logins, and no > personalization Can we add some comments into the code somewhere explicitly stating that we cannot add user specific data because of this design? At some point someone may have an idea to enhance this with user data and we don't want that to happen without addressing the data leak.

(In reply to comment #42) > Can we add some comments into the code somewhere explicitly stating that we > cannot add user specific data because of this design? At some point someone > may have an idea to enhance this with user data and we don't want that to > happen without addressing the data leak. Hmm, not sure where such comments would work best... if we wanted to enhance this with per-user data, I think it would be a pretty serious redesign of the thing. It's built to cache very aggressively, keyed only on the URL itself.

Thinking about load testing this thing - anyone CC'd here have input on that? I was thinking about filing a bug asking for a day or so worth of logs from AUS to synthesize a load model. Anyone have any other feedback on this snippet server? Next steps?

Oh, also, is there a way to possibly get the set of known values for all the placeholders in the snippet URL? (eg. from our builds?) I have to imagine we have this data somewhere, but I don't know where to ask http://snippets.mozilla.com/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/ That would be useful for warming up caches, generating static files, etc.

(In reply to comment #44) > Thinking about load testing this thing - anyone CC'd here have input on that? I > was thinking about filing a bug asking for a day or so worth of logs from AUS > to synthesize a load model. > > Anyone have any other feedback on this snippet server? Next steps? Filed bug 606537 to request AUS logs. Tips / suggestions welcome

(In reply to comment #39) > (In reply to comment #14) > > (In reply to comment #12) > > > did you enable cross-site requests on your webserver? > > > > Looks like adding an Access-Control-Allow-Origin: * worked. > > Not to jump the gun on the security review, but, can we use something other > than Access-Control-Allow-Origin: *? The problem is that this creates an > information leakage issue where any site can issue an xmlHTTP request and > retrieve the info from > /1/Firefox/4.0b8pre/20101010030401/Darwin_Universal-gcc3/en-US/nightly/Darwin%2010.4.0/default/default/ > > Right now that just includes the timestamp of the last generated snippet; > however, there is always the possibility that future enhancements could add > more meaningful data here that would also be exposed. FWIW, I filed bug 606555 regarding the Access-Control-Allow-Origin header. It just doesn't look like "about:home" works as an origin there, and there's not really any other combination that enables the about:home page to load content from a snippets server.

(In reply to comment #39) > Not to jump the gun on the security review... Speaking of a security review, when/how would we like to do one? Other than a load test and performance tweaks, I think the basics of this thing are done until/unless anyone has any feature requests. (Pretty quiet on that front - no news is good news?) FWIW, this is a pretty bare-bones Django app, using out-of-box admin pages and login for content management. Any known issues with that setup I can patch up right away?

Coates - that's a question for you. ^^ Beard - what's next? Need some framing for testing/staging/deployment so we can give QA a heads-up.

(In reply to comment #49) > Coates - that's a question for you. ^^ > > Beard - what's next? Need some framing for testing/staging/deployment so we > can give QA a heads-up. Also, since I really just focused on caching and performance, this thing is really light on content features. I expect those will be the first feature request / wishlist items. That's something we should probably think about, with respect to what we want to show on the start page in particular. We can dump almost any HTML into the snippets, but I haven't really put much guessing into what that content will entail. (eg. styling, scripting, etc)

RE: load testing - is filing an IT bug the best way to kick that off? We'd need a stack (of disposable VMs?) including apache, mod_wsgi, python, memcache, mysql, and a caching proxy in front. On my laptop, I threw an hour of peak AUS traffic through sed/awk for a load model, created snippets and rules to match a chunk of unique URLs from the logs, and ran siege [1] on the logs. Most hits came out of the Squid caching proxy I set up, a few hit memcache, and barely any queried the DB unless I changed content and invalidated memcache. I got an average of 570qps over an hour of testing. Next step would be to see how something closer to a real production rig would handle it. [1]: http://www.joedog.org/index/siege-home

(In reply to comment #48) > (In reply to comment #39) > > > Not to jump the gun on the security review... > > Speaking of a security review, when/how would we like to do one? Other than a > load test and performance tweaks, I think the basics of this thing are done > until/unless anyone has any feature requests. (Pretty quiet on that front - no > news is good news?) > > FWIW, this is a pretty bare-bones Django app, using out-of-box admin pages and > login for content management. Any known issues with that setup I can patch up > right away? Please point me to a stage server when its ready. I'll do a few basic checks, but there shouldn't be much to look at from the server side. The admin page is an issue. What can a user do with admin access? Can they change the content sent to the user? Would a compromised admin account lead to a massive compromise of many users? If so then we need to take several steps to lock that down.

(In reply to comment #52) > Please point me to a stage server when its ready. I'll do a few basic checks, > but there shouldn't be much to look at from the server side. Staging server is here: http://snippets.stage.mozilla.com/ http://snippets.stage.mozilla.com/admin/ Source is here: http://github.com/lmorchard/home-snippets-server > The admin page is an issue. What can a user do with admin access? Can they > change the content sent to the user? Would a compromised admin account lead to > a massive compromise of many users? If so then we need to take several steps to > lock that down. That's definitely a description of the content management pages in general. We can restrict certain users' permissions to modify only content and not other users and permissions—but changing the content is the real purpose of the management pages. I've been thinking about tying the authentication to LDAP, but I'd like to know what you have in mind for locking it down. eg. Restrict IPs that can access content management? Other ideas?

Oh, and I have a password for the "admin" user on staging and can provide it, but figured just posting it in the bug here would be not-so-great

I've filed a security request for this review. We can discuss the particular security issues within that bug or within security bugs identified as part of the review. Please add yourself to bug 592431 if you'd like to take part.

What is the standard icon size for these images?

There is no limitation on the size, since the snippets are arbitrary HTML.

laura forrest will be following up shortly with the content plan for the first phase of the snippets service (as well as filing additional bugs as needed), but essentially we're looking at taking and cloning existing snippets that we push to google as a base and maintaining feature parity with what we have now (e.g. centered image + text and inline link with snippet chosen randomly from available set)

This bug is being tracked as part of this set: http://areweprettyyet.com/4/syncPromotion/

The goal here is to ease into this new service and establish feature parity in the short-term. Here's an overview of the content plan: https://intranet.mozilla.org/Marketing/StartPageSnippets#Firefox_4_Snippets_-_Content_Plan The first step includes adding upgrade snippet messaging to Beta 8 browser versions and below shortly after Beta 9 is live. Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=625516

Filed bug 625750 (confidential) to try to get load testing to happen.

Bug 609105 is the security review bug. There is only one outstanding item (bug 610021). This bug is regarding how the page is deployed to prod - so don't block deployment on this behind resolved (i.e. circular block). The main issue is that we need the production deployment to not publicly serve the /admin page. It can only be accessible to internal or vpn users.

Les, want me to start filing bugs on the UI (https://wiki.mozilla.org/Firefox/Projects/Firefox_Start#Design_Specification)? Are we expecting any visual-treatment changes from the UX team, still?

(In reply to comment #63) > Les, want me to start filing bugs on the UI > (https://wiki.mozilla.org/Firefox/Projects/Firefox_Start#Design_Specification)? > Are we expecting any visual-treatment changes from the UX team, still? Not sure which UI you mean... there's about:home and there are snippets from bug 630359 in staging. I don't have much control over about:home in general, but I can edit snippets

This service is now live. I'll say that satisfies the request to create the service. So, I'll mark this fixed to prevent the comment thread getting any longer. Please file additional bugs if any issues with the service arise.

Quick question about the existing snippet service: let's say we wanted to broadcast an original snippet every day, and have some of them target specific calendar days in specific locales, would the service work for that type of usage, or would we want to add additional features for that type of workflow?

(In reply to comment #66) > Quick question about the existing snippet service: let's say we wanted to > broadcast an original snippet every day, and have some of them target > specific calendar days in specific locales, would the service work for that > type of usage, or would we want to add additional features for that type of > workflow? This is probably worth a new bug to discuss this, since this one is closed. But, quickly: By "existing snippet service", do you mean the new one we're using with Firefox 4+ or the old one hosted by Google before Firefox 4? It's doable in the new one; not sure about the old one. For the new one, we'd target by locale on the server and push out content to pre-load, say week or month worth. Then, use JS on the client side to reveal snippets on the appropriate days based on the local clock. FWIW, the mechanism for delivering snippet content is heavily cached on both the server and client sides, such that it can take up to 48 hours for changes to be reflected. So, best to push lots of content out early and control visibility in JS.