Let's use this bug to track all changes that need to be made for the site for Golan Levin's app. This phase is scheduled to be released on May 17th.

==Questions== The python app for grabbing similar marks is quite lightweight. Should we run this app in a separate WSGI instance or fork the markup project and implement it in Django? Similarly, should the analysis information from our app be stored in its own (isolated) database? ==Prototyping & Testing== We would like to obtain a .tgz dump of (all?) existing Marks as soon as possible, so that we can test with the Marks produced by the Mozilla MarkUp app. (Currently we are testing with approximately 10,000 GML marks from 000000book.com. We expect there may be statistical differences in the two datasets that require our attention.)

(In reply to comment #1) > The python app for grabbing similar marks is quite lightweight. Should we > run this app in a separate WSGI instance or fork the markup project and > implement it in Django? If it's quite lightweight anyway, I'd suggest implementing it as a django "app" inside markup (i.e., ffdemo/phase2, for example). If, however, the learning curve is too steep for you, I can be convinced otherwise. > Similarly, should the analysis information from our app be stored in its own > (isolated) database? I suggest keeping the data in the same database as the actual app, since (if I understand this correctly) the data will also be shown by the app in the same context as its other data. Again unless I am mistaken, your app is not overly database-heavy, correct? > ==Prototyping & Testing== > > We would like to obtain a .tgz dump of (all?) existing Marks as soon as > possible, so that we can test with the Marks produced by the Mozilla MarkUp > app. (Currently we are testing with approximately 10,000 GML marks from > 000000book.com. We expect there may be statistical differences in the two > datasets that require our attention.) That's an IT bug. Will a DB dump of the markup_mark table do? The database contains the marks as BASE64-encoded data. If you needed it, say, as separate .gml files, you'd need to drop that into a DB and write a little script accessing it, then de-base64ing and writing to disk. Let me know what you're looking for.

Thanks for the reply. This comment contains a new request, for a new function to be implemented. In ffdemo/markup/requests.py there is a method, recent_marks which returns a JSON bundle of the most recent marks. When used, this is found at the URL, requests/recent_marks. In my app, I would like to issue a request for a similar bundle -- but specifically, one which contains *specific* GMLs that I would like to call forth from the database. Having this handler will reduce the overhead of making many individual requests. I expect to request about ~30 marks at a time, using their unique URLs. Thoughts?

(In reply to comment #3) > I expect to request about ~30 marks at a time, using their unique URLs. > Thoughts? All in JavaScript? Running 30 separate HTTP requests at a time is not a good idea. If, however, you're accessing the marks all on the backend, you don't need a URL for this, you can just access the DB directly.

> All in JavaScript? Running 30 separate HTTP requests at a time is not a good > idea. What we're asking for is an interface for fetching multiple marks at once. I could write it into the app and send a pull request if you want, but I didn't want to touch your code without asking first. This should actually reduce the amount of server overhead since most of the computation will occur on the client and most of the data will be contained in static files. In its original conception the app passed marks made by the user to the server for analysis and received the 30 most similar marks in return. Now the analysis occurs in the browser and the user merely asks the server for marks with certain IDs. The only other asset the server needs to serve is the precomputed data file (essentially a big matrix describing the data.) This is computed nightly and can be served statically.

(In reply to comment #5) > What we're asking for is an interface for fetching multiple marks at once. I > could write it into the app and send a pull request if you want, but I > didn't want to touch your code without asking first. Ah yes, all of them at once works. Feel free to add this and send a pull request, if that works for you. > The only other asset the server needs to serve is the precomputed data file > (essentially a big matrix describing the data.) This is computed nightly and > can be served statically. All right!

Resolving won't fix because we will not be moving ahead with this part of the project.