Trying to get the PDXR project moving again. The team met with David Humphrey last week and we meet again ourselves to map out a few things. The goal is create enough documentation that it could be implemented by another team (although I hope it won't come to that.)
Samer and I put together a simple diagram of the project architecture so that we could have some sort of model to which to refer as we try to describe the what the project is:
I'm going to try and elaborate on the diagram here a bit.
Mozilla-Central is the main repository for Mozilla source code. This is the raw fodder out of which documentation will be created. How it is created doesn't really appear on our diagram unless you count the "How does this happen?" note. Well, how it happens is through a static analysis tool called Dehydra. In layman's terms, Dehydra reads the Mozilla source code and annotates it in a way that can be turned into documentation in an automated fashion. This part is another project entirely and as it relates to our project, the relevant work is being done by David Humphrey. However, our team will need to setup the Dehyra tool to feed our update server. (More on that in a moment.) David has assured us that this part is not a huge job. It might be useful to sit down with him sometime in the near future and get our own development setup running to play around with and use for research. (David has a setup, but we wouldn't want to muck it up.) For now, let's just take it as read that the Mozilla source code has been processed and fed to the update server.
So what it the update server? Although this project is all about building a tool for browsing documentation off-line, that documentation is in a constant state of flux. It would be useful to be able to refresh it when need be. The update server needn't be terribly complex. A simple Apache server and some cgi work should suffice. However, it needs to be worked out exactly what kind of update is being pushed. Is it a binary diff of the sqlite database? And if so, will it need to keep a collection of diffs based on a original dataset in order to service users who haven't sync'd for some time. There's also the question of whether the update mechanism build into Mozilla for extensions is useful to us. We need to do some research on how extension updates work.
Now we come to the heart of the project. The extension. As our diagram shows, the Portable DXR (our browsable source code documentation) is an application that lives inside our extension. The diagram should probably be revised a bit to show that the cgi tools (Perl or possibly Python support) exist as part of an embeddable web server. When you remove the DXR part, what you are left with is the platform on which it runs. It's also worth pointing out that the sqlite support is available in two ways. Local storage capabilities using sqlite are build into Mozilla applications and can be accessed by an extension using C++, javascript (and I believe Python). However, sqlite database files can be accessed by any software that understands the format, so sqlite support will also be available through the cgi tools. The DXR application will probably use the cgi tools to access sqlite files if our presentation layer is traditional XHTML. But if we decide to use XUL, we can use the browser's built in functionality directly from JavaScript. In either case, I suspect we will end up leveraging the built in sqlite support for the updates. But we need to do some research on that.
The last part is the the presentation of the documentation, the DXR proper. David has a something now that uses XHTML, Ext-JS for the front-end. We could port that to the extension, but there would need to be modifications for sure. For instance, David's DXR doesn't give the user the option of updating. If we are combining this with Prism and XULRunner, the only UI elements the user will see will be in the displayed page. So we would need to include UI elements there ourselves.
I've been reading some blog posts from Mark Finkle about mixing XUL and HTML. I think that may be one way to go. Also, we would want to take advantage of Samer's jquery skills.
I haven't said anything yet about how one embeds a server in an extenions. I already have a proof-of-concept extension right now. It uses SHTTPD and supports Perl and PHP for cgi. However, the browser runs as a separate process from the browser and this could lead to problems where one is running and not the other. The plan is to find a suitable light weight server and compile it as a binary extension to the browser. We won't have to write the server from scratch, but there will be considerable work finding a suitable server that we can hack to work as a binary component on Windows, Mac and Linux. And then there is the not-trivial task of getting the cgi tools to run across all three platforms.
Probably our next step is to do some research. Actually, if we are doing this in a proper project management way, we need to map out how we going to do the research, how long it should take, who is going to do it, and so forth.
So that's kind of where the project is right now.
