James Boston's Code Blog

New patch:
http://jamesboston.ca/patches/patch112808.txt

I've figured out how to move my unit test into the tree along with a few simple programs that are used by the test. There is an existing directory for unit tests at xpcom/tests/unit and that's where test_nsIProcess.js can now be found. I also created a new directory, xpcom/tests/simple-programs, that contains the source for, well, simple programs. When the tree is built, they get compiled into executables that I can use to test the functionality of nsIProcess. Links to the executables appear in obj-ff/dist/bin which is convenient for me.

Now it's just a matter of polishing the actual unit test code to make sure it covers every edge case. At the moment, it simply runs through all the functions to see if the return codes make sense.

One of the issues I've had with creating a unit test for nsIProcess is figuring out the full path to my unit tests within the build system.

The answer is much simpler than I expected.

This will return /home/james/mozilla/src/obj-ff/dist/bin which is exactly what I need.

The other issue is building a few simple executable files at compile time. I need binaries that do things like exit, never exit, and crash. (These are actually the files I need the full path for.)

Ted Mielczarek has pointed me in the right direction with regard to makefiles (which are my nemesis). Just write a TestProgram.cpp and then set SIMPLE_PROGRAMS = TestProgram in the makefile.

The only other stumbling block left is figuring out where the unit tests should go in the tree. I've been using the example file in the devmo tutorial. There is probably more makefile pain ahead.

I've consulted with Wan-Teh Chang, the NSPR module owner, regarding Unicode support. Apparently, Unicode support has been added to NSPR's file I/O functions (to support Unicode file pathnames). However, the new functions have only been implemented on Windows. Moreover, they are conditionally compiled only if the MOZ_UNICODE macro is defined.

Searching for that macro leads to functions for opening files and directories. I'm not sure that's what I'm looking for or if it's useful to what I'm doing. I'm a bit confused, because I know already that nsIFile (or nsILocalFile) supports Unicode for paths and filenames. And I have no idea what to do about Unix and Unicode. That's actually a much bigger problem in nsIProcess because Mac and Linux both call PR_CreateProcess, whereas I have the option of avoiding that with Windows with existing code.

Wan-Teh indicated that a Unicode variant of PR_CreateProcess is something that could make it into the tree.

New patch:
http://jamesboston.ca/patches/patch111108.txt

After discussions with Benjamin Smedberg, Mark Finkle, Jason Orendorff, and Ted Mielczarek (among others) about Unicode, I've decided to decouple nsIProcess from the Netscape Portable Runtime, at least as far as process creation goes. I may still be able (need?) to use it for piping between processes. Benjamin brought home to me how long it would take to get Unicode support into the NSPR. Unfortunately, my approach to managing processes depends on the NSPR. So that approach is out. Unicode in the NSPR will have to be my next project.

Currently, nsIProcess only uses the NSPR to start non-Windows processes. It doesn't support Unicode at all. My plan is to port the Unix processes creation stuff over from the NSPR into nsIProcess.

I've gone back to an early patch I wrote to fix the kill method under Windows without using the NSPR, so that's back in. I've also merged code for Unicode support from an extension written by dafi:
http://dafizilla.wordpress.com/2008/10/08/nsiprocess-windows-and-unicode/

So with this patch Windows users at least can start and stop processes and use Unicode arguments. I've tested this with some Aramaic and it works.

The run method is now defined in xpidl to take wide characters like this:

In practice this translate to:

The method that assembles the command line looks like this now:

Since my approach to fixing nsIProcess has been to take advantage of code that alraedy exists in the Netscape Portable Runtime, I've been hunting in there for something that might be useful for inter-process communication. What I need is something for capturing the stdout of a process or for sending stuff to its stdin. Well, it looks like the NSPR has an API for i/o redirection.

nsprpub/pr/include/prproces.h#59

nsprpub/pr/include/prio.h#1891

The PRProcessAttr* appears to be the key here. If you look at the implementation for PR_ProcessAttrSetStdioRedirect you can see that that PRPRocessAttr* contains the information about piping:

nsprpub/pr/src/misc/prinit.c#520

Although PRProcessAttr* is the fourth term in the PR_CreateProcess signiture, it isn't used in the nsIProcess invocation:

xpcom/threads/nsProcessCommon.cpp#371

I think I can use the NSPR API to create pipes and then pass them to PR_CreateProcess as a PRProcessAttr*. In this way I can expose i/o piping for processes in nsIProcess.

There isn't any documentation for this that I can find, but someone has been good enough to write some unit tests for this stuff that gives me an idea of how it works:

nsprpub/pr/tests/pipeping.c#119

I think this stuff may have been created with file i/o in mind, but I'm not sure that matters.

I had a chance to test the latest patch on Linux. The same solution works for both Mac and Linux. That means that the run and kill methods of nsIProcess now work across three platforms as described here:
developer.mozilla.org/En/NsIProcess.

But the way I return the PID is just too hacky to make it into the tree. My approach assumes too much about the internal stucture of a PRProcess type that can't be verified at compile time or run time. It's a brittle solution. Benjamin Smedberg, the guardian of XPCOM, recommends re-implementing PRProcess for my needs.

What I really need to do is publish the nsIProcess2 API design I promised several weeks back. It would be easier to figure out what needs to be re-implemented and get advice if I had an over-arching design to guide me.

New patch:
http://jamesboston.ca/patches/patch102808.txt

This should now allow nsIProcess to return the PID of a new process on Mac as well as Windows. I haven't tested this on Linux yet, but I suspect this patch will also work there.

The reason is that Mac processes are started here using the _MD_CreateUnixProcess function:
md/unix/uxproces.c#526

I think all those Mac directories in the NSPR are for OS 9. When you follow the process creation in a debugger you end up in Unix land, which suits me fine. Hurray for Darwin.

Here's my latest patch:
patch102208.txt

This new patch prevents the init method of nsIProcess from being re-used until an existing process has been killed. I've done this so that a reference to an existing process is not lost. If it is lost, it can't be killed. (Note to self: how does this affect blocking processes?)

It also implements returning the PID of a created process but only on Windows. The solution is a bit hacky.

After some experimentation, I've come up with a way to get the PID of the process started by nsIProcess, but it's as ugly as I thought it would be:

Imagine repeating something like that for each platform. I could put the struct definition into the header file, but that's just moving the ugly around. And as I blogged earlier, I'm still not convinced there is even a need to return the PID.

Regarding my xpcshell testing, or lack thereof, I think maybe I'm on the wrong track with creating return values. The methods have errors that propagate across the XPConnect divide. For instance the run method has the error NS_ERROR_FILE_EXECUTION_FAILED. I think I can catch those in javascript.

I've been continuing my digital archaeology of nsProcessCommon.cpp at bonsai.

I think I may not have as much to worry about as I thought. For instance, the fixes for Mac don't seem to apply anymore. Using the NSPR I've been able to open Mac process with and without arguments. For Windows, the only patch that worries me is one for bug 80383. This is where all the windows code was original added in. It looks like it solved a problem with XPInstall using nsIProcess. And the patch was tested on WinNT and Win95! Good grief. I'm not sure any of this applies anymore. Most of the XPInstall API is deprecated in Gecko 1.9 code.

I found a little bit of history while searching for info about this. It looks like a draft proposal for the nsIProcess interface from Dec 2000:
http://www.mail-archive.com/mozilla-xpcom@mozilla.org/msg00072.html

In my nsIProcess code I want to do something like this to return the pid of a value which is stored somewhere inside of PRProcess* mProcess:

But I can't. The code in nsProcessCommon.cpp is not aware of the internal stucture of mProcess. If I've unwound the macros correctly, a PRProcess is actually this on Windows:

The code I'm working with has no idea what HANDLE and DWORD are. I don't think there's a way to do this without some ugly code. Moreover, the NSPR doesn't have any methods for returning pids. That's probably because there is no actual need to do so. As long as you have mProcess to pass to the NSPR, you can control the process.

I'm thinking of changing this:

to this:

where isRunning is a true or false. Am I just lazy? Right now the return value is always zero. Surely this is an improvement.

I've got to say, the hardest thing about Mozilla code is the macros. Trying to figure out what PRProcess or other types actually represent across multiple platforms requires unwinding a lot of defines.