too many tangents

I’m having a wonderful time delving into lots of bits of AROS that I haven’t seen before. What started as simply wanting to make IN: work has lead me into the depths of the shell and beyond.

As discussed, I’ve implemented a new system call:

LONG Pipe(CONST_STRPTR name, BPTR *reader, BPTR *writer);

It returns two pipes attached to the same “thing”, denoted by name. I expect that this will be a simple volume specifier with no details (eg PIPE:), but could potentially have console-like options attached to it (say PIPE:4096 to set the buffer size). It could conceivably also take a full path for use with named pipes. Whatever, its mostly there to allow flexibility for the future.

Underneath, I’ve defined a new IOFS action and a new packet:

#define FSA_PIPE 45
struct IFS_PIPE {
    STRPTR       io_FileName;
    struct Unit *io_Writer;
};

/*
LONG                      (dp_Res2)
ACTION_PIPE(BPTR lock,    (dp_Arg1)
            STRPTR path,  (dp_Arg2)
            BPTR reader,  (dp_Arg3)
            BPTR writer); (dp_Arg4)
*/
#define ACTION_PIPE 1800

Of course, only the IOFS version is used so far, and I haven’t implemented the translation in packet.handler yet. I’ll probably wait until the time that its needed - its about eight lines.

I’ve modified pipefs.handler to handle this new action, and its doing well. You call it with just PIPEFS: as the name, and it creates a single pipe, two handles on it, and returns them. When both handles are closed, the pipe disappears.

Next, I taught the shell about it. Its internal Pipe() function which Open()’d PIPEFS:__UNNAMED__ and did some duplication and mode-changing magic is now gone, replaced by a call to the new system one. To test, I wrote a tiny program called minicat that acts like Unix’s cat - opens the named file and puts it onto standard output, or if you don’t specify a file, reads from standard in:

int main(int argc, char **argv) {
    BPTR in, out;
    char buf[256];
    LONG len;

    if (argc > 1) {
        if ((in = Open(argv[1], MODE_OLDFILE)) == NULL) {
            Fault(IoErr(), "minicat", buf, 255);
            fprintf(stderr, "%s\n", buf);
            return 1;
        }
    }
    else
        in = Input();

    out = Output();

    while ((len = Read(in, buf, 256)) > 0)
        Write(out, buf, len);

    if (argc > 1)
        Close(in);

    return 0;
}

Running minicat somefile.txt | minicat does exactly what you’d expect. minicat somefile.txt | minicat IN: does the same thing. This confirms it - pipes are working, as are the standard streams. Hurrah!

Something I did notice when watching the pipefs.handler debug output is that when using a shell pipe, the shell actually seems to be closing a side of the pipe that it already closed. I haven’t looked into it in depth, but it seems that it closes both halves of the pipe when one of the commands complete, but it spawns the second command with PRF_CLOSE* flags so it tries to close the pipe on shutdown. It can be plainly seen in the pipefs output as the usage count of the pipe drops to -1. Of course at that point the pipe doesn’t even exist any more, and memory has been freed. I can only assume that its the lack of memory protection that has allowed this to go unnoticed for so long. I’ll dig down into that a little this afternoon.

And the point of all this hacking, if you remember, was to make it so that Type something.txt | More IN: would work. Well, after all this, it doesn’t. From what I can tell, it never could have worked, because of the incorrect way it allocates its internal buffers. It tries to allocate enough memory to hold the entire file in memory, but if the file isn’t a “real” file (ie its a console), then it just allocates a 64KB buffer instead:

    if (IsFileSystem(filename)) {
        Seek(fh, 0, OFFSET_END);
        new_filelen = Seek(fh, 0, OFFSET_BEGINNING);
    }
    else
        new_filelen = 0x10000;

The problem here is that PIPEFS: is a filesystem (it has directories, named files, etc) and returns 1, but files aren’t seekable. Thus, new_filelen becomes -1, which causes an error further down, and the program aborts.

The right way to do is is to test if the handle is seekable. If it is, then More should read the files in chunks and Seek around as the user moves through it. If its not then the only option is to read it from start to finish, so More should maintain its own in-memory buffer, growing it as necessary. That is more than a trivial change though and I’ll need to study the More code in depth before deciding whether its something I want to work on right now.