despairlabs :: blog

a 'complete' success

I gave ellie a demo of the Midnight clone the other day, and one of the first things she did was to try and do a tab complete in the little command-line interface. It was immediately obvious that any modern command-based interface needs completion and history, so I set out to find a way to provide it.

A quick CPAN search didn’t really uncover anything. A couple of the Term::Readline variants claim to have support, but the interface seemed rather clunky (reasonable, since it comes from C). I use and love IO::Prompt, largely because of its trivial interface. The answer became clear - IO::Prompt requires tab completion.

The whole thing took three evenings to implement. It was pretty straightforward. I started by adding support for a -complete option, which took a list of possible values. After that it was just a case of hooking up the tab key, comparing the current input to the values in the list, and replacing part or all of it with the matched value. This worked wonderfully well, and did great things for my confidence - I’m always a little unsure if I’m doing something the correct way when I go to work on someone elses code - particularly when its written by a known hero like Damian Conway :P

Adding the characteristic “beep” when you only get a partial match was next. Trivial, of course - just emit ASCII code 0x7 at the proper time; the terminal takes care of the rest. A -bell option went in alongside - I’m a firm believer in being as flexible as possible.

The next hallmark of tab completion is displaying possible values when there is more than match for the current input. Since I already knew about the available matches, its no effort to print them out, but it wants to look nice too. A little column calculator went in to make things pretty. I also added a “show all <N> matches?” prompt when its likely that showing them all will scroll your terminal. Obviously, getting a prompt is no problem (this is a prompting module, after all :P ), but I also found that prompt() is totally reentrant - it doesn’t restore the previous terminal settings when it exits, opting instead to return it to “cooked” mode. I haven’t looked in any depth, so I don’t know if IO::Prompt or Term::ReadKey is at fault here. Either way, it caused tab and other keys to not be detected correctly after the “show all” prompt. The workaround was to simply chuck the terminal back into raw mode, and it coped nicely.

So that just about finished it, but then while writing examples I started to realise that the whole thing was actually pretty useless. The reason: the vast majority of command line inputs are actually a set of individual and sometimes unrelated fragments. Think about completion for files. If I had to provide an array of every possible file that could be chosen in a given situation, I’d have to provide a list of every file on the system, each with full path. Obviously, this is ridiculous. What’s wanted is a way to complete only portions of the input line, with the possible values selected by looking at the surrounding context. More than a humble array can achieve. What was needed was a callback.

I made it so that -complete could handle a coderef as well as an array - the simple array code might still be usable in places, and its certainly easier to understand. I figured it would be enough to simply pass the current input to the callback, and have it look at the contents and return a list of possible values based on it.

This worked, but had problems. The callback code was pretty complex, and when there were multiple possible values, displaying them was awful, because my code only knew how to complete entire input lines, not fragments of lines. So the callback would have to return the full input line with each possible outcome. Perhaps an example: we’re writing a program that has to load files, using a “load /path/to/file” command. We want to do shell-style completion for the file/path portion of the input. To work correctly, the callback has to look for the “load " at the start, then split up the file path, look inside the directory, and return any files there. But, it has to return the full input line, so it returns something like:

As well has having piles of redundant information, if my code were to display them, it would show the entire lines, when it should have just shown foo, bar and baz (just like your shell). Obviously, IO::Prompt needed to be smarter - it had to understand fragments and do the splitting itself.

This actually took me about two days of thought to figure out - the bus to work is a great place for pondering. The solution was to have prompt() split the input and pass all of those to the callback, and only do completion on the final item in the split. So in the above example, the callback would return qw(foo bar baz), and thats that. A -split option was added that takes a regex to be passed to Perl’s split builtin.

Implementing this took quite a bit of internal gymnastics to achieve because I was having to essentially write the same code for two cases (full line and split fragments). Generalising aspects of the code (mostly the list matching and the inpup replacement code) was proving quite finnicky, until I made the leap of logic that told me I’d made the correct choice with the interface. If we’re trying to complete for a full line, that line becomes a split with no delimiting character, and one fragment.

Five minutes later, it was done. And the callback is trivial. The most complex callback you’ll ever likely write for this sort of thing is one to do file paths, because of the special cases - there’s files and subdirs. I included that in the examples. It weighs in at just eight lines.

All in all, a roaring success. Damian now has the patch. Haven’t heard back yet, but I’ll see him next week so I’ll hassle him then if necessary.