Talking out of the box V
By esj on Friday 11 March 2011, 02:01 - Permalink
In the previous two blog entries, we discussed two important concepts, name transformation and landmarks. In this blog entry, we'll discuss how to use these concepts in a practical application.
In the code editing application, you need to be able to identify landmarks (natural language grammar), transport the landmark to a special external tool and then on return, replace the landmark with the chosen symbol. A very crude mock up of the sequence is shown in the image below: This simple example turns out to be a useful tool in building out a full speech driven programming environment. In its current form, it's good for bootstrapping. It gives the disabled programmer the tools necessary to deal with the hardest part of working with existing code: its symbol space. Some enhancements include:
- increasing the number of visible cache entries
- plug-ins for user defined symbol transformations
- recursive invocation (i.e. selecting a method used as an argument within an argument)
- predictive symbol completion
Beneath this little application hide a host of interesting problems and user interface features. The first major feature is splitting the user's attention away from the main application into a separate frame of reference. This allows break attention or focus from the task they had been doing two new task of generating symbol. It's going to take a bit of research to find out if the shifting focus is truly a good idea or not. On the right-hand side, the names of the transformation algorithms are found a used as a memory jog for the user.as to what to say when selecting a symbol to be returned. As an alternative, numbering each yield 1-5 could also be used as a clue to what the user should say to retrieve that field.