Let's JusTalk to the Web

Description

This is an experimental vocal web driver. If it works for you please tell others, if you have difficulty please tell me!

Accessibility software is normally directed at making blind and partially sighted users, and those with difficulty in using keyboards, into using an interface for dextrous, sighted people.

By pressing Ctrl-Space, this extension records an utterance, such as "click on okay" and operates the page as if it has been instigated by the mouse. This extension currently supports the notion of page navigation, clickable widgets, i.e. links and buttons - checkboxes and radio buttons next; and, value holding widgets, i.e. input text boxes and text areas.

Typical supported commands are as follows. The most up to date description is provided by the extension itself - just ask, "how do i interact with this page"
i) navigation commands:
go to X.Y.Z
go back
go forward
click on the X link
ii) page description:
describe the page
do you have X (where X may be "a date button")
iii) page interaction:
click on X (where X may be "X" or "the X button")
set the value of X to Y
get the value of X

Following speech act theory, all utterances are supported by a positive reply, such as "okay, the date button has been clicked", or "sorry, I do not understand".

This extension is EXPERIMENTAL. It also supports simple lists of instructions, i.e. unnamed functions, aka lambdas, such as "set the value of name to Martin and then click on okay". The eventual aim of this extension is at introducing all the Enguage(tm) software for Android onto the web (see video below), and thus interfacing with software purely through voice.