Spoken Language Support for Software Development Andrew Begel

Spoken Language

Support for Software

Development

Andrew Begel

Advisor: Susan L. Graham

Computer Science Division, EECS

University of California, Berkeley

1

Interactive Software Development

Human-Computer

Interaction

Programming

Languages

Human-Centric

Software Tools

Software Engineering

2

Motivation

while (counter < limit) {



}

• Programmers conventionally use keyboard

– Long hours at keyboard leads to higher risk of RSI

• Can speech-based programming be an alternative?

• Combines an unambiguous domain

(programming) with an inherently ambiguous input modality (speech)

– Great for exploring ambiguity handling in a new context

3

Programming by Voice

while counter is less than limit do ...

• My Goal

– Find out how developers use code verbally. Use this to develop a naturally verbalizable input form.

– Build development environment that supports verbal authoring, navigation, modification.

• Extend conventional compiler analyses to support ambiguities generated by speech.

– Learn how developers can use voice-based programming, and iterate design.

4

Challenges

 Programming languages were not designed to be spoken.

 Speech is inherently ambiguous. Programming tools were not designed for ambiguity.

 Speech tools are poorly suited for programming tasks.

 Programmers are not used to verbal software development.

5

Talk Outline

• Introduction and Motivation




• Program Analyses for Ambiguous Inputs

• Program Navigation and Editing

• Conclusion

6

Programming by Voice for (int i = 0; i < 10; i++ ) {

▌

}

7

Current Tools are Awkward!

VoiceCode

[Desilets 2004] for loop … after left paren … declare india of type integer … assign zero … after semi

… recall one … less than ten … after semi

… recall one … increment … after left brace

1

2

3

4 for (▌ ; ; ) {

}

8


VoiceCode



… recall one … increment … after left brace for (int i = 0;▌ ; ) {

}

9


VoiceCode



… recall one … increment … after left brace for (int i = 0; i < 10;▌ ) {

}

10


VoiceCode



… recall one … increment … after left brace for (int i = 0; i < 10; i++ ) {

▌

}

11

Programming by Voice Related Work

Multiple

Tasks

Begel ‘05

Arnold ‘00

Snell ‘00

Desilets ‘01 ‘04

Authoring

Only

Gray ‘03

Computer-Centric

Price ‘00 ‘02

Human-Centric

12

How do Programmers Speak Code?

• 10 programmers read Java code out loud

(Begel ‘05)

• Most programmers spoke the same way

13


for int i equals zero i less than ten i plus plus for (int i = 0; i < 10; i++ ) {

▌

}

14


Awkwardness by Design (Structural)

(int)foo

(3 + 5) * 7

15


Individual Inconsistency

System.out.println

vs.

System out println bar sub i bar of i vs. vs. i from bar

16


Native English speakers vs. non-native speakers (Pronunciation) tur vs. t u r println array[i++] vs .

array[i]++

17

A Natural Way to Code for int i equals zero i less than ten i plus plus for (int i = 0; i < 10; i++ ) {

▌

}

18

Too Many Ambiguities for int i equals zero i less than ten i plus plus

Spelling of ID?

KW or ID?

KW or #?

4 int eye equals 0 aye less then ten i plus plus for (int i = 0; i < 10; i++ ) {

▌

}

19

Sometimes It’s Non-Obvious

for times equals 8 file 2 load times equals one for (times = 8; file(2, load); times == one) {

▌

} fore *= 8; file.tooLode.times = won ▌

4; times = ate(file).to(load).equals(1) ▌

20

Design Tradeoffs

Command

Language

Easy to analyze, but prescriptive


Natural

Language

Flexible, but ambiguous

21

Spoken Java

• Semantically identical to Java

• Syntactically easier to say than Java

– Methodology generalizable to any computer language

1. All punctuation has English equivalents

• Open Brace, End For Loop

2. Most punctuation is optional

3. Provide verbalization for all abbreviations

4. Relaxed phrasing for better fit with English

• (int)foo  “cast foo to integer”

• foo = 6

• foo[i]++

 “set foo to 6”

 “increment the ith element of array foo”

22

SPEED: Speech Editor

• Build an editor that supports naturally verbalized programs

• SPEED : SPE ech ED itor

• Based on IBM ViaVoice, Eclipse IDE, Harmonia

– Spoken Java Language for Composition

– Spoken Command language for Navigation,

Editing, Template instantiation, Refactorings,

Search

– Audible and visual feedback

• Similar to JavaSpeak

(Smith 2000)

23

Harmonia Analysis Framework

• Framework to support interactive editors

– Language-based, programmer-oriented tools

• Incremental analyses

– Lexing

(Wagner ‘97)

, GLR Parsing

(Wagner ‘97, Begel ‘04)

, Static

Semantics

(Garrison ‘87, Begel, Jamison)

• C, Java, Titanium, Cool, Flex, Bison

– Also, languages where indentation and CRs are significant

• Interactive Program Transformations (Java)

• CodeLink

(Toomim et. al.

‘04)

• Shorthand Editing

24

Talk Outline


• Programming by Voice



Program Analyses for Ambiguous Inputs


• Conclusion

25

Traditional Compiler Analyses for (i = 0; i < 10; i++ ) {



}

Programming languages are designed to be unambiguous

Lexical

Analysis

Parsing Semantic

Analysis

For Loop

Local

Var

FOR I FOR Assign Expr i int

I = 0

26

Ambiguity-Aware Analyses for i equals zero ...

Handles input stream, syntactic and semantic ambiguities

Lexical

Analysis

Ambiguous

Parsing

Semantic

Ambiguity

Resolution

Ambig Stmt

FOR I

4 EYE

For Loop Assign Expr

FOUREYE

FOR Assign Expr

= 0



 i four eye

Local

Var

Local

Var int

?

I = 0

27

Scan Input Stream

Commercial

Speech

Recognizer

Homophone

Dictionary

Lexical

Analysis

28

Homophones Cause Ambiguities

4 for i equals fore eye = fou r aye ==

Concatenated words cause them too for i =

4 i equals foreeye == foriequals fore ayeequals foureyeequals

29


Lexical

Analysis

FOR I

4 EYE

XGLR

Ambiguous

Parsing

Ambig Stmt


FOUREYE = 0

FOR Assign Expr

I = 0

Semantic

Ambiguity

Resolution



 i four eye

Local

Var

Local

Var int

?

30

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

31

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

32

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

33

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

34

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

35

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

36

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

37

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

38

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

39

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

40

GLR Parsing

[ Tomita 85 ]

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

41

GLR Parsing

[ Tomita 85 ]

Stmt

Stmt

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

IF

KW

Expr

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

42

GLR Parsing

[ Tomita 85 ]

Stmt

Stmt

IF

KW

Expr

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

IF

KW

Expr

Stmt

Stmt

IF

KW

Expr

FOO

ID

ELSE

KW

BAR

ID

43

GLR Ambiguity Support

1. Fork when there is more than one possible interpretation (tree)

44

XGLR Ambiguity Support

1. Fork when there is more than one possible interpretation (tree)

2. Fork when the lexical input is ambiguous

45

XGLR Parsing

[Begel 04]

IF FIFTY FIVE < X

46

XGLR Parsing

[ Begel 04 ]

IF FIFTY FIVE

KW

< X

47

XGLR Parsing

[ Begel 04 ]

IF

KW

FIFTY FIVE < X

48

XGLR Parsing

[ Begel 04 ]

IF

KW

55 <

#

50 5

# #

50 FIVE

#

FIFTY 5

ID

ID

#

FIFTY FIVE

ID ID

X

<

<

<

<

X

X

X

X

49

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

55

#

50

#

50

#

FIFTY

ID

FIFTY

ID

<

Op

5

FIVE

ID

5

#

#

FIVE

ID

X

<

<

<

<

X

X

X

X

50

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

50

#

50

#

FIFTY

ID

FIFTY

ID

.

.

(

(

<

Op

5

FIVE

ID

5

#

#

FIVE

ID

X

<

<

<

<

X

X

X

X

51

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

50

#

50

#

FIFTY

ID

FIFTY

ID

.

.

(

(

5

FIVE

ID

5

#

#

FIVE

ID

<

Op

5

FIVE

ID

5

#

#

FIVE

ID

X

<

<

<

<

<

<

<

<

X

X

X

X

52

X

X

X

X

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

50

#

50

#

FIFTY

ID

FIFTY

ID

.

.

(

(

5

FIVE

ID

5

#

#

FIVE

ID

<

Op

5

FIVE

ID

5

#

#

FIVE

ID

X

<

<

<

<

<

<

<

<

X

X

X

X

53

X

X

X

X

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

50

#

50

#

FIFTY

ID

FIFTY

ID

.

.

(

(

5

FIVE

ID

5

#

#

FIVE

ID

<

Op

5

FIVE

ID

5

#

#

FIVE

ID

X

<

<

<

<

<

<

<

<

X

X

X

X

54

X

X

X

X

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

.

(

(

FIVE

ID

5

#

FIVE

ID

<

Op

<

<

<

X

X

X

X

55

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

(

(

5

#

FIVE

ID

(

.

FIVE

ID

.

.

FIVE

ID

(

FIVE

ID

.

)

)

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

.

(

(

(

FIVE

ID

FIVE

ID

5

#

FIVE

ID

(

IF

KW

55

#

<

Op

<

<

<

<

<

<

<

<

<

X

X

X

X

X

X

X

X

X

X

56

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

(

(

5

#

FIVE

ID

(

.

FIVE

ID

.

.

FIVE

ID

(

FIVE

ID

.

)

)

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

.

(

(

(

FIVE

ID

FIVE

ID

5

#

FIVE

ID

(

IF

KW

55

#

<

Op

X

X

X

X

X

X

X

X

X

57

<

Op

<

Op

<

Op

<

Op

<

Op

X

ID

<

Op

<

Op

<

Op

<

Op

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

(

(

5

#

FIVE

ID

(

.

FIVE

ID

.

.

FIVE

ID

(

FIVE

ID

.

)

)

<

<

<

<

<

Op

Op

Op

Op

Op

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

.

(

(

(

FIVE

ID

FIVE

ID

5

#

FIVE

ID

(

<

<

<

<

Op

Op

Op

Op

IF

KW

55

#

<

Op

X

ID

X

X

X

X

X

X

X

X

X

58

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

.

(

(

(

(

5

#

FIVE

ID

FIVE

ID

5

#

FIVE

ID

<

Op

)

)

<

Op

<

Op

<

Op

<

Op

<

Op

X

ID

X

X

X

X

X

59

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

.

(

(

(

(

5

#

FIVE

ID

FIVE

ID

5

#

FIVE

ID

<

Op

)

)

<

Op

<

Op

<

Op

<

Op

<

Op

X

ID

X

ID

X

ID

X

ID

X

ID

X

ID

60

XGLR Parsing

[ Begel 04 ]

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

FIFTY

ID

55

#

.

(

(

(

(

5

#

FIVE

ID

FIVE

ID

5

#

FIVE

ID

<

Op

)

)

<

Op

<

Op

<

Op

<

Op

<

Op

X

ID

X

ID

X

ID

X

ID

X

ID

X

ID

61

XGLR Parsing

[ Begel 04 ]

IF

KW

FIFTY

ID

(

5

#

)

<

Op

X

ID

IF

KW

FIFTY

ID

(

FIVE

ID

) <

Op

X

ID

IF

KW

IF

KW

IF

KW

FIFTY

ID

FIFTY

ID

FIFTY

ID

(

.

(

FIVE

ID

5

#

FIVE

ID

IF

KW

55

#

<

Op

<

Op

<

Op

<

Op

X

ID

X

ID

X

ID

X

ID

62

XGLR Parsing

[ Begel 04 ]

FuncCall

IF

KW

<

Op

X

ID

IF

KW

IF

KW

IF

KW

IF

KW

IF

KW

FIFTY

ID

(

5

#

Expr

)

FuncCall

FIFTY

ID

(

FIVE

ID

Expr

)

Expr

FIFTY

ID

FIFTY

ID

FIFTY

ID

(

.

(

FIVE

ID

5

FIVE

ID

Expr

#

55

#

<

Op

<

Op

<

Op

<

Op

<

Op

X

ID

X

ID

X

ID

X

ID

X

ID

63

XGLR Summary

• Generalization of traditional GLR algorithm

– Forks on structural and lexical ambiguity

– Preserves subtree sharing when parses have different yields

– Preserves efficiency when parses get out of sync

• Determine parse position w.r.t. ambiguous input

• Blender: Combined lexer and parser generator for XGLR

64

GLR Parsing Genealogy

Tomita

1985

Farshi

1991

Rekers

1992

Scannerless

Visser

1997 van den Brand

2002

Wagner

1997

Begel

2004

Incremental

Input Stream

Ambiguities

Johnstone et. al.

2002

65


Lexical

Analysis

FOR I

4 EYE

XGLR

Ambiguous

Parsing

Ambig Stmt


FOUREYE = 0

FOR Assign Expr

I = 0

Semantic

Ambiguity

Resolution



 i four eye

Local

Var

Local

Var int

?

66

Disambiguation Example class Loader { public void load() {

String filetoload = null;

InputStream stream = getStream();

...

▌

}

} file to load equals stream dot read string filetoload = stream.readString();

67

Many Interpretations

 file.toload

 (file, 2, load)

 file.to(load)

 file.to.load

 file(to, load)

 file(to.lode)

 file(to(lode))

 file(toload)

 file(2, load)

 filetoload()

 filetoload

68

Incremental Semantics

• What does this name mean?

• What names are visible at this program point?

– Or, What can I say here?

• Visibility Graph

[Garrison 1987]

– Incrementally updated data structure for scopes, names and bindings

– Designed Visibility Graph algorithms for name propagation and incremental update

– Used for type checking, too

• Doesn’t <insert favorite IDE here> do this?

69

Program Context Can Help class Loader { public void load() {



...

▌

}

} class Loader scope

[ load, Method, ()  void ] method load scope

[ filetoload, LocalVar, String ]

[ stream, LocalVar, InputStream ]

70

Program Context Can Help class Loader { public void load() {



...

▌

}

} class Loader scope




[ load, Method, ()  void ]

71

Semantic Disambiguation class Loader scope





 file.toload

 (file, 2, load)

 file.to(load)

 file.to.load

 file(to, load)

 file(to.lode)

 file(to(lode))

 file(toload)

 file(2, load)

 filetoload()

 filetoload

72






Is “

file

” a visible

variable

name?

 file.toload

 (file, 2, load)

 file.to(load)

 file.to.load

 file(to, load)

 file(to.lode)

 file(to(lode))

 file(toload)

 file(2, load)

 filetoload()

 filetoload

73






Is “

file

” a visible

method

name?

 file(to, load)

 file(to.lode)

 file(to(lode))

 file(toload)

 file(2, load)

 filetoload()

 filetoload

74






Is “

filetoload

” a visible

method

name?

 filetoload()

 filetoload

75

Manual Disambiguation

• Some ambiguities cannot (and should not) be automatically resolved: print(“line”) vs.

println() if (pred1) then if (pred2) then foo() else bar() if if if if bar() foo() bar() foo()

• If ambiguities remain, ask the user how to resolve them. (e.g. [Mankoff 00])

76

Talk Outline






Program Navigation and Editing

• Conclusion

77

Study - Navigation by Speech

1. Eight navigation tasks with commercial VR tools

2. Search through text (w/o Find dialog), e.g.

Find the sentence where Romeo cries out about his fate after killing Tybalt.

Metrics

1. Time to scroll to right page

2. Number of commands

3. Number of recognition errors

4. Number of system mistakes

Results

1. Cognitive load is too high too many commands, misestimation errors

2. Voice recognition induces too much delay/errors for accurate control

78

SPEED Navigation

• Shorthand Editing class Loader { public void load() {


String filetoload = null; filetoload = “file.txt”;

▌

}

} class MenuLoader extends Loader {

...

}

} public void save() {

...

79


• Shorthand Editing class Loader { public void load() {



▌

}


...

}


...

80


• Shorthand Editing down class Loader { public void load() {



▌

}


...

}


...

81


• Shorthand Editing protected class Loader { protected void load() {



▌

}


...

}


...

82


• Shorthand Editing private class Loader { private void load() {



▌

}


...

}


...

83


• Shorthand Editing down class Loader { private void load() {



▌

}


...

}


...

84


• Shorthand Editing push down class Loader { private void load() {


InputStream stream = getStream(); filetoload = “file.txt”;

▌

}


...

}


...

85



• Context-Sensitive

Mouse Grid

[Begel]

– Related Work: Tree

Hierarchy Navigation

[Smith 2004] class Loader { private void load() {



▌

}


...

}


...

86


1



Mouse Grid

[Begel]



[Smith 2004]

2 class Loader { private void load() {



▌


...

}


...

}

87




Mouse Grid

[Begel]



[Smith 2004]

1

1 class Loader { private void load() {



2 filetoload = “file.txt”;

▌


...

}


...

}

88




Mouse Grid

[Begel]



[Smith 2004]

1 1 class Loader {

2

4 private void load() {

1 String filetoload = null;


3 filetoload = “file.txt”;

▌

}

}


...


...

89




Mouse Grid

[Begel]



[Smith 2004]

1 1 3 select class Loader { private void load() {



▌


...

}


...

}

90

SPEED Editing



Mouse Grid

[Begel]



[Smith 2004] edit class Loader { private void load() {



▌


...

}


...

} filetoload equals quote file.txt quote

91

SPEED Editing



Mouse Grid

[Begel]



[Smith 2004] edit class Loader { private void load() {



▌


...

}


...

} file to load equals stream dot read string

92

SPEED Editing



Mouse Grid

[Begel]



[Smith 2004] put it back class Loader { private void load() {


InputStream stream = getStream(); filetoload = stream.readString();

▌


...

}


...

} file to load equals stream dot read string

93

Study - SPEED Usability

Goal: Understand how SPEED can be used by expert programmers

1. Train expert Java programmers on SPEED

2. Author new code

– Build a Linked List data structure with associated algorithms

3. Modify existing code

– Change abstraction representation and update algorithms

94

Metrics

• User Metrics

– Speed

– Vocabulary mistakes, Forgotten vocabulary

– Grammatical mistakes, Grammatical substitutions

• SPEED Metrics

– Word tokenization errors

– Disambiguation errors

– Irresolvable ambiguities

95

Talk Outline







Conclusion

96

Contributions

1. A study of programmers to understand and design a naturally verbalizable input for programming

2. An interactive editor designed for spoken interaction

3. The use of syntax and semantics of programming for disambiguation

– Enhanced lexical, syntactic, semantic analyses for support of verbal ambiguities

4. Evaluation of design and tools by studying programmers using voice for software development

97

Future of Programming by Voice

1. Improved automation of semantic disambiguation

– Use ideas from NLP, Machine Learning (team styles)

2. Early pruning of ambiguities using analysis feedback

3. Higher-level linguistic programming tools

– Transformations, Paraphrasing

– Phonetic search, Audible feedback

4. Support more software engineering tasks by voice

– Debuggers, IDEs, Comments, Code reviews

5. Design spoken variants of other formal languages

– General (C, C#) Scripting (PL, OS), Design (HCI),

Command (Robotics), Domain-specific languages (SQL)

98

Future Research Directions

• Data mining the program interaction history

– Task inference, reminders, coding guides

• Improving non-code programmer communication

– Design history, Code orientation, Software spelunking

• Programming in the “Large”

– Multiple hi-res displays, whiteboards

• Visualizing effects of compilation and optimization

– Scalable task-oriented program visualization

• Scalable Debugging

– Concurrent apps, long-lived apps

99

Any Questions?

Andrew Begel: abegel@cs.berkeley.edu

100

Spoken Language Support for Software Development Andrew Begel

Spoken Language

Support for Software

Development

Andrew Begel

Motivation

Programming by Voice

Programming by Voice

Sometimes It’s Non-Obvious

Harmonia Analysis Framework

Program Analyses for Ambiguous Inputs

Homophones Cause Ambiguities

Lexical

Analysis

Semantic

Ambiguity

Resolution

Lexical

Analysis

XGLR

Ambiguous

Parsing

file

variable

file

method

filetoload

method

Program Navigation and Editing

Study - SPEED Usability

Conclusion

Contributions

Future of Programming by Voice

Any Questions?

Related documents

Products

Support

Spoken Language Support for Software Development Andrew Begel

Spoken Language

Support for Software

Development

Andrew Begel

Motivation

Programming by Voice

Programming by Voice

Sometimes It’s Non-Obvious

Harmonia Analysis Framework

Program Analyses for Ambiguous Inputs

Homophones Cause Ambiguities

Lexical

Analysis

Semantic

Ambiguity

Resolution

Lexical

Analysis

XGLR

Ambiguous

Parsing

file

variable

file

method

filetoload

method

Program Navigation and Editing

Study - SPEED Usability

Conclusion

Contributions

Future of Programming by Voice

Any Questions?

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib