Reports, Publications, and Presentations
Summary of Activity To Date
We began with the development of the ClearSpeak speech style, using as its
conceptual basis classroom practice and existing internal ETS rules and best
practices, into a system that could be coded for use by the MathPlayer software.
To that end, we produced a comprehensive and organized listing of types and
subtypes of math expressions, along with notes on how they are currently spoken
by MathPlayer and how they should be spoken.
The design philosophy of the ClearSpeak speech system is to speak
mathematical expressions/objects so as to lessen or remove for learners who are
blind or have low vision the accessibility barrier to learning the mathematical
material being presented. ClearSpeak seeks to accomplish this by speaking
expressions in a way that is both familiar and unambiguous. Since students are
most familiar with the way their teachers speak in the classroom, ClearSpeak
speaks math in that way—but with adjustments to make sure the spoken math is
unambiguous. The verbiage introduced to make an expression/object unambiguous
can make spoken expressions harder to understand than the print versions of
those expressions. ClearSpeak seeks to minimize the effect of this by extensive
use of pauses (which do not require any memory); using language that integrates
well with the expression; and "translating" print conventions into their
mathematical equivalents if a sighted learner would do so instantaneously—doing for the learner with blindness or low vision what the sighted learner does
almost unconsciously. Finally, since in different contexts, it is desirable to
have the same math expression speak differently, ClearSpeak provides (through
preferences) a mechanism for speaking them in a way appropriate to commonly
encountered contexts and (through exact speech) a mechanism for speaking them in
any way the document author desires.
 Developed ClearSpeak prototype based on the design philosophy and on the
listing of math expression types.
Based on this design philosophy and on the foundational document
described above, we determined that ClearSpeak should be implemented through
three mechanisms, listed from the most to the least automatic:
Rules. Speech rules are the standard or default way for
speaking math structures, based on information that can be parsed from the
MathML representation of the expression/object. Math expressions are
analyzed based on type of element (fractions, exponents, parenthetical
expressions, radicals, etc.), level of complexity, and on other relevant
features present in the expression. Based on that analysis, the appropriate
rules are invoked for speaking the expression. In addition, rules include specifications of how various symbols (e.g. raised dot, set
membership, etc.) are to be spoken.
Preferences. Because math structures that will be parsed the
same way may at times (to resolve ambiguities or for instructional or
assessment purposes) need to be spoken differently from the applicable
rules' defaults, we developed a system of preferences that authors of math
documents could use to specify for a given expression whether, for example,
a fraction would be spoken as [numerator] over [denominator] or as a common
fraction. Thus "12/15" by rule would be spoken "twelve over fifteen", but if
the "ordinal" fraction preference is invoked, it would be spoken instead as
"twelve fifteenths".
Exact Speech. Finally, for cases in which the developed rules
and preferences are incomplete, not applicable to a particular expression,
or do not address a particular pedagogical or assessment need, we
implemented the ability to insert exact speech, including additional pauses,
pitchchanges, and speedchanges when authoring a math expression/object.
 Developed specifications for rules, preferences, and tested in MathPlayer,
Styles can be referenced in documents, and preferences are recognized.
MathPlayer now recognizes the rules and preferences developed for
ClearSpeak. We are able to specify in a web document that the ClearSpeak
style (as opposed to the preexisting speech styles Simple Speech and
MathSpeak). Web documents also speak the math in accordance with the
preferences and exact speech that had been set up for that document.
 Implemented interactive navigation. Math expressions can now be
navigated interactively using MathPlayer, in Microsoft Word, Internet
Explorer, and Firefox.
 Achieved support for MathPlayer by the NVDA screen reader. Using NVDA,
math expressions in a Word document or on the web can be spoken and
navigated along with the text, and can send Nemeth Code to an attached
refreshable braille display.
 Designed, conducted, and analyzed four feedback studies and a final
pilot.
Reports
2017:
 "Expanding
Audio Access to Mathematics Expressions by Students With Visual
Impairments via MathML"
Frankel, L., Brownstein, B., and Soiffer, N. (2017), Expanding
Audio Access to Mathematics Expressions by Students With Visual
Impairments via MathML. ETS Research Report Series.
doi:10.1002/ets2.12132
2016:
 "Development
and Initial Evaluation of the ClearSpeak Style for Automated
Speaking of Algebra"
Frankel, L., Brownstein, B., Soiffer, N. and Hansen, E. (2016),
Development and Initial Evaluation of the ClearSpeak Style for
Automated Speaking of Algebra. ETS Research Report Series.
doi:10.1002/ets2.12103
 "An
Evaluation of the Usefulness of Prosodic and Lexical Cues for
Understanding Synthesized Speech of Mathematics"
Frankel, L. and Brownstein, B. (2016), An Evaluation of the
Usefulness of Prosodic and Lexical Cues for Understanding
Synthesized Speech of Mathematics. ETS Research Report Series.
doi:10.1002/ets2.12119
Presentations
2016:
 "ClearSpeak
for Math Accessibility  Overview"
(YouTube video) This video is included on two playlists (Math
and Science; Special Education) compiled by the Institute of
Education Sciences as part of their December 6, 2016 post. See
How IES is Supporting TechnologyDelivered assessments.
2015:
 "Math
Accessibility in 2015: Browsers, Word, and Beyond"
(29.3 MB PPTX file) presented by Neil Soiffer, Lois Frankel, and Beth
Brownstein at CSUN, March 5, 2015
 "MathML
to Voiced Math"
(17.8 MB PPTX file, PowerPoint not necessary to view the presentation)
presented by Lois Frankel, Beth Brownstein, and Neil Soiffer at the
Principles of Schools for the Blind (POSB) Math/Science Institute, April,
2015.
2014:
 "Understanding
Mathematical Expressions through Interactive Navigation"
(20.3 MB PPTX file) presented by Sina Bahram, Neil Soiffer, and
Lois Frankel at CSUN, March 19, 2014
 "Navigating
Math Expressions with Synthetic Speech"
(349 KB PPTX file) presented by Lois Frankel and Susan Osterhaus
at CEC, April 2014
 "Breakthroughs
on Math Accessibility"
(47.5 MB PPTX file) presented by Neil Soiffer for the Montana
Accessibility Interest Group, October 3, 2014
2013:
 "Making Math More Fully Accessible for Users of TextToSpeech Technology"
(21.6 MB PowerPoint file)
presented by Beth Brownstein and Neil Soiffer at ATIA 2013 Orlando, February 1,
2013.
 "Navigable, Customizable TTS for Algebra" (paper,
PowerPoint slides  16.8 MB PPTX file)
presented by Lois Frankel
and Neil Soiffer at CSUN, San Diego, CA, March 1, 2013
 "Making a Computer Speak Algebra However You Want,"
(12.2 MB PowerPoint file)
presented by Susan Osterhaus at CEC 2013 Convention and Expo, April 5, 2013
 "Making a Computer Speak Math Like a Teacher Would,"
(12.2 MB PowerPoint file)
presented by Beth
Brownstein and Susan Osterhaus at 2013 NCTM Annual Meeting & Exposition,
April
2013
 "Empowering
Visually Impaired Learners via Spoken Algebra," (streaming video)
presented by Beth
Brownstein, Lois Frankel, and Neil Soiffer at the ETS Research Forum,
October 2013
2012:

"Expanding Audio Access to Mathematics Expressions by Students with Visual
Impairments via MathML," Presented by Susan Osterhaus at the 2012 GMI Crossover
Symposium on Technology for the Blind and Visually Impaired, Hosted by the
University of Colorado at Colorado Springs, May 12, 2012.
Presentation

"Expanding Audio Access to Mathematics Expressions by Students with Visual
Impairments via MathML," Presented by Lois Frankel at Council of Chief State
School Officers (CCSSO) National Conference on Student Assessment (NCSA), as
part of the panel, Research on Providing Audio Access to Assessment Content,
June 28, 2012.

"Making a Computer Speak Math However you Want," presented by Lois Frankel
and Susan Osterhaus at AER International, July 19, 2012.
