Skip NavigationDesign Science: How Science Communicates
Products Solutions Store Support Reference Company View Cart
 
 

Reports, Publications, and Presentations

Summary of Activity To Date

We began with the development of the ClearSpeak speech style, using as its conceptual basis classroom practice and existing internal ETS rules and best practices, into a system that could be coded for use by the MathPlayer software. To that end, we produced a comprehensive and organized listing of types and subtypes of math expressions, along with notes on how they are currently spoken by MathPlayer and how they should be spoken.

The design philosophy of the ClearSpeak speech system is to speak mathematical expressions/objects so as to lessen or remove for learners who are blind or have low vision the accessibility barrier to learning the mathematical material being presented. ClearSpeak seeks to accomplish this by speaking expressions in a way that is both familiar and unambiguous. Since students are most familiar with the way their teachers speak in the classroom, ClearSpeak speaks math in that way—but with adjustments to make sure the spoken math is unambiguous. The verbiage introduced to make an expression/object unambiguous can make spoken expressions harder to understand than the print versions of those expressions. ClearSpeak seeks to minimize the effect of this by extensive use of pauses (which do not require any memory); using language that integrates well with the expression; and "translating" print conventions into their mathematical equivalents if a sighted learner would do so instantaneously—doing for the learner with blindness or low vision what the sighted learner does almost unconsciously. Finally, since in different contexts, it is desirable to have the same math expression speak differently, ClearSpeak provides (through preferences) a mechanism for speaking them in a way appropriate to commonly encountered contexts and (through exact speech) a mechanism for speaking them in any way the document author desires.

  1. Developed ClearSpeak prototype based on the design philosophy and on the listing of math expression types.

    Based on this design philosophy and on the foundational document described above, we determined that ClearSpeak should be implemented through three mechanisms, listed from the most to the least automatic:
    1. Rules. Speech rules are the standard or default way for speaking math structures, based on information that can be parsed from the MathML representation of the expression/object. Math expressions are analyzed based on type of element (fractions, exponents, parenthetical expressions, radicals, etc.), level of complexity, and on other relevant features present in the expression. Based on that analysis, the appropriate rules are invoked for speaking the expression. In addition, rules include specifications of how various symbols (e.g. raised dot, set membership, etc.) are to be spoken.

    2. Preferences. Because math structures that will be parsed the same way may at times (to resolve ambiguities or for instructional or assessment purposes) need to be spoken differently from the applicable rules' defaults, we developed a system of preferences that authors of math documents could use to specify for a given expression whether, for example, a fraction would be spoken as [numerator] over [denominator] or as a common fraction. Thus "12/15" by rule would be spoken "twelve over fifteen", but if the "ordinal" fraction preference is invoked, it would be spoken instead as "twelve fifteenths".

    3. Exact Speech. Finally, for cases in which the developed rules and preferences are incomplete, not applicable to a particular expression, or do not address a particular pedagogical or assessment need, we implemented the ability to insert exact speech, including additional pauses, pitch-changes, and speed-changes when authoring a math expression/object.
       

  2. Developed specifications for rules, preferences, and tested in MathPlayer, Styles can be referenced in documents, and preferences are recognized.

    MathPlayer now recognizes the rules and preferences developed for ClearSpeak. We are able to specify in a web document that the ClearSpeak style (as opposed to the pre-existing speech styles Simple Speech and MathSpeak). Web documents also speak the math in accordance with the preferences and exact speech that had been set up for that document.
     
  3. Implemented interactive navigation. Math expressions can now be navigated interactively using MathPlayer, in Microsoft Word, Internet Explorer, and Firefox.
     
  4. Achieved support for MathPlayer by the NVDA screen reader. Using NVDA, math expressions in a Word document or on the web can be spoken and navigated along with the text, and can send Nemeth Code to an attached refreshable braille display.
     
  5. Designed, conducted, and analyzed four feedback studies and a final pilot.

Reports

2017:

  1. "Expanding Audio Access to Mathematics Expressions by Students With Visual Impairments via MathML"
    Frankel, L., Brownstein, B., and Soiffer, N. (2017), Expanding Audio Access to Mathematics Expressions by Students With Visual Impairments via MathML. ETS Research Report Series. doi:10.1002/ets2.12132

2016:

  1. "Development and Initial Evaluation of the ClearSpeak Style for Automated Speaking of Algebra"
    Frankel, L., Brownstein, B., Soiffer, N. and Hansen, E. (2016), Development and Initial Evaluation of the ClearSpeak Style for Automated Speaking of Algebra. ETS Research Report Series. doi:10.1002/ets2.12103
  2. "An Evaluation of the Usefulness of Prosodic and Lexical Cues for Understanding Synthesized Speech of Mathematics"
    Frankel, L. and Brownstein, B. (2016), An Evaluation of the Usefulness of Prosodic and Lexical Cues for Understanding Synthesized Speech of Mathematics. ETS Research Report Series. doi:10.1002/ets2.12119

Presentations

2016:

  1. "ClearSpeak for Math Accessibility - Overview"
    (YouTube video) This video is included on two playlists (Math and Science; Special Education) compiled by the Institute of Education Sciences as part of their December 6, 2016 post. See How IES is Supporting Technology-Delivered assessments.

2015:

  1. "Math Accessibility in 2015: Browsers, Word, and Beyond"
    (29.3 MB PPTX file) presented by Neil Soiffer, Lois Frankel, and Beth Brownstein at CSUN, March 5, 2015
  2. "MathML to Voiced Math"
    (17.8 MB PPTX file, PowerPoint not necessary to view the presentation) presented by Lois Frankel, Beth Brownstein, and Neil Soiffer at the Principles of Schools for the Blind (POSB) Math/Science Institute, April, 2015.

2014:

  1. "Understanding Mathematical Expressions through Interactive Navigation"
    (20.3 MB PPTX file) presented by Sina Bahram, Neil Soiffer, and Lois Frankel at CSUN, March 19, 2014
  2. "Navigating Math Expressions with Synthetic Speech"
    (349 KB PPTX file) presented by Lois Frankel and Susan Osterhaus at CEC, April 2014
  3. "Breakthroughs on Math Accessibility"
    (47.5 MB PPTX file) presented by Neil Soiffer for the Montana Accessibility Interest Group, October 3, 2014

2013:

  1. "Making Math More Fully Accessible for Users of Text-To-Speech Technology"
    (21.6 MB PowerPoint file) presented by Beth Brownstein and Neil Soiffer at ATIA 2013 Orlando, February 1, 2013.
  2. "Navigable, Customizable TTS for Algebra" (paper, PowerPoint slides -- 16.8 MB PPTX file)
    presented by Lois Frankel and Neil Soiffer at CSUN, San Diego, CA, March 1, 2013
  3. "Making a Computer Speak Algebra However You Want," (12.2 MB PowerPoint file)
    presented by Susan Osterhaus at CEC 2013 Convention and Expo, April 5, 2013
  4. "Making a Computer Speak Math Like a Teacher Would," (12.2 MB PowerPoint file)
    presented by Beth Brownstein and Susan Osterhaus at 2013 NCTM Annual Meeting & Exposition, April 2013
  5. "Empowering Visually Impaired Learners via Spoken Algebra," (streaming video)
    presented by Beth Brownstein, Lois Frankel, and Neil Soiffer at the ETS Research Forum, October 2013

2012:

  1. "Expanding Audio Access to Mathematics Expressions by Students with Visual Impairments via MathML," Presented by Susan Osterhaus at the 2012 GMI Crossover Symposium on Technology for the Blind and Visually Impaired, Hosted by the University of Colorado at Colorado Springs, May 12, 2012. Presentation

  2. "Expanding Audio Access to Mathematics Expressions by Students with Visual Impairments via MathML," Presented by Lois Frankel at Council of Chief State School Officers (CCSSO) National Conference on Student Assessment (NCSA), as part of the panel, Research on Providing Audio Access to Assessment Content, June 28, 2012.

  1. "Making a Computer Speak Math However you Want," presented by Lois Frankel and Susan Osterhaus at AER International, July 19, 2012.

- top of page -
ETS and the ETS logo are registered trademarks of Educational Testing Service (ETS). MEASURING THE POWER OF LEARNING is a trademark of ETS. Copyright © 1996-2017 Design Science and ETS.
 
All rights reserved. | Privacy statement