Lesson Plan: Visualizing Sound by Captioning Nonspeech Sounds

Image of a man kissing something in his hands, with a the title "Visualizing Sound" in brackets.Although closed-captioning is usually thought of in terms of accessibility, it also highlights the larger rhetorical significance of sound—a significance which is most notably taken up in sound studies. The convergence of rhetoric with sound studies has become increasingly salient to rhetorical theory as noted in the review essay “Auscultating Again: Rhetoric and Sound Studies” from RSQ.

Sound in the multimodal classroom has likewise become a relevant site for rhetorical instruction, invention, and exploration. Scholars like Cynthia Selfe, Erin Anderson, and Steph Ceraso (to name only a few) use sound as locus for examining the rhetorical possibilities of aurality, material vocality, and bodily listening.

Sean Zdenek’s Reading Sounds: Closed-Captioned Media and Popular Culture offers an important insight into the rhetoricity of captioning sound and draws our attention to how sound is never merely present or transparent; rather captioning nonspeech sounds in particular demonstrates how a captioner rhetorically contextualizes, clarifies, formalizes, equalizes, linearizes, time-shifts, and distills sounds (8-10). In short, a captioner as rhetorician not only makes sound visible but also makes consequential decisions about affect and meaning within the constraints of the medium while augmenting and supporting a given narrative.

Building on Zdenek’s work on the rhetorical import of captioning, this lesson plan asks to students to consider how writing sound through captioning is an inventive, creative, and rhetorical act.

Learning Objectives

This lesson plan asks students to caption nonspeech sounds for a clip from the silent movie Nosferatu in order for students to engage in the rhetorical invention, writing, and visualization of sound. Captioning a silent movie emphasizes the rhetorical significance of nonspeech sound as well as foregrounds the captioner’s decision processes. Through captioning nonspeech sound, students will develop an understanding of not only how sound functions rhetorically, but also how their own choices as writers contribute the affective and meaningful dimensions of a text.

Students will learn:

  • How to recognize the rhetorical significance of nonspeech sounds and how they contribute to the emotional arc, purpose, and understanding of a narrative text.
  • How sound effects, paralanguage, and music provide rhetorical support to a narrative text through qualified, discrete and sustained, and single and multiple sound-captions.
  • How to use their new understanding of nonspeech sounds to caption a short clip from a silent movie, and write sound where there was none before.

Assignment Length

You will need two class periods for this assignment.

Required Materials

You will need iMovie and an internet connection.

Skills Necessary

You will need basic iMovie editing and YouTube video-capturing skills, both taught in this lesson plan.

This lesson plan describes students working on independent projects; but it could equally work for a collaborative project. If the class is in a non-computer classroom, instructors might request students who own laptops to bring them to class and work collaboratively in groups.

Although this lesson plan utilizes iMovie to caption video, other tools are also available that may be useful for this project. MovieCaptioner can be used in conjunction with QuickTime 7, which allows users to easily caption video four seconds at a time. Alternatively, Aegisub is a cross-platform open source tool for creating and modifying captions and subtitles.

Assignment Description

This lesson plan takes place over two class periods.

Day One: Introduce the rhetorical significance of nonspeech sound by watching a short video about foley artists. Then watch a the first act of an animated TV show to demonstrate how nonspeech sounds are captioned. Next, introduce and discuss the different ways in which nonspeech sounds are captioned. Finally, assign students to write about their own observations after watching a captioned episode of some kind for homework.

Day Two: Discuss with the class what the students discovered in their homework. Then introduce the prompt and watch the clip that students will be captioning. Next, provide students with an interactive tutorial for the tools they will be using. Finally, allow students to work on the assignment for the remainder of the class.

Instructor Preparation

  • Become familiar with the concepts, practices, and examples from Sean Zdenek’s supplemental website, particularly “Chapter 2: Reading and Writing Captions.”
  • Get acquainted with the video materials about foley artists and decide on an appropriate animated video with closed-captions to demonstrate how nonspeech sounds are written.
  • Practice with the tools KeepVid and iMovie as explained in the skills workshop.

Student Preparation

Between the first and second day, students should review the different types of nonspeech sounds from Zdenek’s website and watch a TV show of their choice, observing the ways in which nonspeech is captioned.

In-Class and Assignment Instructions

Day One:
(1) Introduce the importance of nonspeech sound with a YouTube video about foley artists. Discuss how sound is not simply a natural and present feature in movie narratives, but rather a rhetorical contribution to the emotional arc, purpose and understanding of the text.

(2) Watch the first act of an animated TV show with the closed captions on. I would suggest a show like the Simpsons because of the variety of nonspeech sounds typically captioned (episodes are usually free on the parent company’s website, which instructors may want to cue before class to avoid watching commercials). During the video, ask students to write down as many different nonspeech sounds and the way they are captioned as possible. Note that nonspeech sounds are usually marked within parentheses or brackets. Discuss with the class their findings.

(3) Introduce “Chapter 2: Reading and Writing Captions” from Sean Zdenek’s supplemental website. Discuss the seven different types of nonspeech captions, focusing primarily on Sound Effects, Paralanguage, and Music. Then show examples from the site of qualified, discrete and sustained, and single and multiple sound captioning. Next discuss the importance of the use and choice of nouns, verbs, adjectives, and adverbs when making captioning decisions. The choice of using “and,” “as,” and “then” is also important when captioning multiple sounds in a single caption. For homework, ask students to watch a show with closed captions on, and list the variety of choices captioners make in different situations when captioning nonspeech sounds. Instructors may want students to write a short, one-page analysis of their findings to discuss on day two.

Day Two:
(1) Discuss with the class what the students discovered about captioning nonspeech sounds from watching a show with the closed captions on.

(2) Introduce the assignment prompt (detailed below), then watch the video edit from the silent movie Nosferatu (chosen for its silent yet compelling narrative and public domain status). Have the class discuss how they might caption this movie and allow time for questions about the assignment.

(3) Demonstrate for students how to use KeepVid to capture the video clip of Nosferatu for editing in iMovie and then demonstrate how to add title captions to the video clip in iMovie (both are part of the skills workshop below). Allow students time to work on the assignment in class, and if needed, to complete it as homework.

Assignment Prompt:
General–This assignment asks students to caption ten or more nonspeech sounds for a two minute video clip of a silent movie. Their captions will: (1) support the emotional arc of a text, (2) contribute to the purpose of scene, (3) never be wasted on superfluous sounds that 
may confuse viewers, and (4) honor and respect the sequence and arc of the narrative.

Specifics–This assignment asks students to incorporate a variety of nonspeech captions, including Sound Effects, Paralanguage, and Music, in which they will rhetorically support the narrative through qualified, discrete and sustained, and single and multiple sound-captions.
Image of Nosferatu with a caption that reads "uncanny silence" in brackets.

Skills Workshop

KeepVid WorkshopScreen Shot of the homepage for KeepVid.com with a YouTube link in the download window.
Make the YouTube link of Nosferatu available to students via email so they can copy it. Have students open KeepVid in a browser and then ask them to paste the Youtube link into the download window and click download. (Ignore any false warnings from advertisers.)Screen Shot of the download page for KeepVid.com with MP4-480p highlighted.
Have students right-click on the MP4-480p option and select “Download Linked File As.”Screen Shot of the save window for KeepVid.com with file saved as "Nosferatu Clip" to the desktop.
Have students name their file with “Save As,” mostly likely to their desktop or a project folder. This will provide an mp4 file of Nosferatu for students to edit and add captions in iMovie.

iMovie Workshop
To import your video clip, click the “Import” button at the top of iMovie.
Then select the file you want to import and click “Import Selected” on the bottom right corner of iMovie.
To start a new project, click the “New” button at the top of iMovie and select “Movie.

Select “No Theme” and save project as “Captioning Project.”
Highlight your entire video clip.
Then drag the highlighted clip into the project area for editing.

Captioning Workshop
To add a caption, select “Titles,” then drag “Lower Third” into the project area where you want the caption to appear.

Delete the top title “Name” and write your caption on the “Description” line.

For example, “gentle smack” is written on the “Description” line.
Press return when you are done writing your caption and it will appear like this.

To adjust the length of time that your caption appears on screen, drag either end of the title bar in the project area until you have your caption properly synchronized.
To produce a playable mp4 when you are finished with your captions, select “Share” and select “File.” In the next window select “Next” and then save the file to your desktop or project folder.

Captioning Example

Assessment Suggestions

Instructor may decide to assess this assignment as a stand-alone project or a step toward a larger project in which students continue to work with their video. For example, having students add foley sound after they have already written their captions.

For portfolio-based and Learning Record assessment, instructors may want to provide feedback and allow for revision. Instructors may also want to assign a short refection to students, having them address both the drafting and revision processes.

For traditional assessment, instructors may want to grade students on the (1) quantity of captions–ten or more, (2) quality of the captions–how well captions contribute to the emotional arc, purpose, and understanding of the narrative, and (3) variety of captions–captioning sound effects, paralanguage, and music through qualified, discrete and sustained, and single and multiple sound-captions.

Leave a Reply

Your email address will not be published. Required fields are marked *