Lesson Plan: Perspective API – Rhetorical Metadata, Pathos, and the Future of Internet Moderators

Teaching students how to analyze an argument is no simple task, as rhetorical analysis exists on a spectrum of complexity ranging from the most commonly encountered modes of persuasion, to analytical frameworks that push the definition of “rhetorical action” to its limit and beyond. What this disparity demonstrates is that rhetoric is generative, inventional, it stipulates through rhetorical action its own limits, it defines itself through its own capacities.

Moving students from summary to analysis in Unit 2, students are asked to derive from a text not only its central claims and key evidence for the claims, but also detail how an argument is put together and why specific choices have been made. What crucially comes into focus in this unit is the relationship between the argument (and its author) and the audience receiving the argument. Rhetoric is by definition incapable of neutrality, persuasion persuades in some direction, consciously or otherwise. As this unit emphasizes, the audience, the “object of persuasion” will become the the subject of rhetorical analysis in determining what argumentative strategies are to be employed in a given situation. The implication beneath this demand of attending to one’s audience is that there is no single rhetorical situation, no universal audience, what’s rhetorically effective in one situation will not necessarily be effective in the next.

Credit: https://www.perspectiveapi.com

This lesson plan is designed to augment the issue of audience that is encountered in the second unit of the RHE 306 instructor guide, by having students engage an emergent internet tool that recapitulates traditional understandings of audience and rhetorical affect, and re-stipulates the rhetorical situation surrounding public internet discussions. The digital tool is the recently developed “Perspective API”, an application programming interface that claims to “make it easier to host better conversations”, by “using machine learning models to score the perceived impact a comment might have on a conversation”.

Perspective is supposed to serve the function of helping online developers and publishers “use the score (perceived impact) to give realtime feedback to commenters or help moderators do their job, or allow readers to more easily find relevant information”. What the API does in its current instantiation is analyze comments algorithmically to produce a percentile score of a comment’s perceived “toxicity”. The creators define toxic as “a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion”. Further, the API can organize an entire comments-section into a spectrum of toxicity, which can be described as providing supposed rhetorical metadata.

The lesson plan frames this digital tool through an emphasis on audience, where the Perspective API is argued to present a “universal rhetorical situation”, a static, algorithmic audience. Students will play with the rating system Perspective uses and try to rhetorically analyze the API’s determinations, and either agree or disagree with the results of these experiments. Further, the lesson plan offers an activity in which students will search through what they have written for the course so far, and try to find an argumentative strategy (contained in a phrase they wrote), a conclusion, assertion, characterization, etc. they employed that the API views as fifty percent or more “toxic”. The students will then be prompted write a response to the conclusions of the API as well as how this application comes to bear upon the broader discussion of audience and rhetorical analysis.

Learning Objectives

The lesson is designed to develop a nuanced understanding of the relationship between rhetorical exigency and audience. The Perspective API is taken up because it attempts to do that which instructors of 306 ask of their students, to, through rhetorical analysis, come to a notion of a particular rhetorical strategy’s perceived impact upon the audience. Testing the perceived efficiency of the API will help students understand how rhetorically situating an audience is itself rhetorical, we have to make arguments about what we think an audience thinks before we develop a rationale for attempting to persuade them. Perspective does this algorithmically, meaning its argument for its perception of toxicity is static, not situated within a material context. Pushing against the API and having students find at least one of Perspective’s evaluations to be inaccurate or incorrect, allows students to see the algorithmic dimensions of their own assumptions, and encourages them to embed themselves and their arguments within a context, to an audience in that context, rather than proceed as an a algorithm would.

1. Introduce Perspective API as an algorithmic intervention upon rhetorical analysis of an audience.

2. Through testing the evaluations Perspective makes, problematize the theoretical ground the API exists upon, through emphasis on situated contextualization as it relates to audience analysis.

3. Dichotomize logical and rhetorical analysis, and through this demonstrate the capacity and value of rhetorical analysis in attending to one’s audience.

  • Assignment Length

    One class day for lesson and activity.

  • Required Materials

    At least one computer with internet access and projection capabilities. Students will need devices or be in a DWRL computer classroom.

  • Skills Necessary

    No extraneous skills required, Instructor must interact with the digital tool the activity uses.

Access and Adaptability

Perspective API is visually oriented, but the experiment offered gives its impact rating both in a color coded symbol, as well as a text-based percentile rating. Since it does give the results in text, the experiment will work perfectly well with a screen reader.

Since this lesson relies on using an algorithmic mode of audience analysis to contrast with the rhetorical analysis taught in the course, there must be at least one device in the classroom to enact the lesson plan. However, Perspective’s experiment-tool runs smoothly on mobile OS, and so students can use their personal devices including smartphones to play with the API. And if not everyone has access to the website, the activity can be done as a class, with students collaborating and suggesting phrases or statements to the instructor, to test, or can be done in groups.

Assignment Description

The Assignment should be enacted during or after week six, where the topic of audience comes to the fore. The assignment can be effective as an introduction to the issue of audience, or as an concluding challenge to the issue.

Begin the assignment by asking students if they think a computer algorithm could make viable appeals to pathos. This will draw out if students see audience as either static or situated. Make the point that appeals to pathos are necessarily in tension to some degree with logical, mathematical evaluations, pathos fundamentally involves feeling, emotion, rather than reason.

Introduce the Perspective API to students through this frame: as an attempt to solve a human problem with computer logic. If there is a concern over a luddite narrative developing, remind students that computers solve “fundamentally human” problems every day, the complicated problems we face as humans are reflected in the efficiency and capacity of the computer itself.

But remind students that this situation is different, since the API is to implemented to shape and constrain the rhetorical situation of human opinion in digital environments. This means that the API is operating on the metaphor that there is a detectable standard of “toxicity” inherent in certain communications, there is some substance, some empirically detectable unit of negative rhetorical affect that is context-independent and defined outside of the human motivation behind it. This is of course taking the experiment and rhetorically amplifying its implications, the tool was built in TensorFlow and relies on limited models, but the website presents in primacy its early adoption by Wikipedia, The Economist, The New York Times, and others, while presenting itself rhetorically as through implication greater than the sum of its current capacities. A demonstration of this conceptual apprach is seen in “Definition of Man” into the conversation, and have students read the section “Inventor of, and Invented by the Negative” where Burke asserts that there are no negatives in nature, or outside of symbolism, since everything is nature is positively as it is in nature, as one finds it, and the idea of the negative only comes when there is an evaluative framework enacted around a human motive (this is either good or bad for me, but outside of this, it simply is as it is).

Once the scene is in place, bring up Perspective on the projector, and show them some test experiments with the “Writing Experiment” Input box. Show students similar sentences that, based on subtle changes, go from non-toxic to very toxic, to get students questioning how the API analyzes the rhetoric inherent in a phrase. Here is one example of how the algorithms workings can be demonstrated to the students:

The coding is cleaver, and can distinguish arguments with evidential claims from those without, even if the sentiment itself is the same. It can also detect qualified claims or “hedging” in an effort to distinguish dogma from invested pathos.

Talk to students about how it attempts to distinguish form from content, argumentative conclusions from the rhetorical means of coming to the conclusions.

Now, have students get on the computers (or if you’re not in a DWRL classroom, have students get their devices out) and go to https://www.perspectiveapi.com.

Have them individually experiment with the algorithm, and give them some time to find two to five instances where they disagree with the toxicity level. Have them find examples of where they think the rating is too high, and too low.

After students find two to five instances in each case, have students discuss their rationale for their own rhetorical analysis, and defend their divergence from the rating of the algorithm. If so desired, the actual logic of the toxic-test can be engaged, as it will expose the treatment of pathos being employed, and can be contrasted with the students own analysis.

Have students turn their experiment results in at the end of class, for a participation grade, or if the students are enthusiastic about the lesson and activity, assign them a one-page writing prompt that either asks for students to A: further theorize how the algorithm works and how its differentiated from a persons rhetorical analysis, and the metaphors at play in the concept of “rhetorical metadata”, or B: ask students to find an argumentative move in something they have written for the course that Perspective finds 75% or more toxic, and respond to this claim from the algorithm.

Instructor Preparation

  • Introduce students to concept of pathos, and the rhetorical analysis of an audience
  • Read the information on the Perspective web site, and experiment with the writing-toxic-analysis tool
  • Prepare sample-phrases for Perspective that demonstrate the mechanisms by which a phrase is considered toxic (or use the example given in the assignment description)

Student Preparation

  • (If not in DWRL classroom) bring device that has internet access.

In-Class or Assignment Instructions

  • Go to: https://www.perspectiveapi.com
  • Scroll down to find the “Writing Experiment” Box. Open up a word processor on computer or have a pen and paper ready.
  • Begin testing the rhetorical analysis the algorithm does on phrasings: use argumentative strategies from class to phrase statements of fact and opinion. Do not use expletives or slurs of any kind: doing this will prevent you from seeing how the algorithm is working rhetorically, how it looks at argumentative strategies, and attempts to assign motives to statements isolated from a speaker or context. The rating system is subtle and complex, it does not simply look for “bad words”, but its more advanced analytical efforts won’t be seen if one deliberately uses “bad words”: it will rate it as toxic regardless of any other rhetorical strategy.
  • Find and record (on computer document on personal sheet of paper): 1. an instance where you agreed with the toxicity rating 2. an instance where you thought the toxicity rating was too high 3. an instance where you thought the toxicity rating was too low. In each instance, try to determine 1. why you think the rating was given as such 2. why you disagree (or agree) with the rating given by the algorithm.

Assessment Suggestions

There are two traditional grading options for the assignment, and one “portfolio-style” option that includes a multimodal project assignment.

If the instructor wants to extend this lesson beyond a single class meeting, then there could be assigned a small project, where one develops their own rhetorical experiment, and compares it with the results given in the Perspective experiment.

Have students develop a number rhetorically charged statements that are nonetheless inoffensive and neutral in their term (but not necessarily in how they employ the terms). Have them ask a number of friends, peers or whomever they wish, to assign a value to the statements in terms of toxicity, defined as: “a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.”

Have students code the response options with four answers:
1. Very Toxic
2. Toxic
3. Slightly Toxic
4. Neutral

Have students plug in the same statements into the Perspective experiment, and record results.

Give prompt to students:

In a multimodal fashion, develop an artifact that presents the experiment and its results. This can take the shape of a video, a presentation, data visualization, or the creation of an infographic.

Leave a Reply

Your email address will not be published. Required fields are marked *