Welcome to JOUR 75204: Metrics & Outcomes

Course description

Our success as social journalists will be measured not just by who sees our work, or even by more sophisticated measures like how much time they spend with it or whether they share it, but rather by its impact - how well we helped the community accomplish its goals.

In this course, you will learn that certain metrics - such as "pageviews and unique users" - can mislead, leading to sensationalism and degraded value, credibility, and reputation. You will learn to instead to select the metrics with which you can best judge impact. That may mean measuring how informed a community becomes through your journalism, or how they are being engaged to react, contribute and participate toward their desired outcomes, or whether they met their goals. You will learn the tools and techniques needed to gather and analyze audience data and other signals to understand what does and does not help a community succeed.

Course outcomes

After completion of this course, you will:

Syllabus Overview

Faculty

lam.vo@journalism.cuny.edu

I'm an interactive editor for Al Jazeera America, where I lead a team of coder-journalists and cover poverty-related stories. Before coming to Al Jazeera, I was a multimedia producer for NPR's Planet Money where I created interactives, data visualizations, videos and photographs to explain economic issues in visual ways. From 2009-2011 I worked for The Wall Street Journal in Asia and was the editorial lead for video content in the region. I worked with more than a dozen bureaus to create both breaking news stories, regular series and produced enterprising multimedia projects. My reporting in Asia includes stories about China's property market, the 2011 Japanese earthquake and climate change issues in Vietnam's Mekong Delta.

As a beat reporter at Al Jazeera America, I try to have an open conversation with my audience and potential sources. My colleague E. Tammy Kim and I try to involve our audiences during our reporting trips via social media channels, solicit feedback as we travel for our stories and disseminate information about poverty in our Al Jazeera Poverty Community.

Grading

Your grade in this 11-week portion of the course is determined by four factors:

Your participation includes attending all classes and being punctual, being active in discussions, workshops and critiques, and participating in all in-class hands-on activities.

Plagiarism and Copyright

It is a serious ethical violation to take any material created by another person and represent it as your own original work. Any such plagiarism will result in serious disciplinary action, possibly including dismissal from the CUNY J-School. Plagiarism may involve copying text from a book or magazine without attributing the source, or lifting words, code, photographs, videos, or other materials from the Internet and attempting to pass them off as your own. Please ask the instructor if you have any questions about how to distinguish between acceptable research and plagiarism.

In addition to being a serious academic issue, copyright is a serious legal issue.

Never "lift" or "borrow" or "appropriate" or "repurpose" graphics, audio, or code without both permission and attribution. This guidance applies to scripts, audio, video clips, programs, photos, drawings, and other images, and it includes images found online and in books.

Create your own graphics, seek out images that are in the public domain or shared via a creative commons license that allows derivative works, or use images from the AP Photo Bank or which the school has obtained licensing.

The exception to this rule is fair use: if your story is about the image itself, it is often acceptable to reproduce the image. If you want to better understand fair use, the Citizen Media Law Project is an excellent resource.

When in doubt: ask.

Lesson Plan Changes

The lesson plan below is subject to change depending on the progress and success of the students in the class.

Lesson Plan Overview

Lesson Lecture Case Studies Hands-on Assignments
LESSON 1: The #socialj metrics philosophy (June 3) Who is your community? Students present their communities
LESSON 2: Media-centric, quantitative metrics (June 10) How clicks became king NYTimes nail salon story Google Analytics: how to implement it
LESSON 3: Engagement and attention metrics (June 17) Chartbeat’s argument Earthquake safety (CIR) http://californiawatch.org/earthquakes
Guest: Tony Haile, CEO, Chartbeat
Preparing your interviews: types of surveys how to ask questions
LESSON 4: Community interviews (June 24) Students present community interviews Guest: Karen Zraick and Francesca Barber, New York Times Deliverables: the results of the interviews with some of your community members
LESSON 5: Tools, tools, tools (July 1) Day in the life of a social media editor Patient harm series (ProPublica)
Guest: Danielle Powell, Social Media Editor at Al Jazeera America
Social analytics: where to get the data
LESSON 6: Social media war room (July 8) The Ombudsman, the social media editor and other outward facing roles Planet Money T-Shirt project (NPR)
Guest: Blair Hickman, Audience Editor, The Marshall Project
Social analytics: how to use them
LESSON 7: Qualitative measurement (July 15) Impact as a bar for success Listening post (WWNO)
Guest: Cole Goins, engagement editor, the Center for Investigative
Workshopping metrics based on your interviews and the CIR guidelines
LESSON 8: Towards a Taxonomy of impact (July 22) Students present taxonomy of impact Guest: Katherine Lanpher, features editor, Al Jazeera America Outcomes tracker, a checklist Deliverables: Metrics by which you will judge the success of your journalism
LESSON 9: Toward community-centric metrics (July 29) What does your community need and want?
Amanda Zamora, Audience Editor, ProPublica (tentative)
Rape in the fields (CIR) Bootstraps and FTP review, lab time
LESSON 10: Your dashboards (August 5)
Students present dashboard
GuestLindsay Green-Barber, Media Impact Analyst, Center for Investigative Reporting
Deliverables: Implementation methods of these metrics
LESSON 11: Wrapup (NEEDS SCHEDULING) Where to go from here?

Lesson Plan

Session 1 | June 3, 2015

LECTURE & DISCUSSION

The #socialj metrics philosophy

We will go over the syllabus and expectations for the course, and then we begin with a discussion about the philosophy of metrics and outcomes underlying the Social Journalism program, including shifting from “media centric” to “community centric” measurements, from purely quantitative to mixed/qualitative measurements, from assessing the popularity of our content to judging the success or failure of our efforts to help a community meet its own goals. Come prepared to begin discussing the definitions of success you have formulated thus far for your work in your community.

Session 2 | June 10, 2015

Media-centric, quantitative metrics

LECTURE & DISCUSSION

We must understand first how media measures itself today. We will examine the business models behind mass-media business metrics and the economic reasons why they are used: circulation, reach, share, frequency. We will see how these metrics were imported from legacy to digital media and the impact that has had — often, clickbait and cats. We will talk about the most popular metrics and how they influence news judgment and packaging of content. Then we will examine the various social media metrics used by digital media companies and advertisers and delve into their meaning and impact: followers, shares, likes, retweets, comments, upvotes, trending, and so on.

CASE STUDY

At Nail Salons in NYC Manicurists Are Underpaid and Unprotected (New York Times)

Public editor's response to the piece

HANDS-ON WORKSHOP & LAB

Implementing Google Analytics

HOMEWORK

Post your explainer from the social reporting class on your tumblr!

Session 3 | June 17, 2015

Engagement and attention metrics

LECTURE & DISCUSSION

Now we will move away from clicks to engagement and attention as metrics that at least begin to reward quality in media. We will invite an expert on Chartbeat to come to class to argue their case for attention (that is, time) as the key metric for media, because it both rewards quality content and measures advertising effectiveness. The guest lecturer will also demonstrate the Chartbeat metrics dashboard, which tracks not only page-by-page but also pixel-by-pixel activity on web sites and other digital services, explaining the insights this offers.

CASE STUDY

Earthquake safety (CIR)

HANDS-ON WORKSHOP & LAB

Preparing a good survey using Google Forms

Types of surveys

ASSIGNMENT

Using interviews, observations, and possibly even surveys or other methods of investigating your community, you will prepare a document outlining *their* goals and giving evidence (quotes, observations, etc.) backing that up. What do people in your community believe needs to change? What are they concerned about? What are their needs and goals.

Interview community members and present the following:

  • demographic information on them
  • Their needs and goals
  • What information they want you to report on and disseminate
  • best ways to distribute information to them (online, real-life, mobile, etc.)

Put together a survey and get at least 5 respondents for your community study. Write a 350 word memo on your tumblr and prepare to talk about your findings for at least 5 minutes.

How to send me your assignments:

Copy this format and fill in the details in your Googledoc that you submitted:

  • Assignment 1
  • Tumblr post link
  • Survey link
  • Survey results

Your google docs can be found here!

Session 4 | June 24, 2015

Community interviews

LECTURE & DISCUSSION

In this lecture you will present your findings from your interviews with your community members.

Session 5 | July 1, 2015

Tools, tools, tools

LECTURE & DISCUSSION

This class will be a workshop on many of the best social media measurement tools in use in the field: the dashboards and content-creation tools used to track and join conversations online. These are the tools used by social media editors as well as marketers and PR people to do their jobs. You will get hands-on experience with these tools.

HANDS-ON WORKSHOP & LAB

Using social analytics

Session 6 | July 8, 2015

Social media war room

LECTURE & DISCUSSION

In this class, groups of students will work together using social media dashboards for hypothetical and real sites to understand how media properties monitor and respond to social media discussion about them. Though in this program these skills can be perceived as overly “media centric” and “brand centric,” it is critical that students understand how these functions are performed because some of your employers will expect you to manage work such as this.

CASE STUDY

Planet Money T-Shirt project (NPR)

Yellow Rain (WNYC)

HANDS-ON WORKSHOP & LAB

Social tools and social strategies:

  • reacting to reactions
  • smart group moderation
  • creative calls to action and dialogue

Session 7 | July 15, 2015

Qualitative measurement

LECTURE & DISCUSSION

Having read the white papers on media impact by Chalkbeat [not to be confused with Chartbeat] and USC, we will shift our focus to more qualitative measurements. These methodologies stress signals of impact: how often a report is mentioned in the press or by public officials; how often a documentary made by CIR was screened for the community it is serving; whether laws have been proposed and passed. Note, however, that even these more advanced philosophies of measurement are still primarily mediacentric: how much attention did our content get and what impact did our content have?

CASE STUDY

Listening post (WWNO)

HANDS-ON WORKSHOP & LAB

Workshopping metrics based on your interviews and the CIR guidelines (Post-It notes!)

ASSIGNMENT

We will work on a dashboard creation: integration of goals, tools, signals, and metrics to monitor and judge success and failure. Students will be assigned to create their own dashboards, which they will use in their work in their communities the following term.

Session 8 | July 22, 2015

Towards a Taxonomy of impact

LECTURE & DISCUSSION

With Chalkbeat’s and CIR’s work as examples, you will present a system for collecting and monitoring the metrics and measurements you wish to track and develop a “dashboard” of some kind for tracking impact. You’ll need to think about both quantitative data (traditional metrics, social media metrics, etc.) as well as qualitative data.

Session 9 | July 29, 2015

Toward community-centric metrics

LECTURE & DISCUSSION

Now we shift from media-centric to community-centric measurement, from popularity to outcomes, arriving at the heart of what we want to accomplish with social journalism and advancing the art past almost all of the journalistic landscape today. Now we ask what a community’s own goals are and how the community will judge success against them. We will discuss how we might do this in our communities.

CASE STUDY

Rape in the fields (CIR)

HANDS-ON WORKSHOP & LAB

Basic data analysis and spreadsheets

Session 10 | August 5, 2015

Your dashboards

LECTURE & DISCUSSION

Presentation of all the dashboards and how you monitor them

Session 11 | August 12, 2015

Onwards

LECTURE & DISCUSSION

Wrapping up