Ethical and Social Issues Around Data Collection

This unit is about programs that work with many values at once: hundreds of test scores, a year of temperatures, every student in a school. Before learning how to store and process that much data, it is worth asking where the data comes from. Very often it comes from people, and that means a program that collects it is holding something that belongs, in a real sense, to someone else.

A data set is a collection of related pieces of information, such as the sign-up forms for a club or the readings from a weather sensor. When the information is about specific people, it is personal data: anything that tells you about a particular person, such as their name, home address, phone number, birthday, location, grades, or health. Privacy is a person’s ability to control who knows that information about them.

Whenever personal data is typed into a computer, privacy is at risk. Once a program collects and stores the data, it can be:

• seen by people who should not see it, for example if the stored file is copied, a device is lost, or someone breaks into the system;
• used for a purpose the person never agreed to, such as selling a list of email addresses collected “only for club reminders”;
• combined with other data to reveal more than any single piece does;
• kept long after it is needed, so the risk lasts for years.

None of these risks require a bug in the code. A program can work perfectly and still put people’s privacy at risk, simply because it holds their data.

Now vary one thing: look at pieces that seem harmless alone. A birthday by itself matches thousands of people. A grade level matches a few hundred students. But “born on March 3, in 10th grade, in the robotics club” may match exactly one student. Pieces of data that reveal little separately can identify a specific person when they are combined. That is why “each field is harmless” is not the same as “the data set is harmless.”

If holding personal data creates risk, the programmer who designs the program decides how much risk the users carry. Users usually cannot see the code; they have to trust that whoever wrote it was careful. So when developing a new program, a programmer should try to safeguard (protect) the personal privacy of the people who use it.

Good habits that do this include:

The first habit is the strongest: data that was never collected can never be exposed, misused, or combined.

String phone = "5550198342";
String lastFour = phone.substring(phone.length() - 4);
System.out.println("Phone on file: " + phone);
System.out.println("Reminder goes to the number ending in " + lastFour);

Phone on file: 5550198342
Reminder goes to the number ending in 8342

String phone = "5550198342";
String lastFour = phone.substring(phone.length() - 4);
System.out.println("Reminder goes to the number ending in " + lastFour);

Reminder goes to the number ending in 8342

Now vary the kind of data. An email address can be shortened the same way, keeping just enough for the owner to recognize it:

String email = "jordan.lee@example.com";
int atSign = email.indexOf("@");
String masked = email.substring(0, 1) + "***" + email.substring(atSign);
System.out.println("Code sent to " + masked);

Code sent to j***@example.com

indexOf("@") finds the position of the @ (index 10). substring(0, 1) is "j", and substring(atSign) is "@example.com". The program puts "***" between them. The idea is the same as before: display the smallest piece that does the job.

String studentId = "40917325";
String shown = "****" + studentId.substring(4);
System.out.println("Locker assigned to student " + shown);

Locker assigned to student ****7325

Some harms from data and programs are not about privacy at all. They are about fairness: a program that treats one group of people worse than everyone else, over and over.

Recall from Topic 3.2 that a program can work perfectly for typical users and still fail people its designers did not picture. Now imagine that failure is not random. It always lands on the same kind of person. Algorithmic bias describes errors in a program that are systemic (built into how the program works, not caused by chance) and repeated (they happen every time the situation comes up), and that create unfair outcomes for a specific group of users.

Compare that with an ordinary glitch. If a server crashes for ten minutes, whoever happened to be online is affected; next time it might be anyone else. That is a reliability problem, but it is not bias. Bias is when you can predict who will be treated unfairly, because the cause is in the program’s rules or the data behind them.

import java.util.Scanner;

public class NameCheck
{
    public static void main(String[] args)
    {
        Scanner signUps = new Scanner("Rivera Ng Okafor Li Thompson");
        for (int person = 1; person <= 5; person++)
        {
            String lastName = signUps.next();
            if (lastName.length() >= 3)
            {
                System.out.println(lastName + ": accepted");
            }
            else
            {
                System.out.println(lastName + ": rejected, name too short");
            }
        }
    }
}

Rivera: accepted
Ng: rejected, name too short
Okafor: accepted
Li: rejected, name too short
Thompson: accepted

Now vary where the bias comes from. A hypothetical delivery app decides who gets same-day delivery using a list of neighborhoods where past orders arrived on time. Neighborhoods where the company never delivered before have no past orders, so they are never on the list. Every resident of those areas is denied same-day delivery, again and again, even though no line of code mentions them. Here the unfair rule was not typed in by hand; it came from the data the program relied on. Bias can enter through the code’s rules or through the data, and the next subsections look at the data side.

import java.util.Scanner;

public class SpaceCheck
{
    public static void main(String[] args)
    {
        Scanner signUps = new Scanner("Garcia\nDe La Cruz\nPark\nVan Dyke");
        for (int person = 1; person <= 4; person++)
        {
            String lastName = signUps.nextLine();
            if (lastName.indexOf(" ") == -1)
            {
                System.out.println(lastName + ": accepted");
            }
            else
            {
                System.out.println(lastName + ": rejected, invalid name");
            }
        }
    }
}

Garcia: accepted
De La Cruz: rejected, invalid name
Park: accepted
Van Dyke: rejected, invalid name

if (lastName.length() > 0)

Programmers often use data to learn something new: which feature users want, what the average score was, how busy a place will be next week. Using data to predict values or reach conclusions beyond what was directly measured is called extrapolating. Every conclusion like that is only as good as the way the data was gathered.

The collection method is how a data set was gathered: who was asked or measured, where, when, and how. If the method reaches some kinds of people or situations more than others, the data set does not represent the whole group, and any conclusion about the whole group can be wrong. That tilt in the data is called bias, the same word as in algorithmic bias, because a program built on tilted data tends to produce tilted results. So before using data to draw conclusions, a programmer should ask how it was collected and whether that method could be biased.

int surveyed = 40;
int wantLaterHours = 36;
int percent = wantLaterHours * 100 / surveyed;
System.out.println(percent + "% want later gym hours");

90% want later gym hours

Now vary only the collection method. Suppose instead the council hands the same question to every homeroom, reaching 400 students from every part of the school:

int surveyed = 400;
int wantLaterHours = 120;
int percent = wantLaterHours * 100 / surveyed;
System.out.println(percent + "% want later gym hours");

30% want later gym hours

Same question, same program, same arithmetic (120 * 100 / 400 is 30), and a very different answer. Nothing about the code changed. The only difference is who the data came from. A correct program cannot fix data that was collected in a biased way.

Even when the collection method is fair, the data set itself can be flawed. Real data is messy: a sensor turns off, someone skips a question, a value is typed wrong.

A data set is incomplete when some values are missing. It is inaccurate when some values are wrong. Either way, a program that uses the data can work incorrectly (producing wrong results) or inefficiently (doing extra, wasted work). As in Topic 3.2, the program may have no logic error of its own; the trouble comes in with the data.

import java.util.Scanner;

public class StepAverage
{
    public static void main(String[] args)
    {
        Scanner readings = new Scanner("8200 7900 0 8400 8100");
        int total = 0;
        int days = 0;
        for (int reading = 1; reading <= 5; reading++)
        {
            int steps = readings.nextInt();
            total += steps;
            days++;
        }
        System.out.println("Days counted: " + days);
        System.out.println("Average steps: " + total / days);
    }
}

Days counted: 5
Average steps: 6520

Now vary how the missing day is recorded. If the tracker records a missing reading as -1, a value no real step count can have, the program can recognize and skip it:

import java.util.Scanner;

public class StepAverageSkipMissing
{
    public static void main(String[] args)
    {
        Scanner readings = new Scanner("8200 7900 -1 8400 8100");
        int total = 0;
        int days = 0;
        for (int reading = 1; reading <= 5; reading++)
        {
            int steps = readings.nextInt();
            if (steps >= 0)
            {
                total += steps;
                days++;
            }
        }
        System.out.println("Days counted: " + days);
        System.out.println("Average steps: " + total / days);
    }
}

Days counted: 4
Average steps: 8150

The if on line 13 adds only values that are 0 or more, so the -1 is skipped and only four days count: 32600 / 4 is 8150. Notice what made the fix possible: the data marked the missing value clearly. When a missing value looks exactly like a real one, as the 0 did, no program can tell them apart. Complete, accurate data starts at collection time.

Flawed data can also waste work. If a contact list contains each person three times because of repeated imports, a reminder program sends three identical messages to everyone and does three times the work it needed to. The program runs correctly; the data makes it inefficient.

The last check comes before any code is written: is this data set even the right one? A data set can be fairly collected, complete, and accurate, and still be the wrong tool for your question.

Data is usually gathered to answer one particular question or to describe one particular topic. Its contents reflect that purpose: what was measured, about whom, when, and where. If you try to use it to answer a different question, it may simply not contain the information needed, and any answer or prediction you pull out of it may be wrong.

Now vary the question in time instead of topic. Suppose the school wants to predict checkouts for next year, but the library is moving to a new building twice the size, near the cafeteria. Last year’s data was collected under very different conditions, so extrapolating from it is risky. The data is the same; the question changed, and the data no longer fits it well.

A hypothetical student, Priya, is building a program that suggests a study partner to each student in her school. Follow her through every idea in this chapter.

Privacy (4.1.1, 4.1.2). Her first draft of the sign-up form asks for name, email, classes, home address, and phone location. She asks of each field, “does the program need this?” Matching study partners needs names, classes, and a way to contact them, so she keeps those and drops the address and location. Her confirmation screen shows the email as p***@example.com instead of the full address, and she plans to delete the data at the end of the school year.

Collection method (4.1.4). To decide which subjects to support first, she posts a survey on the computer science club’s page. Most responses ask for help with programming. She realizes the survey only reached club members, who are exactly the students most interested in programming, so she sends it to every homeroom before deciding.

Incomplete and inaccurate data (4.1.5). Some students left “free period” blank, and her first version treated a blank as “no free periods,” so those students were never matched. She changes the form to require the field and has the program report anyone whose data is missing instead of silently skipping them.

Algorithmic bias (4.1.3). In testing, students who take classes at a nearby college are never matched, every time, because her rule only compares classes from the school’s own course list. The error is systemic, repeated, and always hurts the same group, so it is algorithmic bias. She fixes the rule so it compares free periods even when a class is not on the list.

Fitness of the data set (4.1.6). A teacher offers last year’s study-hall attendance as “data about who studies together.” Priya notes that attendance shows who was in the same room, not who studied well together, so she does not use it to rank matches.

At each step, the code mattered less than the questions Priya asked about the data: should I collect it, how was it gathered, is it complete and correct, is my rule fair, and does it answer my question?

• Collecting and storing personal data puts privacy at risk, even in a program with no bugs, and harmless-looking pieces can identify someone when combined.
• Programmers should safeguard users’ privacy: collect only what is needed, use it only for its purpose, show as little as possible, and delete it when done.
• Algorithmic bias is a systemic, repeated error that creates unfair outcomes for a specific group; it can come from a program’s rules or from its data.
• Before drawing conclusions or extrapolating, check the collection method for bias; correct code cannot fix data gathered from the wrong people.
• Incomplete or inaccurate data makes a program work incorrectly or inefficiently, and a data set gathered for one question may not answer a different one.

import java.util.Scanner;

public class StepAverageTypo
{
    public static void main(String[] args)
    {
        Scanner readings = new Scanner("8200 7900 8400 81000");
        int total = 0;
        int days = 0;
        for (int reading = 1; reading <= 4; reading++)
        {
            int steps = readings.nextInt();
            total += steps;
            days++;
        }
        System.out.println("Days counted: " + days);
        System.out.println("Average steps: " + total / days);
    }
}

Days counted: 4
Average steps: 26375

String libraryCard = "29140068";
String shown = "****" + libraryCard;
System.out.println("Card: " + shown);

Card: ****29140068

String shown = "****" + libraryCard.substring(4);

Card: ****0068

Check each statement. If the answer is “not yet,” reread the linked subsection.

• I can explain why collecting and storing personal data puts privacy at risk. Reread
• I can describe ways a programmer can safeguard users’ privacy. Reread
• I can define algorithmic bias and tell it apart from a one-time bug. Reread
• I can explain how a biased collection method can make conclusions wrong. Reread
• I can explain how incomplete or inaccurate data makes a program work incorrectly or inefficiently. Reread
• I can decide whether a data set fits the question I want to answer. Reread