Introduction to Using Data Sets

Your phone counts your steps every day. A coach writes down every runner’s lap time. A weather station records the temperature every hour. One number on its own says very little: “8,400 steps” does not tell you whether this was a good week. Put a whole week of step counts side by side, though, and you can start asking real questions. Was I more active on weekends? How many days did I reach my goal?

Computing is full of situations like this, where the interesting information lives in many related values taken together rather than in any single one. A data set is a group of particular facts or measurements about the same thing, gathered so that they can be looked at together. Each piece of information in it is a data value. A week of step counts is a data set; each day’s count is one data value. Topic 4.1 used the term while looking at the risks of collecting data; this chapter looks closely at what a data set is and how a program works with one.

Three things make a collection a data set rather than a random pile of numbers:

Data values do not have to be numbers. A list of the songs a radio station played this morning is a data set of words. A record of each day’s weather written as one letter (S for sunny, R for rain, C for cloudy) is a data set of single characters.

Now change one thing: mix in a value that does not belong. Suppose someone adds a row “Maya’s height: 160” to the table. It is a specific piece of information about Maya, but it is not a step count for a day, so it does not describe the same subject. Adding up the column would now mix steps with centimeters and give a meaningless number. A data set is only as useful as the agreement among its values.

Collecting data is never the goal by itself. Maya did not record her steps so that she would have a table; she recorded them to find something out. A data set becomes useful when someone analyzes it: examines and works with its values to solve a problem or answer a question.

The question decides what work has to be done. Most everyday questions about a data set fall into a handful of familiar kinds, and you already wrote loops for most of them in Topic 2.9:

Each row asks something different of the same seven values, and each needs a different thing remembered while the values are examined. (A flag, from Topic 2.10, is a boolean variable that records whether something has happened yet.)

Now vary the question in a different way: ask “How late is the Route 4 bus usually?” The kind of question (what is typical?) is one you can process, but this data set cannot answer it. Every value describes Route 9. As you saw in Topic 4.1, a data set collected for one question may not be appropriate for a different one. Before planning any processing, check that the data set actually contains information about what is being asked.

When you added up the ten Route 9 delays, you probably did not take in all ten numbers at once. You went through them in order: 0, then 4 (total 4), then 2 (total 6), and so on, keeping one number in your head. That habit is exactly how a program analyzes a data set.

Think of a cashier at a grocery store. The cashier does not lift the whole cart onto the scanner. Items go past one at a time, each one is scanned, and the register updates the total right then. At the end, the total is ready. Where the analogy stops working: a cashier can glance back at an item already bagged, but a program reading values one after another often cannot, so it must remember whatever it will need later as it goes.

When a data set is analyzed, its values are accessed and used one at a time, and each value is processed according to what the question needs. Every such algorithm has the same three-part shape, which you can write in plain language before any code exists:

1. Set up. Before looking at any value, give the thing you are tracking its starting value (a total of 0, a count of 0, the first value as the best so far).
2. Visit each value, one at a time. For each value, do the processing the question needs: add it, test it and maybe count it, compare it to the best so far.
3. Report. After the last value, the tracked result is the answer (or, for an average, one division away from it).

Now change the question but keep the shape: “On how many days did Maya reach 8,000 steps?”

1. Set goalDays to 0.
2. For each day’s step count, in order: if the step count is at least 8,000, add 1 to goalDays.
3. After the last day, report goalDays.

Visiting 6,200, 8,400, 7,100, 9,300, 5,800, 10,200, 8,900 in turn, goalDays goes 0, 1, 1, 2, 2, 3, 4. The answer is 4. Setup and reporting look the same as before; only the processing inside step 2 changed. That is the point of the shape: once you know it, a new question means deciding what to set up and what to do with each value.

Some questions are harder to plan in words alone. “What was the warmest night?” or “Did the temperature keep rising?” require remembering something and comparing against it, and it is easy to get the details wrong in your head. Programmers plan these on paper first, using a picture of the data.

A data set can be shown in a diagram such as a table or a chart. A table lists the values exactly, which is what you need to follow an algorithm step by step. A chart, such as a bar chart, turns each value into a shape whose size shows the value, so patterns jump out at a glance. Here is Maya’s week as a simple text bar chart, where each # stands for 1,000 whole steps:

Mon   6,200  ######
Tue   8,400  ########
Wed   7,100  #######
Thu   9,300  #########
Fri   5,800  #####
Sat  10,200  ##########
Sun   8,900  ########
                    ^ goal: 8,000

The chart answers “which days reached the goal?” visually (the bars that pass the 8 mark), and it suggests the algorithm: check each bar against the goal line, one bar at a time.

To plan the details of an algorithm, add columns to the table for everything the algorithm must remember. This planning table has one row per data value and one column per tracked result. Filling it in, row by row, is running the algorithm by hand before it becomes code. If the last row holds the right answer, the plan works; if not, the table shows exactly which row went wrong.

Now vary the question to a pattern: “On how many nights was it warmer than the night before?” The best so far is no longer the right thing to remember; each value must be compared with the one just before it. So the planning table needs a column for the previous value, and after each comparison, the current value becomes the new previous value.

Two nights (3 and 5) were warmer than the night before. Drawing the table forced a decision you might have missed in words: the first night has nothing before it, so it can only become the first previous value.

A planning table is only useful if it turns into a working program. You already have the tools. In Topics 1.13 and 1.14, a Scanner built from a string handed out its values one at a time, and each nextInt() call moved it forward so the same value could not be read twice. In Topic 2.9, accumulator variables built up a result inside a loop. Put those together and you have a program that processes a data set exactly the way this chapter has described: one value at a time, never all at once.

The step from plan to code is direct. Each column you tracked in the planning table becomes a variable. The setup becomes statements before the loop. The “for each value” step becomes the loop body, with one nextInt() call to get the current value. The report becomes a print statement after the loop. (Later in this unit you will learn ways to store a whole data set inside a program so that its values can be visited again. For now, each value is read, used, and left behind.)

import java.util.Scanner;

public class HighestTemp
{
    public static void main(String[] args)
    {
        Scanner lows = new Scanner("-3 -8 -1 -6 -4");
        int highest = lows.nextInt();
        for (int night = 2; night <= 5; night++)
        {
            int temp = lows.nextInt();
            if (temp > highest)
            {
                highest = temp;
            }
        }
        System.out.println("Warmest low: " + highest);
    }
}

Warmest low: -1

Now vary the processing. Here is the “how many days reached 8,000 steps?” plan from 4.2.3 in Java:

import java.util.Scanner;

public class StepGoalCount
{
    public static void main(String[] args)
    {
        Scanner weekSteps = new Scanner("6200 8400 7100 9300 5800 10200 8900");
        int goalDays = 0;
        for (int day = 1; day <= 7; day++)
        {
            int steps = weekSteps.nextInt();
            if (steps >= 8000)
            {
                goalDays++;
            }
        }
        System.out.println("Days at goal: " + goalDays);
    }
}

Days at goal: 4

The setup (line 8) is a counter starting at 0, not a value read from the data, so the loop reads all seven values, with day running from 1 to 7. Inside the loop, line 11 reads the current day’s steps and line 12 tests it; goalDays goes 0, 1, 1, 2, 2, 3, 4, matching the written trace. A data set does not have to be numbers, either. The same visit-and-process shape works on a string of single-letter weather codes using the character loop from Topic 2.10, as you will see in the questions below.

import java.util.Scanner;

public class StepTooMany
{
    public static void main(String[] args)
    {
        Scanner weekSteps = new Scanner("6200 8400 7100 9300 5800 10200 8900");
        int total = 0;
        for (int day = 1; day <= 8; day++)
        {
            int steps = weekSteps.nextInt();
            total += steps;
        }
        System.out.println("Total steps: " + total);
    }
}

Exception in thread "main" java.util.NoSuchElementException
	at StepTooMany.main(StepTooMany.java:11)

String weather = "SSRCSRS";
int rainyDays = 0;
for (int i = 0; i < weather.length(); i++)
{
    if (weather.substring(i, i + 1).equals("R"))
    {
        rainyDays++;
    }
}
System.out.println("Rainy days: " + rainyDays);

Rainy days: 2

import java.util.Scanner;

public class StepTotalSixDays
{
    public static void main(String[] args)
    {
        Scanner weekSteps = new Scanner("6200 8400 7100 9300 5800 10200 8900");
        int total = 0;
        for (int day = 1; day <= 6; day++)
        {
            int steps = weekSteps.nextInt();
            total += steps;
        }
        System.out.println("Total steps: " + total);
        System.out.println("Average per day: " + total / 7);
    }
}

A hypothetical student, Maya, wants to know two things about her week: how many steps she took in all, and how many she averaged per day. Follow the whole process.

The data set (4.2.1). Seven specific values, one step count per day, all from the same week: 6,200, 8,400, 7,100, 9,300, 5,800, 10,200, 8,900.

The questions (4.2.2). “How much in all?” needs a running total. “What is typical?” needs the same running total, divided by the number of days. Both are about Maya’s steps this week, which is exactly what the data records, so the data set can answer them.

The plan in words (4.2.3). Set total to 0. For each day’s steps, add them to total. After the last day, report total, and report total divided by 7.

The plan in a table (4.2.4). One row per value, one column for total: 0, 6,200, 14,600, 21,700, 31,000, 36,800, 47,000, 55,900. The last row gives 55,900 steps, and 55,900 ÷ 7 is about 7,985.7.

The code (4.2.5). The column total becomes the variable total, set up before the loop. The loop runs once per value and reads each with nextInt(). The two reports go after the loop.

import java.util.Scanner;

public class StepTotal
{
    public static void main(String[] args)
    {
        Scanner weekSteps = new Scanner("6200 8400 7100 9300 5800 10200 8900");
        int total = 0;
        for (int day = 1; day <= 7; day++)
        {
            int steps = weekSteps.nextInt();
            total += steps;
        }
        System.out.println("Total steps: " + total);
        System.out.println("Average per day: " + total / 7);
    }
}

Total steps: 55900
Average per day: 7985

The total matches the planning table exactly. The average prints 7985, not 7985.7, because total / 7 divides two int values and drops the decimal part (Topic 1.3). For step counts, a whole number of steps is a sensible answer; if the decimal mattered, you would divide by 7.0 instead, so that the division uses double values (Topic 1.3). Each idea in the chapter did one job: the data set held the facts, the question chose the processing, the written plan and the table worked out the details, and the code carried them out one value at a time.

• A data set is a group of particular facts or measurements about one subject. Each data value is one of those pieces, and results calculated from the data are kept separate from it.
• A data set is analyzed to solve a problem or answer a question. The question decides the processing (a total, a count, a best so far, a comparison with the previous value), and the data must actually cover what is asked.
• Analysis visits the values one at a time: set up what you track once, process each value at its turn, and report after the last value.
• A data set can be shown as a table or chart, and a planning table with a column for each tracked result lets you run and check an algorithm before coding it.
• In Java, each planning-table column becomes a variable, setup goes before the loop, one value is read and processed per pass, and the loop must run exactly once per value to read.

import java.util.Scanner;

public class RiseCount
{
    public static void main(String[] args)
    {
        Scanner readings = new Scanner("41 44 47 46 50");
        int previous = readings.nextInt();
        int rises = 0;
        for (int hour = 2; hour <= 5; hour++)
        {
            int current = readings.nextInt();
            if (current > previous)
            {
                rises++;
            }
            previous = current;
        }
        System.out.println("Rises: " + rises);
    }
}

Rises: 3

import java.util.Scanner;

public class StepGoalCountReset
{
    public static void main(String[] args)
    {
        Scanner weekSteps = new Scanner("6200 8400 7100 9300 5800 10200 8900");
        int goalDays = 0;
        for (int day = 1; day <= 7; day++)
        {
            goalDays = 0;
            int steps = weekSteps.nextInt();
            if (steps >= 8000)
            {
                goalDays++;
            }
        }
        System.out.println("Days at goal: " + goalDays);
    }
}

Days at goal: 4

Check each statement. If the answer is “not yet,” reread the linked subsection.

• I can explain what a data set and a data value are, and tell a data value from a calculated result. Reread
• I can match a question to the processing it needs and tell when a data set cannot answer it. Reread
• I can describe in written steps an algorithm that visits a data set one value at a time. Reread
• I can use a table or chart, and a planning table, to plan and check an algorithm. Reread
• I can trace Java code that reads a data set one value at a time and find setup and loop-bound mistakes. Reread