After Sputnik – American scientific literacy tested online: An interview with Jon Miller

Jon Miller is the John A. Hannah Professor of Integrative Studies at Michigan State University. Since 1987, he has directed the Longitudinal Study of American Youth (LSAY). In addition, Prof. Miller has measured scientific literacy in the U.S. and around the world for 30 years. As a Fellow of the American Association for the Advancement of Science, Jon has been a leader in the area of measurement and analysis of the public understanding of science and technology. For six years, he was President of the International Council for the Comparative Study of the Public Understanding of Science and Technology. He's acted as consultant to the National Institute of Science and Technology Policy in Japan; the China Association for Science and Technology; and the European Commission. Our interview with Professor Miller appears below.

Your work includes studies of public understanding and perception of science, in both the U.S. and other countries. How do you see the U.S. in terms of scientific literacy as compared to other nations?

I have a recent paper comparing the United States* to 33 other countries, and we are second in the world; only Sweden has a higher percentage of adults who are more scientifically literate. Norway, Denmark, Finland and the Netherlands are quite close to us, and the six of us as a cluster are pretty far ahead of the rest of Europe and Asia. In one sense that's good news. The bad news, however, is that only 25% of Americans are scientifically literate; 75% are not. What we've determined is that the U.S. is the only major country in the world that requires university students to take one year of general education for a baccalaureate. In most of the rest of the world, people earn one in three years by skipping the general education and going directly into a major. We do something called structural equation modeling that predicts who is/isn't going to be scientifically literate and why; the strongest single predictor in the U.S. is having participated in a college science course.

Now, it turns out the high school science courses are probably not very good when you look at the Third International Mathematics and Science Study (TIMSS) and the one before that. But half of our students go on to college, which is still high compared to many parts of the world. The Japanese send 20% to college, so we're still sending far more. And the half of Americans who attend college almost always get a year of college level science – that's unique.

Can you provide another highlight in terms of your findings?

Generally speaking, Americans are very receptive to science and have high expectations for it. One comparison in particular captures the whole idea. In 1957, six months before Sputnik, a national sample of Americans was asked, “On balance, is the world better off or worse off because of science?” 88% of people said “better off,' and in 2007 we asked the same exact question in a Knowledge Networks survey. Again, 88% said “better off.” We've measured it 15 times in between, but the point is, over the last 50 years – even before the beginning of the space race – Americans believed that science was a good thing. They still believe that it makes your life healthier, easier and more comfortable and provides more opportunities for the next generation. People came out of the Second World War being very receptive to science .

Antibiotics had just become available. People thought DDT was a wonderful thing. Americans have been willing to fund science, and what's even more important, they're willing to buy it—digital cameras, cell phones, computers—and we fly everywhere in airplanes. I think we're going to be coming back to nuclear power: that's going to be one of the few ways to avoid burning oil. Not only is burning oil very bad for the environment, but it's also $100+ a barrel. Ultimately I think that down the road, fusion technology will provide electricity, but we've got probably a hundred years of transition before we get to that. Our children and grandchildren will live in interesting times; it's not going to get less interesting.

Could you speak a little bit about the importance of online survey research quality to your work?

Knowledge Networks' recruitment approach – which includes people or households that do not have either a computer or an Internet connection, and providing access through their television sets with an MSN-type box – has been very useful in closing the digital divide issue on online surveys. Many of the other companies that do online surveys are basically just waiting for people to volunteer on various websites, which is problematic for two reasons. Having taught statistics for 30 years, I'm very wary about whether a sample of volunteers is anything like a sample of non-volunteers. If you have a sample of volunteers, chances are that you'll get answers that sound like they're from people more willing to volunteer than they actually may be. We also know that people who volunteer tend to have other upscale characteristics that probably tend not to be a true reflection of the American population. The probability basis of Knowledge Networks is what truly distinguishes it for me. KN's online questionnaire formats are also good. The staff has expertise in this area, and we get data back in a very timely fashion. From the point of view of a research director, when you're trying to run a study, those are the things you want.

Is there a particular survey technique – a way of asking questions – that has evolved from your work?

Again, if you're trying to measure things over time, you need to have questions that have a certain durability. In the 70's, I realized that things like fluoridation and Salk's polio vaccine – 50's issues – had gone by the board. So I complained to my friend, Leon Lederman, “I need science that will be true in 100 years,” and he said, “In 100 years, 1,000 years, I think that we'll still believe that atoms and molecules are the way it works.” I said “Okay, let's do that.” We also thought, “DNA will be with us for as long as we're out – also the nature of the universe, or earth going around the sun.”

In terms of eliciting an accurate response, we developed a two-step process. We'd say, “When you read a newspaper or a magazine, and you see the word 'molecule,' do you have a clear understanding of what that means?; a general sense of what it means?; or not much of an idea?” If they have a clear understanding or a general sense, we'd then say, “Well, tell me… what's a molecule?”; but if they say, “Not much idea,” they're usually telling you the truth. 99.9% of them can't give you a syllable on it. We ask people about other topics – radiation, stem cells, and carbon footprint – in the same way. I tell people that scientific literacy is the ability to read the Tuesday New York Times; it's the best science writing in the world.

What advantages does an online platform offer your work?

Many years ago when people did personal interviews – nowadays prohibitive due to cost, as well as by the fact that you simply can't get to the very rich and the very poor – we used to hand people a card with a question on it, or a set of choices or some metric. It had 0 to 10 and a line drawn between them with intermediate points, where the respondent could put either numbers or even words. The card provided the respondent with a visual way to gauge his/her response. Some of the national health surveys are still done this way, and it greatly improves the quality of the response; you can see the metric laying in front of you. Some people are better at imagining things, which can make a big difference, particularly across educational backgrounds. You have a much more physical thing to look at—especially with large print.

In a telephone interview, you can't hand a respondent anything. You can only speak to them, so they have to be able to receive the message, in psychological terms, then encode it, decode it—think about it and respond. Some people are just not as fast as others. In addition, some interviewers speak quickly, which complicates. We find more response bias of respondents who are yea-sayers – trying to be compliant – because they don't understand the question, or it's said too fast. Or sometimes they don't mentally process the words into ideas and concepts that they can use.

With Knowledge Networks' online format, of course, all this is on-screen. Some people can read a question and answer in three seconds. Others take 30 seconds, which is okay – if that's how long it takes to digest the question, so be it. From the point of view of quality, this gives me a greater sense of comfort – that the person sees the question as well as the metric.

Many people who are serious about this kind of research want to know where our response biases are; sometimes, we even find things by accident. Change a word or two in a question and all of a sudden you get a very different answer – even a different pattern of answers than before. It just reminds us that people don't always “hear” the same thing. Don Dillman and others are doing very useful work with screen formatting, and the difference it makes in putting space around things, as well as font size. I know that Knowledge Networks has been contributing to that kind of work, because it's what you do. If some formats are better, that's what we all should be doing.

In which area of science do you feel the U.S. will make real headway in the future?

The 1957 study that I mentioned earlier turned out to be sort of the first, best, and last mission of scientific understanding prior to the beginning of the space race – which was not just a hardware race, but the beginning of an emphasis on science in our country. We cranked up graduate education; undergraduate education; high school education in science and mathematics across the country; and we substantially increased science budgets in health and medicine and basic sciences across the board. It was beginning of a 50-year era of science in this country.

You could argue that the 20th century was a century of physics: we did airplanes, radios, televisions, transistors, computers and satellites – all of which use various forms of applied physics. You move electrons across various objects – or as one physicist said, who was not taken with the space program: If you burn enough kerosene, you can move something up an incline.

However, the 21st century is going to be the century of biology. Not only have we decoded the map to the human genome, but we're close to mapping human proteins; used together, you can probably account for 95% of what we call human illness. Genes produce proteins, which either help prevent or cause various kinds of problems. More people are seeing the practical advantages to understanding genetics better than we have in the past; the wheels are in motion.

*Data source: KnowledgePanel®.
Image source: NASA