The TESS Opportunity for the Social Sciences:
An Interview with Dr. Jeremy Freese

Jeremy Freese, Ph.D. is one of the country's leading experts on connecting the biological, psychological and social aspects of people's lives as they intersect with social change. He is also one of two Principal Investigators with TESS (Time-sharing Experiments for the Social Sciences) – a testing ground for the social sciences, funded by the National Science Foundation. As a Professor of Sociology at Northwestern University, his interest areas range from science and technology to health & illness and social inequality. He has done work on social science methods and co-authored a book on the analyses of categorical data. Before teaching at Northwestern, Dr. Freese was a Professor of Sociology at the University of Wisconsin-Madison and a Robert Wood Johnson Scholar in Health Policy Research at Harvard University. The following interview, conducted in June 2009, digs into aspects of this important program.

What is the overall mission of TESS, and who funds it?

Our mission is to promote experimentation in the social sciences, by providing an increased opportunity to conduct surveys – especially for graduate students and assistant professors. In other words, it is for people for whom it would be very difficult and inefficient to go out and raise the money to do a specific experiment themselves, but who have great ideas for studies. We can do that study at no cost to them and at a considerable savings in time.

In the social sciences, you have a large amount of survey research – which is great for getting descriptive information about the population. But surveys are very limited for being able to make claims about causes and effects. They can also be limited in terms of the quality of the measures that can be obtained just from asking people simply to report their attitudes, opinions, or beliefs. Meanwhile, experiments in the social sciences are overwhelmingly on undergraduate students. It's unclear how well results from undergraduates can be generalized to the full population. The program promotes doing experimental research on the general U.S. population.

Do you plan to change TESS from what it was earlier – will there be new features?

A major change is that we're now allowing larger experiments than before. The proposals we're accepting now have a maximum study size that's two or three times what people were able to do in the past. Let's say that you have an experiment that involves ten survey questions. Before, the maximum number of respondents you could have was 800 respondents with a 10-question experiment. Now, depending on conditions, we may be able to survey 2,000 to 2,500 respondents.

The other thing is, there's a lot that people were actually able to do with TESS experiments using the Knowledge Networks platform that we're hoping to better publicize, such as the ability to do pre-test and post-test experiments because of the panel design; to pay respondents based on their decisions; to do experiments that make use of the response time that it takes a person to answer – in addition to all the multi-media possibilities for Knowledge Networks' panel. We're hoping that those get more use, and we're publicizing them more prominently on our Web site: http://tess.experimentcentral.org.

Who is eligible to submit a proposal, and what is "Time Sharing"; why is it beneficial?

We accept proposals from any graduate student, fellow, or faculty member who's connected with the social or behavioral sciences – anywhere in the world.
Time sharing means that, rather than having each individual separately collect demographic variables for the study and figure out how to do the sampling, these are done centrally. In this respect, KnowledgePanel® is intrinsically a time-sharing platform. On top of that, we offer savings in administrative costs by handling the relationship with Knowledge Networks for individual investigators. So instead of each investigator having to contract separately with Knowledge Networks, we handle all of that and then field the individual experiments for free, using the money we've obtained from NSF.

What aspects of KnowledgePanel® drew TESS to use it as its core sample?

The first aspect was the Internet-based sample. That is a natural fit for what TESS wants to do. The Internet allows so many possibilities with experimentation, from providing different kinds of experimental materials to respondents, to allowing different types of measurements. The Internet also allows a lot of flexibility, in terms of what questions you ask of what respondents – more than phone surveys, and certainly more than you can in a mail survey. Given that some Internet-based surveys aren't the best fit for TESS, and given that we are doing general population experiments, we wanted to be sure to get the population sampling right. Knowledge Networks has a sophisticated sampling model, and it also has a lot of flexibility with respect to the questions that you can ask and the different things that you can do. For us, it was a fairly obvious decision to go with KnowledgePanel.

Have you done any of your own research with Knowledge Networks?

I haven't done anything yet, but I've been so impressed with working with Knowledge Networks so far. One thing about being part of TESS is that you get all these ideas for your own projects from seeing the interesting things that others are doing. Certainly I'm considering the possibility of using Knowledge Networks. I think that the panel design offers a lot of great possibilities for looking at how respondents change over the course of six months or a year in their attitudes on different topics. For example, one of the things in which I'm interested is public opinion about genetics policy. That's changed very quickly; and so the idea that you can get changes in attitudes a year or so apart is very appealing.

What criteria will you use to determine who gets funded by TESS?

We've recruited a team of 40 associate PIs – leading researchers across the social sciences – who help us select reviewers. People send us proposals, and every proposal is reviewed by at least two outside reviewers. We also internally evaluate each proposal. We evaluate the methodology of the experiment, and whether it seems like it's a good fit for the overall goals of TESS. For a TESS experiment to go forward, it has to pass both external and internal review.

What sorts of experiments need to be done on social issues?

We're in the middle of a lot of different kinds of social policy changes – especially with the change in presidential administration. Public opinion is vitally important to policy decision-making. One of the things that experiments allow you to do is evaluate the more straightforward measures in public opinion using multiple and indirect ways. Social issues are a broad class of experiments that TESS can address to improve the measurement of public opinion – specifically opinions about policy. Areas of particular relevance include attitudes about health care and about the financial system.