2003 Annual Colloquium

on Research in Mathematics and Science Education


Regina M. Panasuk

To advance the field of educational research and to make it more rigorous in support of evidence-based education, in November 2002 the Institute of Education Sciences (IES) was established. Through its research initiatives and national research and development centers, IES engages in research activities that will result in the provision of high quality education for all children, improvement in student academic achievement, reduction in the achievement gap between high-performing and low-performing students, and increased access to and opportunity for post-secondary education.

Currently, there has been an explosion of the articles related to scientifically based standards. What constitutes rigorous scientific methods for conducting educational research is an issue of the most recent debates among researchers and legislators. The National Research Council, the US Department of Education, the National Research Foundation and other research and legislative organizations issued documents that intend to discuss and clarify the nature of scientific inquiry in education to at best support high quality educational research endeavors.

In summary, to be qualified as a scientifically based research, investigation must

(i) employ systematic, empirical methods that draw on observation or experiment;

(ii) involve data analyses that are adequate to support the general findings;

(iii) rely on measurements or observational methods that provide reliable data;

(iv) make claims of causal relationships only in random assignment experiments or other designs (to the extent such designs substantially eliminate plausible competing explanations for the obtained results);

(v) ensure that studies and methods are presented in sufficient detail and clarity to allow for replication or, at a minimum, to offer the opportunity to build systematically on the findings of the research;

(vi) obtain acceptance by a peer-reviewed journal or approval by a panel of independent experts through a comparably rigorous, objective, and scientific review; and

(vii) use research designs and methods appropriate to the research question posed.


By no means do the above standards reject the qualitative methods employed to obtain reliable evidence and knowledge. Nonetheless, we must be very clear that qualitative case studies alone do not allow the ways of knowing which elements are crucial or relatively more important than the others. In other words, those studies should not claim that they identify causal effects, but rather provide description that can include associations among variables.Unfortunately, there is a common tendency to make invalid inference by establishing a causal effect from simple correlations.

I believe we have to be aware of and give a serious consideration to the current trend and demand for high quality research.To inform you about a recent discussion related to scientifically based research, I obtained permission from the American Educational Research Association (AERA) to reprint an article published in Educational Researcher, Jan/Feb 2003, Design Experiments in Educational Research, by Cobb, Confrey, diSessa, Lehrer, and Schaubl.This feature article of the journal is focused on the goals and procedures of conducting experiments to develop theories.

You may want to visit the Educational Researcher on line and examine other articles in its recent issue.



Table of Contents


Design Experiments in Educational Research

Paul Cobb, Jere Confrey, Andrea diSessa, Richard Lehrer, and Leona Schauble
reprinted from Educational Researcher

The Effects of Varying Retention Intervals of Three and Eight Month, Within Block Scheduling, on High School Studentsí Retention of Algebra II Concepts and Skills and Achievement in a Follow-On Precalculus Course

David. E. Arias, Andover High School 

A Study of Teachersí Perceptions of Learning Organizations in Taiwan

Cheng-Hung Chen and Shu-Fang Chien, UML


Analysis of Student Outcomes: Evaluation of the DOE Middle School Mathematics Initiative

Donahue Institute


 Educational Resources

 A Passion for Pi, or a Studentís Voyage through Mathematical Discovery

Doug Ruby and Danielle Tarnow, UML 

Chemistry Curriculum: Why Study Student Alternative Conceptions?

Peggy LaBrosse, Holis High School, NH