Academic research: What’s good, what’s bad and how can one be sure?

I teach at a large research university and deeply appreciate the opportunities I have to help shape the careers of aspiring school leaders and, in some ways, the institution of public education itself. As is expected of professors, I have written quite a bit and conducted several research studies.
But I often wonder how much of what I’ve written has actually made a difference in the lives of school practitioners.

Research for school reform
Today, public schools have entered an era of high-stakes accountability and standards-based instruction. Decisions about educational programs and activities are expected to be closely aligned with empirical research and date. Therefore, it can be argued that scholars and practitioners must redouble their efforts to bridge the gap between theory and practice.

School leaders, however, should take what we professors have to say with a grain of salt. Here’s why: Many of us have lost touch with the day-to-day complexity of what life in school is really like, and more importantly, the extent to which grand ideas generated from our research actually benefit school reform efforts.

Self-reported data
Much of the qualitative and quantitative research conducted today depicts self-reported experiences, judgments, and perceptions of reality. Asking about and understanding what, how and why particular individuals think or behave the way that they do can be quite interesting and useful. Well-written narrative accounts of self-reported data can be fascinating, very persuasive and authoritative in tone. However, three particularly harmful weaknesses of self-reported information include fallible memories, biased mental models, and narrow frames of reference.

Imperfect memory
Memories are distorted by time, lack of use, new stimuli and emotions. Vivid as a memory might be, it is never objective and always represents an individual’s subjective account of reality and truth. Granted, memories are often all a researcher has to go no, but at best they represent approximations of past events.

Distortions
Self-reported date are deeply influenced by an individual’s mental models. In plain Language, a mental model represents a person’s worldview, assumptions, beliefs and core values. As first explained by psychologist Kenneth Graik in the early 1940s, the mind constructs “small-scale models” of reality that it uses to anticipate events.

Mental models can be constructed from perception, imagination or the comprehension of
discourse. Like memories, mental models are unique to individuals and never fully represent objective truth. The way we think and feel about the world shapes our interpretation of events past, present and future.

Frame of reference
An individual’s ability to frame situations and problems from multiple perspectives bears upon the credibility of his interpretation of events. Narrow-minded people see the world through a singular lens and often fail to empathize with other’s perspectives or feelings. They also fail to consider alternative explanations of the truth or reality.

Correcting subjective reports
To correct for the distortions of memory, mental models and framing, competent researchers look for corroborating evidence. They evaluate self-reported information in light of the narrative explanations of others, confirming or disconfirming documentation and observation. Without such careful craftsmanship, self-reported data should always be treated with caution and never extrapolated to explain the behaviors, perceptions, values or beliefs of larger groups.

Implement or ignore?
As results of studies are presented, practitioners must consider empirical research as encyclopedic rather than plenary. In education, even the most credible research is subject to differing interpretations and rarely depicts the final word. Good research is a road map, rarely a destination.

Relevant research
Useful evidence can come in many forms. Obviously, a well-designed research study provides the most credible form of evidence about what works in schools. But whether the research is qualitative or quantitative in form is less important than its relevance to problems of practice. And the most important issue for practitioners is not the broad relevance of research to the field, but the relevance of research findings to particular contexts and circumstances

Anecdotal evidence
Firsthand accounts of what works and what doesn’t are important sources of information. At the very least,. Such accounts can (and should) provoke inquiry into the research literature. Corroborating the anecdotal evidence with reports from several expert practitioners and from the research literature, of course, is more credible than simply relying on individual anecdotal accounts. However, the intriguing potential of anecdotal evidence is that it can stimulate experimentation and innovation.

Trust your instinct
If an empirically tested strategy or program doesn’t feel right for your school or district, it probably isn’t right. Intuition can be useful for judging when to slow down and search for additional information.

A recent study of 92 school principals revealed that the principals frequently used their intuition when making important decisions. Most important, the study also revealed that, even though intuitively derived decisions didn’t always turn out well, the principals always regretted it when they failed to follow their instinct.

Specialized Terms
Practitioner (n)从事者
Someone involved in a skilled job or activity

Empirical (adj)实证过的
Based on what is experienced or seen rather than on theory

Corroborating evidence (n phr)有力佐证
New information that adds proof or certain reasons to an account, statement, idea, etc.

Plenary (adj)完整的
Complete; including all members or information

Anecdotal evidence (n phr)没有确凿根据的资讯；传言推论
Information that is not based on facts or careful study
Vocabulary Focus

Depict (v)描写
To represent or show something in a picture or story

Vocabulary Focus
Institution (n)制度
A custom or tradition that has existed for a long time and is accepted as an important part of a particular society

Aligned (adj)符合的
Agreeing with and supporting another’s idea or goal

Redouble (v)加倍
To make something much more than before; to increase something

Take (something) with a grain of salt (idiom)持保留态度
To not completely believe something that you are told because you think it is unlikely to be true

Biased (adj)有偏见的
Showing an unreasonable like or dislike for a person or idea based on personal opinions

Discourse (n)论述
Communication in speech or writing

Empathize (v)感同身受；同情
To be able to understand how someone else feels

教育研究与实务的隔阂

学术研究： 什么好、什么不好，又如何能确定？

吴茵茵 译

我在一所大规模的研究型大学任教，我衷心感激能有机会帮助那些有志于担任学校领导阶层职位的学生发展其教育事业，以及就某些方面而言，帮助塑造公立教育的制度。我符合教授职位的工作要求，写了不少文章，也从事了几项研究，不过我常在想，我的著作实质上究竟对教育工作者的人生有多少影响。

促进学校改革的研究
目前公立学校*已进入高风险责任制和标准化教学的时代。大家认为关于教育课程和活动的决策理当紧密切合实证研究与资料。因此，我们可以说学者和教育工作者必须加倍努力缩短理论与实务的差距。

然而，学校领导阶层对于我们教授的意见应保持保留态度。原因如下：我们之中许多人已经不太熟悉每天处理校务必须面临的复杂情况，更重要的是，也搞不清楚自己研究出来的那些崇高伟大的论点对于推动学校改革实际帮了多少忙。
 
*译注：通常指公立的小学和中学。

自我论述资料
目前许多质化和量化的研究皆援引自我论述经验、判断和对显示状况的认可。调查和了解个人有什么样的想法或行为，以及如何和为何有这样的想法或行为，可能非常有趣又有用。写得好的自我论述资料，其内容有可能很精彩迷人，非常有说服力，而且语气上有权威性。不过，自我论述资料却有３个特别有害的缺点，那就是跟事实有出入的记忆、有偏见的心智模式以及狭隘的行动判断参考构架。

有缺陷的记忆
记忆会因为时间过久、不常使用、或新的刺激和情绪影响而扭曲。记忆虽可能鲜明，却绝不客观，只能代表个人对于现实和事实的主观陈述。尽管如此，记忆通常是研究员在研究之初仅有的资料，不过记忆充其量也只呈现往事的概况。

判断力的参考架构
一个人有无能力以多种观点看待情况和问题，会影响他对事情的诠释可靠与否。心胸狭窄的人用单镜头看世界，对于别人的观点或感受往往无法用心去了解，对于事实或现况也想不到可有另一种解释。

修正主义观论述
为了矫正扭曲的记忆、心智模式和参考架构，能干的研究员寻找有力佐证。他们根据别人用叙事方法做的说明来评估自我论述资料，也鉴定这些文献和观察的可信度。若无如此慢工细活，自我论述资料应一直谨慎处理，绝不能用来推断并解释较大群体的行为、认知、价值观或信念。

付诸行动或漠视？
研究成果发表时，教育工作者必须视实证研究为触角广泛的参考，而非完整又绝对的依据。在教育方面，就连最可信的研究 都会受到不同诠释的影响，而且很难描述出定论。详实的研究是地图，很少是目的地。

相关研究
有利的证明各式各样。不用想也知道，对于什么方案在学校行得通，精心设计的研究是最可信的一种证明。然而不论研究在形式上是质化还是量化，都不及它与实务问题的关联性重要。对于教育工作者来说，最重要的不是某项研究能否广泛应用于教育领域，而是研究结果能否应用于特定背景和情境。

无科学根据的轶事
关于什么可以和什么不可以的第一手记述是重要的资料来源。这种技术资料至少能够（也应该）促使研究者深入探究相关的研究文献。以几位内行的教育工作者的自我论述报告和研究文献的技术来佐证没有科学根据的证据，当然比纯粹依赖个人的轶事性技术更加可靠。不过，无科学根据的论证的有趣特点就是它可能激发实验和创新。

Source: Studio Classroom Advanced October 01~03, 2007