to employ someone else from outside the Federal Government). While these new employees are often quite bright and capable, few of them have had extensive training and experience in research. Other recent staff additions have had prior government service, but most of them also have had little training or previous work experience in research and development. As a result, despite continued staff hiring, few of the new employees would be considered well-trained, nationally-recognized researchers and scholars.

As OERI was restructured in 1994 and early 1995, it was necessary to shift at least some of the existing staff to new positions. Sharon Robinson was committed to giving all employees the opportunity to apply for any position in the agency— even though the re-organization of the office was still underway at that time so that management often did not have an adequate sense of the actual staff needs for each unit. 129 Since approximately 95 percent of the staff were granted their first choice of jobs, some employees ended up in positions where they did not have the necessary experience or technical training to do those jobs. 130 While all OERI staff were strongly urged to enroll in team building and customer-service workshops, they did not receive comparable encouragement or opportunities for taking statistical or technical training that might have been particularly helpful in their new positions. 131

C. NATIONAL RESEARCH INSTITUTES

The centerpiece of the OERI reauthorization was the creation of the five National Research Institutes. While the proponents of these institutes had envisioned largescale operations modelled after those in NIH, the actual amount of monies available was rather modest-$43 million for all of them in both fiscal year 1996 and fiscal year 1997 (only about $10 million more than had been spent just on the R&D centers in fiscal year 1995):

1) Achievement Institute-$12.9 million.

2) At-Risk Institute-$12.9 million.

3) Policy Institute $4.3 million.

4) Early Childhood Institute—$6.45 million. 5) Postsecondary Institute-$6.45 million. 132

The funds appropriated for three of the five National Research Institutes were well below what many of its original supporters had believed was necessary to set up such institutions. Indeed, some supporters of the National Research Institutes suggested that given the small amount of total funding, only the Achievement Institute and the At-Risk Institute should have been created at this time with a larger budget of $21.5 million each.

The legislation mandated the creation of the five National Research Institutes and provided a long list of possible topics that each institute might research-more than could be reasonably accomplished given the limited monies available. After considerable internal and external discussion, OERI developed its own general, but often not very specific set of priorities for each institute and established at least one center for each institute.

Initially, the five National Research Institutes put most of their $43 million into seven R&D centers (expanded to eleven centers by fiscal year 1997). While the law stated that they had to spend at least one-third on centers, in fiscal year 1997 they spent $31.15 million on them (57.7 percent). The institutes allocated $14.3 million to field-initiated research-26.5 percent of the total institute funds. Only $8.55 million was spent on special studies, cross-cutting activities, fellowships, or peer reviews (15.8 percent of the total institute budget). 133 Thus, the expenditures for the five National Research Institutes were more diversified than that of the former Of

129 Sharon P. Robinson, "Job Preference Selection for the New OERI," OERI memorandum, September 8, 1994.

130 Sharon Robinson, "The New OERI Structure and Staffing Plan," OERI memorandum, October 26, 1994. A lot of the staff choose to stay where they were, but many did apply for new positions especially since the reorganization of OERI forced many individuals to find a new home.

131 Preston G. Foster, "Note to All OERI Employees," OERI memorandum, April 28, 1995. 132 U.S. Congress, House, Committee on Appropriations, Hearings... Appropriations for 1997, Part 5: Department of Education, p. 1387.

133 U.S. Congress, House, Committee on Appropriations, Hearings

Appropriations for

1997, Part 5: Department of Education, p. 1388; U.S. Congress, House, Committee on Appropriations, Hearings on the Departments of Labor, Health and Human Services, Education, and Related Agencies, Appropriations for 1998, Part 5: Department of Education, 105th Cong., 1st Sess. (Washington, DC: Ú.S. Government Printing Office, 1997), p. 1485.

fice of Research (OR), which had focused mainly on R&D centers. Yet even now the R&D centers received almost 60 percent of the entire budget.

There have been significant changes in the size of the annual funding for the R&D centers. In the late 1960's and early 1970's, the R&D centers' budgets averaged between $3 and $5 million annually (in constant 1996 dollars). Over time, average funding for the centers decreased sharply--in the late 1970's and early 1980's averaging between $2 and $3 million annually (in constant 1996 dollars). With the great expansion in the number of centers in the late 1980's, average annual funding for them became even smaller-reaching a low in fiscal year 1991 of about $1 million (in constant 1996 dollars).134

The National Academy of Sciences (NAS) had recommended in 1992 that the annual funding for all R&D centers should be at least $3 million (or $3.35 million in constant 1996 dollars).135 Others argued for even more money for R&D centersperhaps somewhere in the range of $6 to $8 million annually.136 Given the general consensus that the R&D centers in the early 1990's were much too small and the specific NAS recommendations for at least $3 million minimum, it was somewhat surprising and quite disappointing that Congress mandated only a $1.5 million annual minimum for these institutions.

While the total funding (in constant 1996 dollars) for the centers remained relatively constant from fiscal year 1992 to fiscal year 1997, the number of them has decreased by more than one half. In fiscal year 1992 there were 22 R&D centers while 5 years later there were only 11 institutions. As a result, the average annual funding (in constant 1996 dollars) of the centers has increased substantially from $1.28 million in fiscal year 1992 to $2.76 million in fiscal year 1997-more than doubling in size.

Looking at the average expenditures is somewhat misleading as most centers are still very small today. Only 2 of the 11 R&D centers in fiscal year 1997 are wellfunded annually: (1) the National Center for Research on the Education of Students Placed at Risk ($5 million) and (2) the National Center for Research on Education, Diversity, and Excellence ($4 million). But eight other centers are much smaller at $2.5 to $2.8 million each. Moreover, OERI has just funded another R&D center at only $1.5 million. 137 Given that in the past frequently only half of center funding went directly for research or development expenditures (the rest spent in other areas such as dissemination, administration, and overhead costs), it is likely that serious underfunding is limiting the actual research that can be attempted at these small R&D centers. 138

It is very important to know how the new R&D centers focus their research and development energies. Unfortunately, there is no in-depth analysis of the number of projects in the new centers or of the actual intellectual cohesiveness among projects within each center. It does appear, however, that many of the new centers include partners at several different colleges and universities—thereby making coordination even more difficult. For example, the new $1.5 million Center on Policy and Teaching Excellence is a consortium including the University of Washington (Seattle), Stanford University, Teachers College at Columbia University as well as faculty at the Universities of Michigan and Pennsylvania. 139

Based on the original proposal for the National Center for Research on the Education of Students Placed at Risk as well as discussions with several OERI staff members familiar with some of the other centers, many of the new centers still seem to support too many projects and projects which are too small; and some of these smaller projects which could be closely integrated with each other are not. Moreover, with a few notable exceptions, there is little evidence that most of the new centers are engaged in the systematic, long-term development of educational materials and models-a continued weakness of OERI in general. Instead, the new centers continue the previous practice of focusing almost all of their attention on pursuing basic and applied research or funding dissemination activities.

While annual funding for many of the individual projects within the centers continues to be only $30,000 to $50,000 annually, the size of the field-initiated grants

134 On the changes in the size of centers over time, see Vinovskis, "Analysis of the Quality of Research and Development."

135 Atkinson and Jackson, Research and Education Reform, pp. 150–51.

186 Vinovskis, "Analysis of the Quality of Research and Development."

137 The National Center on Policy and Teaching Excellence at the University of Washington in Seattle was funded by the National Institute on Educational Governance, Finance, Policymaking, and Management.

138 At the same time, of course, OERI was under great pressure to fund some of these smaller centers in order to provide at least some coverage in areas seen as substantively and politically important.

f39 U.S. Department of Education, News (September 9, 1997).

has grown considerably. The Postsecondary Institute, for example, funded six fieldinitiated studies in fiscal year 1996 that averaged $210,000 annually (the grants covered 2 to 3 years of work). Thus, whereas one of the more attractive features of the center system in principle was that it concentrated more research monies on a few large-scale, long-term projects that might be done through grants to individual scholars, today the situation may be just reversed-many of the field-initiated grants are larger than some of the particular projects within the centers.

The research plans for many of the individual projects in the old R&D center applications were not well-developed or clearly articulated. Some of these projects frequently devoted the first year to a review of the secondary literature; afterwards a more careful, detailed design of the subsequent research work was developed. 140 Moreover, center awards often included projects of very mixed quality-some parts excellent and some parts so weak that they probably would not have survived by themselves a rigorous outside peer review.

Successful field-initiated grants, on the other hand, may have more focused and developed research proposals in order to be able to win the more intense competition for individual funding. And as applicants are expected to be aware of the secondary work in the field, there may be less need to devote the first year of the project to reviewing the existing literature before proceeding to their original work. In addition, as most field-initiated grants are for only 2-3 years, they cannot afford the luxury of simply reviewing the secondary literature for the first year before undertaking new research.

Since the center grants focused on a particular problem area, however, it was easier for OERI to target its resources to these sites rather than hold a relatively general and open-ended field-initiated grant competition. The centers also provided opportunities for different researchers to work together on a common set of issues and required them to disseminate their findings more broadly than most individual researchers are willing to do.

These obvious and real advantages to the center approach might be duplicated at least in part by using mission-oriented competitions for field-initiated research. That is, OERI might call for research on a particular area, such as summer learning or the education of pregnant adolescents, and then allow researchers to propose their own topics. Following the practice of many other Federal research agencies, the institutes could convene the funded individual researchers to exchange ideas amongst themselves. Moreover, the institutes might also synthesize the results from these mission-oriented individual grants and disseminate the results more widely among other researchers and educational practitioners. In order to administer and intellectually oversee some of these mission-oriented competitions, OERI might hire a leading expert in that substantive area as an excepted-service employee for 3-6 years. In any case, the advantages and disadvantages of funding research through individual grants or the existing R&D center system needs further careful evaluation in the near future.

Given that there are now fewer, but somewhat larger centers, one might imagine that there will be even closer integration of them-especially because OERI was directed to develop a coherent, long-term research priority plan to guide all of their investments. Unfortunately, the results in terms of the integration of the centers have been rather disappointing so far. Faced with the slow development of an overall OERI research priority plan, which will be discussed later, the center competition was held well before any coherent research plan had been developed. And rather than working closely together to create an integrated strategy for a new set of centers, each of the five institutes pursued their agendas and interests quite independently of each other. Moreover, the announcement of the competition for the seven centers in the Federal Register lacked an overall framework or any coherence, so that the applicants and the reviewers for each of the centers treated them in almost total isolation from each other. The net result is that while the eleven centers may be good individually, they have not been integrated conceptually or operationally with each other.141

140 Not everyone is agreed upon the need for having prior research designs for the center projects. At an OERI workshop, a former chair of the center director's group argued against requiring centers to submit to OERI a detailed research design. This person felt that they were unnecessary and time-consuming to develop. This was a minority view, however, as most other researchers and center directors did see the value of providing research designs for individual projects.

141 As will be discussed later, an alternative to this more sporadic and uncoordinated approach to creating centers as to use a life course analytic framework that had been developed for OERI in June/July 1995 by Vinovskis. For more discussion of this alternative approach as well as a conceptual critique of the announcement in the Federal Register for the center competition, see Maris A. Vinovskis, History and Policymaking: Exploring the Uses of History for Educational

One factor which has contributed to the lack of coordination among the five institutes (as well as among the other units within OERI) is that no one is really in charge, organizationally or intellectually, of the overall research agenda. Under the old system, all of the R&D centers were located within the Office of Research (OR) which was headed by Acting Director Joseph Conaty-an experienced and respected researcher. Now each of the institute directors reports directly to the Assistant Secretary who is able to supervise them only minimally; no one is responsible for overseeing intellectually the research operations of the institutes as a whole. While various ad hoc schemes have been tried to improve coordination among the institutes (with occasional success), in practice the system has become even more fragmented. Moreover, the expected close coordination between the work of the institutes and the rest of OERI has not materialized except in a few isolated instances. Adding to the confusion and lack of research coordination among the five institutes is the fact that OERI itself seems to have less intellectual leadership and direction than 5 years ago. Diane Ravitch had been very interested and involved in the intellectual direction and research agenda of OERI; but Sharon Robinson was less concerned and focused more on translating existing research findings for the agency's customers. And whereas Vinovskis helped to coordinate OERI research activities as the Research Advisor to the Assistant Secretary, when he left in August 1993, Robinson did not appoint anyone to replace him. Although in principle Robinson endorsed the concept of an OERI Research Advisor on several occasions, that position remained vacant. 142 As a result, during the past 4 years there has been a major vacuum in the overall guidance of intellectual and research activities in OERI-one that has become all the more glaring because of the elimination of the Office of Research (OR) and the coordinating and leadership role that some of its directors in the past had played.

In terms of staffing, the five institutes have had mixed experiences. The five acting institute directors were selected from the existing OERI staff and most of them had previous research experience though few were still personally active as researchers. It was expected that OERI would be able to recruit some outside researchers as permanent directors, but the final competition resulted in the appointment of only one outsider as an OERI institute director.

While the overall OERI staff had been reduced by approximately one-fourth from fiscal year 1992, the staffs of the five institutes in fiscal year 1995 were approximately the same size as they had been in the Office of Research (OR). Over time, however, the number of staff members in these institutes has decreased considerably. In June 1997 there were only 64 institute staff members—a substantial decrease from the 77 staff members present in October 1994 (about a one-sixth reduction).

The five institutes fared relatively well within OERI in terms of having generally competent leadership and a sizable staff, but they still lack an adequate number of trained and distinguished researchers. While some current staff have had graduate training and experience in research, most have not. Nor are there many nationallyrecognized scholars in the institutes. In addition the distribution of the researchers among the institutes is uneven. For example, until very recently the Early Childhood Institute for several years did not have any experienced researchers on its staff even though its mission called for working closely with the research community. Moreover, several of the other institutes had only 2 or 3 researchers-hardly enough for a critical intellectual mass to carry out the research and development functions expected of the institutes. As a result, without the national intellectual leadership originally envisioned for the staff of the institutes the production of high quality research and synthesis has not progressed satisfactorily.

One of the primary objectives for the new institutes was to produce high quality research as well as critical syntheses of secondary works that would be helpful to policy makers and educators. So far, most of the institutes have produced only a few publications-usually directories of researchers and service providers, materials for parents, or limited analyses of isolated issues. 143 Regrettably, the shortage of

Policymaking (New Haven, CT: Yale University Press, forthcoming), chapter 8: “A Life Course Framework for Analyzing Educational Research Projects."

142 At one time Sharon Robinson announced that Edgar Epps, a recent OERI consultant from the University of Chicago, would be the Research Advisor. However, he really did not try to fulfill that role and had relatively little impact on the direction and coordination of the overall research activities in the agency.

148 Susan Gruskin, Kim Silverman, and Veda Bright, Including Your Child (Washington, DC: U.S. Department of Education, 1997); Janice Owens, Learning and Earning: Analysis of HEA Title II-B Graduate Library Fellowship Program Recipients, Fiscal Years 1985-1991 (Washing

Continued

active researchers in the institutes as well as the current general lack of OERI encouragement of staff to write for scholarly publications means that it may be unrealistic to expect many broad-based analytical studies or syntheses from the institutes in the near future. 144

Despite the fact that the five National Research Institutes were to be the centerpiece of the new OERI, the agency seems to have relegated them to a secondary role. Despite the modest $43 million appropriations for the institutes for fiscal year 1996, the following year OERI did not ask for any increase in that budget-preferring instead to request an additional $250 million to support a new Technology Literacy Challenge Fund which would be used to integrate computer technology into the_classroom.145 Congress, however, added $16 million for CERI in fiscal year 1997. After considerable debate over how that additional money should be spent, OERI allocated $11 million of those funds for the institutes-most of which has gone for an additional center ($2.0 million), more field-initiated studies ($5.5 million), or special studies ($2.8 million).146

As large programs are added to OERI, the agency has often decided to set them apart from the institutes rather than having the institutes supervise the new activities. For example, although the Achievement Institute is concerned with improving the use of technology in the classroom and specializes in developing better tests, OERI decided not to have that Institute oversee the new high priority initiatives in educational technology or testing. Indeed, staff from the institutes are sometimes temporarily reassigned to work on these new projects under the supervision of the newer unit thus further diminishing the importance and functioning of the National Research Institutes. As a result, one might easily make the argument that in some ways the five National Research Institutes now are less important and less central to the mission of OERI than the Office of Research (OR) was 5 years ago— a startling reversal of what the Congress and others had intended for the new OERI. Perhaps even more surprising is the fact that the diminishing of the relative role of the institutes within OERI has gone largely unnoticed and uncontested by the educational research community and interested policy makers.

D. OFFICE OF REFORM ASSISTANCE AND DISSEMINATION (ORAD)

Throughout the long history of NIE and OERI, legislators and educators frequently criticized each agency for failing to disseminate research information to classroom teachers. These critics have often exaggerated the availability of reliable and useful research information about educational practices and have seriously under-estimated NIE's and OERI's efforts to distribute that knowledge. Indeed, one might argue that NIE and OERI have tried harder and perhaps have been even more successful than many other Federal research agencies in disseminating research information to the public and practitioners.147

Prior to the reorganization of OERI in 1994, OERI had been active in improving program development and disseminating the results through its Programs for the Improvement of Practice (PIP). The reauthorization of OERI created the Office of Reform Assistance and Dissemination (ORAD) which took over most of the functions of PIP. ORAD has four divisions: (1) State and Local Support Division (SLSD), (2) Knowledge Applications Division (KAD), (3) Development and Demonstration Programs Division (DDPD), and Learning Technologies Division (LTD).

Despite the repeated emphasis by policy makers and educators on the importance for ORAD to provide links between research and practice, staffing problems have hampered its operation. Whereas its predecessor, Programs for the Improvement of Practice (PIP), had 73 employees in 1990, ORAD in March 1997 had only 61 staff members (a one-sixth decrease). Thus, while the Congress and the Clinton administration envisioned an expanded and improved level of dissemination and reform assistance through ORAD, the restructured operation had substantially fewer employ

ton, DC: U.S. Department of Education, 1997); National Institute on Early Childhood Development and Education, Directory of Projects, 1997 (Washington, DC: U.S. Department of Education, 1997).

144 4Naturally, there are a few notable exceptions. For example, Clifford Adelman, a member of the Postsecondary Institute, has been a major and innovative scholar using college transcripts. Clifford Adelman, A College Course Map: Taxonomy and Transcript Data (Washington, DC: U.S. Government Printing office, 1990); Clifford Adelman, The New College Course Map and Transcript Files: Changes in Course-Taking and Achievement, 1972-1993 (Washington, DC: U.S. Department of Education, 1995).

145 U.S. Congress, House, Hearings. . . Appropriations for 1997, Part 5: Department of Education, pp. 309-12.

146 U.S. Congress, House, Committee on Appropriations, Hearings... Appropriations for 1998, Part 5: Department of Education, p. 1485.

147 Atkinson and Jackson, eds., Research and Education Reform, pp. 128–29.