 |
 |
 |
 |
 |
||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||
ABSTRACT Subject benchmarking was part of an agglomeration of quality assurance measures that emerged in United Kingdom higher education during the 1990s in large part as a reaction to the precipitous transition from ‘elite' to ‘mass' higher education in the early years of the decade. Rapid growth of student numbers was accompanied by expansion in the number of subjects and areas of study offered as degree programmes. This aroused political attention, and national government, as the main provider of funds, required the higher education sector to take steps to ensure that all its degree programmes were fit for the purpose. Over 50 subject-benchmarking committees issued reports between 1998 and 2001 setting out in some detail what degree programmes in the specialist subjects might be expected to cover. However, despite the initial intentions the benchmark reports were never used for hard regulatory purposes and instead have become developmental tools, particularly in new subject areas trying to attain academic respectability. The whole exercise has demonstrated the enduring collegiality of the academic community in British higher education and the enduring belief that in some real sense degrees from all higher education institutions have something meaningful in common.
BACKGROUNDBefore 1989 the United Kingdom higher education system consisted of two sectors, an autonomous university sector and a public sector under the control of local education (county) authorities. The universities, though publicly funded up to about three-quarters of their income, had almost complete financial and academic independence. Public funds were unconditional provided they were spent in accordance with the universities' charters, which were couched in very broad terms. There were only two external brakes on their freedom to teach their students what they wanted, how they wanted. One was the external examiner system, whereby all award bearing programmes of bachelors degree level and above had at least one examiner from another UK university or, very occasionally, a university considered to be of equivalent standard in another country. The other was the professional and statutory bodies (PSB) such as those for medical doctors, Law and various branches of Engineering. Their interest was based on the fact that recognised university qualifications gave certain exemptions to candidates for professional qualifications. However, the real guarantor of quality and standards was the fact that it was a meritocratic (see e.g. Young 1961), or elite, scholarly system in which most of the students and the staff who taught them were from the top levels of the ability range and universities were jealous of their reputations. ‘Public sector higher education' institutions were under tighter regulation. Their finances were controlled by the local education authorities which owned them, and their degree level teaching was regulated by the Council for National Academic Awards (CNAA) created in 1965, whose essential function was to ensure that the qualifications awarded by the polytechnics and other colleges were equivalent in standard to those of the universities. By the 1980s the polytechnics were chafing under what they considered a restrictive regime compared with the universities and the government was becoming dissatisfied with the control exercised by the local education authorities. The 1988 Education Reform Act gave the major public higher education institutions almost as much autonomy as the universities and the process was completed in the 1992 Higher and Further Education Act which abolished the CNAA and enabled all but a rump of smaller institutions to be transformed into autonomous universities. [1] At the same time as these reforms were occurring, the government stimulated, through a new formula funding methodology, an explosive expansion of student numbers. (see Bekhradnia 2003). Student numbers increased by 75 per cent between 1989 and 1994 and UK higher education was transformed from an elite to a mass system within half a decade. All universities, including the new ones, were allowed to start new degree courses with no need for authorisation from outside the university. A huge number of new degree courses were established in response to student demand. Public funding grew much less rapidly than student numbers following the introduction of student number based formula funding, which encouraged universities and colleges to expand student numbers at marginal costs. (see Williams, 2004) Huge increases in university and college admissions and declining income per student from public funds led to growing concerns about both the academic potential of some of the students embarking on higher education programmes and on the capacity of the institutions to provide them with teaching of satisfactory quality. Subject Benchmarking was one of the many quality related innovations in the national regulation of higher education that emerged in the 1990s in response to these concerns. It reflected a gradual realisation that the assurance and enhancement of the quality of learning and teaching in higher education was not solely an issue of institutional resources and management arrangements, nor of the processes of teaching and learning, though both have been the subject of considerable policy concern over the past two decades, but also of the content of academic programmes.
THE POLICY ISSUE: ‘GRADUATENESS'One debate of the mid 1990s was the relationship between ‘fitness for purpose' and ‘fitness of purpose'. The early work of the Academic Audit Unit (AAU) set up at the end of the 1980s by the academically elite pre-1992 universities was concerned primarily to ensure that the universities had teaching and examining procedures that underpinned their own criteria for the award of degrees. Were the procedures fit for the purpose for which they were designed? The underlying ideology was that the intentions and capabilities of these institutions were good but they may have not been careful about the administrative details. However, this approach was deemed to be inadequate by the government in the early 1990s and it established its own Higher Education Quality Council (HEQC), in part as a replacement for the CNAA which had been responsible for standards in the non-university sector. The immediate origins of the Graduate Standards Programme of the HEQC lay in the concerns about degree standards first voiced publicly by the Secretary of State for Education and Science, John Patten, in April 1994. With the rapid expansion of the early 1990s, the transformation of a large number of previously publicly regulated polytechnics and other higher education institutions into autonomous universities and the establishment of a very much wider range of degree course subjects, concern began to shift from ensuring that universities were meeting their own quality criteria (fitness for purpose) towards a concern that the degree programmes being offered throughout the system were appropriate for bachelors and higher degree programmes (fitness of purpose). Patten asked the Higher Education Quality Council (HEQC) to investigate the broad comparability in the standards of degrees offered by different institutions. In response, the Council embarked on a two-year programme to investigate the issue of ‘graduateness' - what attributes someone who had qualified as ‘a graduate' might be expected to possess. The Council's own rationale for this programme included the following: UK higher education has vastly increased in size and heterogeneity over the last generation and especially during the 1990. The informal mechanisms that were believed to ensure comparability in a small, homogeneous system seem increasingly unlikely to be effective in the present, greatly diversified one. What is more, the rapid increase in the scale and cost of higher education continues to fuel demands for its activities to be more transparent and more publicly accountable; the large-scale introduction of modular programmes has necessitated greater explicitness of purpose and has focused attention on key issues relating to assessment, comparability and related matters that were less visible before; many new subjects have entered HE, or been developed within it, in which degree qualifications had not previously been awarded: these are now confronted with defining their own understandings of graduateness; the diversity of types of programme of study now available has increased. This enables students to attain a degree by many different kinds of learning experience but raises the issue of how to establish the comparability of outcomes; the growth of collaborative work of various kinds (including franchising [sometimes overseas], the validation of the awards of one institution by another or partnerships between higher education and further education institutions) reinforces the need for clarity about the concept of 'graduateness' that is being shared; growth in the number of students at a time of declining per capita resources makes it necessary to establish clearly and publicly what is represented by a degree; increasing diversity in the qualifications of those entering HE programmes, a greater variety of modes of study (including innovations such as work-based learning or the accreditation of prior learning) tend to make insufficient the conventional assumption that a degree represents the successful completion of three or four years of full-time study following the award of A-levels; the growing internationalisation of higher education has made it more important to clarify the standards of UK degrees in relation to those in other countries. (HEQC, 1995, p2) The ‘Graduateness study set out to determine:
There were two main lines of development work: in the first, a pilot project on benchmarking assessment practice, the Council worked with five subject communities [2] in order to establish the feasibility of defining threshold standards. The second stage of HEQC's work encompassed fourteen subject communities [3] in a bid to examine the feasibility of using the concept of ‘graduateness' – the attributes that a person graduating with a degree might be expected to possess. This project developed a profile of graduate qualities on to which subject groups could map their disciplines and identify the qualities of their graduates (Wisby 2002) The outcome of the programme was inconclusive and it became clear that apart from some generalisations about ‘communication skills' and ‘critical thinking' the concept of a graduate, at least in the UK context was very subject specific. Further development work by the HEQC was overtaken in 1997 by the amalgamation of the AAU and the HEQC to form the Quality Assurance Agency (QAA) and the establishment of an official National Committee of Enquiry into Higher Education - the Dearing Committee. The development of subject-based benchmark standards was a key recommendation of the Dearing report (Dearing, 1997). The Committee recognised that the massive expansion of the early 1990s had led to much greater diversity in higher education provision but considered that ‘the task facing higher education is to reconcile that desirable diversity with achievement of reasonable consistency in standards of awards.' (para 10.3) As one of a number of recommendations covering ‘a national framework of qualifications', ‘standards of awards' and ‘quality assurance of the students' learning experiences', recommendation 21 proposed that institutions of higher education should develop, for each programme they offer, a programme specification which …. ‘gives the intended outcomes of the programme in terms of
The report recommended the establishment of small, expert teams to provide benchmark information on standards, in particular threshold standards, operating within a framework of qualifications that was also proposed by the Committee. The government accepted this proposal and the task of formulating these benchmark standards was assigned to the newly established Quality Assurance Agency [4].
THE POLICY INSTRUMENT: SUBJECT BENCHMARKSPilot benchmarking studiesThe purposes of subject benchmarking were to assist higher education institutions in planning programmes of study, to provide baseline information for quality assessors working for the QAA and to inform potential students, professional bodies and employers about the knowledge and competences that can be assumed to be possessed by individuals with particular specialist first degree qualifications. Pilot subject benchmarking groups for Chemistry, History and Law were appointed before the end of 1997 shortly after the publication of the Dearing report and the three pilot subject benchmark reports appeared in 1998. These three subjects were chosen because they represented different academic traditions: in the words of the QAA in its report on the pilot study ‘each statement is different, reflecting the differing traditions and cultures of individual academic disciplines' (QAA 1998b p1). They represented three different degrees of involvement with employment interests. Each pilot group was given open terms of reference to consider in setting out benchmarks for first degrees in the subject. In developing their policy on Subject Benchmarking the QAA claimed that it sought to work not only with subject providers but also with appropriate employment interests and relevant professional and statutory bodies. However, the membership of each of the groups consisted entirely of practising members of academic staff of UK higher education institutions with a slight exception in Chemistry where one member was a representative of the Royal Society of Chemistry, the pre-eminent professional body in the subject. The chair of each group was a prominent member of the relevant higher education subject association and other members were selected by the Quality Assurance Agency after consultation with the Chair and with subject associations and other bodies with an academic interest in the subject. There were no representatives from universities or academic associations in other countries and no representatives of employers or government agencies. The History panel had 16 members, Chemistry 14 and Law 13. The QAA, in its introduction to each of the pilot reports claims that they represent the first attempt to make explicit the general academic characteristics and standards of an honours degree in this subject area in the UK . Certainly they provide the first formal documented evidence (as opposed to informal evidence in novels, autobiographies etc.) of the very different academic experiences provided for bachelor degree students in different subjects that result from the extreme specialization of British first degree courses. In the UK with its strong tradition of subject specialisation, Chemistry is a discipline in which the university curriculum is more or less linearly sequential with what students have learned in secondary school. It is a long established university subject and there is fairly wide agreement amongst its professionals about what a new graduate should know. The assumptions of those who prepared the benchmark report are clearly that many graduates in Chemistry will have entered higher education with a good knowledge of the subject and are likely to use the specialist knowledge they have acquired after graduation. This contrasts with the claims made by the pilot benchmarking group in History, even though this too is a subject that is taught as a specialism in secondary schools. However, the History group was very concerned that the subject benchmark should not ossify the teaching of history in higher education. It aimed to lay out criteria for judging the suitability and adequacy of single-honours degree courses in History; to do this in a way that is as specific as possible without undermining the principle that there are many different suitable and adequate ways of constructing and making available the great richness and diversity of History; to do it in a way that recognises also the need for adaptability to new academic developments in the field, and innovations in course structures and teaching methods. We insist that teaching and learning are evolving processes and that it not our intention to freeze the teaching of History in a particular model. Our benchmarking statement should be seen as a starting point ………… We accept variation in how the vast body of knowledge which constitutes the subject is tackled at undergraduate degree level.' (History Benchmark statement para 4) Law is a subject that most students study for the first time at university, and indeed they often graduate in other subjects before they begin to take specialist Law courses. This benchmarking group did not make a statement of the aims of a Law degree: it appears to have been taken for granted that the purpose of a Law degree is to acquire at least some of the knowledge that enables the graduate to proceed to specialist training to prepare them for practice in some part of the legal profession. Their report concentrates on setting out the ‘minimum achievement which a student should demonstrate before s/he is awarded an honours degree in Law'. The long recognised differences between studying for a degree in the physical sciences and in the humanities and social sciences comes across clearly in these pilot subject benchmark reports. Both Law and History made strong recommendations concerned with ‘analysis, synthesis, critical judgement and evaluation'. They refer to ‘autonomy' and ‘ability to learn' in some form or other. Law, for example: considered that: A student should demonstrate a basic ability
The historians consider that their graduates should have ‘basic critical skills: a recognition that statements are not all of equal validity, that there are ways of testing them….' and ‘intellectual independence'. In contrast the chemists laid particular emphasis on familiarity of ‘chemistry related cognitive abilities and skills'. According to their benchmarking report the main aims of bachelors honours degree programmes in chemistry should be:
The pilot benchmarks and the processes by which they were arrived at were the subject of a rather cursory small scale evaluation by the QAA consisting of a review of documentary material, observation of the benchmarking meetings and interviews with about half the members of the pilot benchmarking groups (BMG). (QAA 1998b) The review dealt with the issue of membership of the BMG and concluded that the process of identifying members for the BMGs works well where, as in Chemistry and Law, there is an accepted 'lead body', a degree of commonality of provision across institutions and limited fragmentation or factions within a discipline. The QAA concluded that where, as in History this is not the case, or where there are competing bodies, they should establish a formal nominating committee drawn from different groups. In all cases higher education institutions and relevant subject departments should be consulted. However, the QAA concluded clearly that The criteria for selection of BMG members should be clear and might include: a range of experience of different forms of provision in the subject; representation from the range of higher education institutions; a balance of gender and age; an appropriate spread of knowledge of the main elements of the subject. Experience of external examining, accreditation or other QA processes is also useful. (QAA 1998b para 8.14) The evaluation concluded that each benchmark report could be expected to take about a year to produce with each member of the (very part time) panel contributing up to three weeks work. In its substantive evaluation the QAA concluded that overall, there was support from all members of BMGs for the process of Benchmarking and broad satisfaction with the information produced, in several cases despite initial scepticism …….. The views expressed by BMG members suggest that the Benchmarking process and its outcomes represent an advance on current practice in the articulation and judgement of standards within subjects. The majority saw the Benchmarking information as providing a national framework or 'meta-level' guide to the subject and for the subject as well as for other interested parties, including students. The frameworks produced were seen as useful for a variety of purposes including design and validation of programmes, examination and review. (QAA 1998b para 6) However, a number of technical and practical issues arose. All the groups reported considerable difficulties in agreeing on threshold standards caused by: difficulties in identifying a single acceptable threshold of attainment within a classification system that has 5 thresholds (fail/pass, pass/third, third/lower second, lower/upper second, upper second/first); grading conventions and performance criteria that appear to identify an ideal performance level and relate other levels to this; the widespread use of norm rather than criterion-referenced assessment; differences in grade points, grading conventions, and classification criteria across the UK higher education system; a culture within subjects that regards a minimum threshold of attainment (at the pass/fail or pass/third boundary) as 'unsatisfactory', 'not worthy', 'negative in terms of attainment and public acceptability', 'not representative of the majority of students', 'certifying attendance rather than attainment'. Each group came to a different solution to this problem so that there is a lack of consistency across the documents. ((QAA 1998b para 8.1) There was also concern about whether the benchmarks were intended to represent what all successful graduates of the programme should be expected to have achieved or whether the concern was with what the course offered them. It was felt that this distinction, although in some senses pedantic does have implications for the work of institutions, external examiners and reviewers as well as the perceptions of stakeholders and possibly as evidence in legal claims against higher education institutions.
Subject benchmarking across the whole of higher educationDespite these reservations by members of the pilot BMG the three pilot reports were quickly followed by the establishment of another 19 subject benchmarking groups. This second group included not only well established university subjects such as Economics, Engineering and English, but also a number that were, in the UK at least, much more recently established as degree courses, such as ‘Hospitality, Leisure, Sport and Tourism'. The benchmarking groups still had between ten and twenty members drawn from senior teaching staff from across the whole higher education sector supplemented, where appropriate, by members of relevant professional bodies and very occasionally employing organisations. Although there was no mandatory template all the new reports included the following sub-headings: Defining principles; Nature and extent of the subject; Subject knowledge and understanding; Teaching, learning and assessment; Standards and levels of achievement – or something very near to them. There were however fairly wide variations in the way the headings were interpreted, not least in the length of the reports which varied from about 2,500 words to over 10,000. The Learning and Teaching Support Network evaluated the first 22 subject benchmark reports (Yorke 2000). In Yorke's opinion Whilst benchmarking can relate both to developmental work and to regulation, the subject benchmarking exercise sponsored by the QAA leans towards the latter. … … Regulation has come to the fore, with the intention being to use benchmarking to provide explicit standards against which institutions' performances can be measured. Jackson (1998, p.5) observed that, in the context of the assurance of quality and standards in the UK, benchmarking might more appropriately be defined as a learning process to facilitate the systematic comparison and evaluation of practice, process and performance to aid improvement and regulation. … … The benchmark statements are broad in character since they have to cater for variety in the approach to subject disciplines and, in some cases, transdisciplinary spread. As a result, their relationship with standards is loosely-coupled and open to interpretation. It is argued that attempts to achieve a high degree of precision in specification are likely to prove counter-productive. (Yorke, 2000 p1) By the end of 2004 another 25 subject benchmark reports for bachelors ‘honours' degrees had been published so by 2005 the courses taken by the great majority of first degree students were covered by a subject benchmark. This final group of benchmark reports is very similar in structure and approach to the previous block but it may be significant that the title of this final group in all cases is ‘Subject Benchmark Statement: Academic Standards: (name of subject)', whereas in the earlier group of subjects the title of the report was ‘Subject Benchmark Statement (name of subject)' and Standards was not always mentioned even as one of the sub-headings. The structure of the reports was not fundamentally different but it is of some interest that the benchmarking panels had been reminded that their primary purpose was to establish threshold ‘standards' for an honours degree . Several also indicated what some described as modal standards, i.e. levels of achievement that half the graduates could be expected to have met.
Overview of the benchmarksAll the benchmark reports open with a statement from the Quality Assurance Agency. Subject benchmark statements provide a means for the academic community to describe the nature and characteristics of programmes in a specific subject. They also represent general expectations about the standards for the award of qualifications at a given level and articulate the attributes and capabilities that those possessing such qualifications should be able to demonstrate. Subject benchmark statements are used for a variety of purposes. Primarily, they are an important external source of reference for higher education institutions when new programmes are being designed and developed in a subject area. They provide general guidance for articulating the learning outcomes associated with the programme but are not a specification of a detailed curriculum in the subject. Subject benchmark statements also provide support to institutions in pursuit of internal quality assurance. They enable the learning outcomes specified for a particular programme to be reviewed and evaluated against agreed general expectations about standards. However, it is in the final section of this paragraph that offer a glimpse of the intended regulatory teeth. Finally, subject benchmark statements are one of a number of external sources of information that are drawn upon for the purposes of academic review and for making judgements about threshold standards being met. However, the detailed benchmarks are drawn up by broad cross sections of the academic teaching profession and not surprisingly most of the specific proposals are often bland and couched in rather general terms. The teeth are filed down. In a subject known to the present author the Economics report starts with a general statement about the areas of knowledge covered by the subject. Economics is the study of the factors that influence income, wealth and well-being. From this it seeks to inform the design and implementation of economic policy. Its aim is to analyse and understand the allocation, distribution and utilisation of scarce resources. Economics is concerned both with how present allocations arise and how they may change in the future. Study of Economics requires us to understand how resources are used and how households and firms behave and interact. This understanding is required at both the individual (micro) and the aggregate (macro) level. The analysis is both static (dealing with output, employment, income, trade and finance) and dynamic (dealing with innovation, technical progress, economic growth and business cycles). The study of Economics requires an understanding of resources, agents, institutions and mechanisms. Moreover, since virtually no economy operates in isolation, it is important that these phenomena are studied in an international context . (Economics benchmarking report para 1.1) The report goes on to identify the study of the factors that characterise the economist's approach. First there is the ability to abstract and simplify in order to identify and model the essence of a problem. Second is the ability to analyse and reason - both deductively and inductively. Third is the ability to marshal evidence and to assimilate, structure, and analyse qualitative and quantitative data. Fourth is the ability to communicate concisely results to a wide audience, including those with no training in Economics. Fifth is the ability to think critically about the limits of one's analysis in a broader socio-economic context. Sixth is the ability to draw economic policy inferences and to recognise the potential constraints in their implementation. (ibid para 1.3) The report covers the normal content of the subject in most universities and colleges but makes little attempt to be prescriptive. The stressed passages in the extract below are mine. any single honours degree in Economics normall y comprises the following elements. A coherent core of economic principles. The understanding of these might be verbal, graphical or mathematical . These principles should cover the microeconomic issues of decision and choice, the production and exchange of goods, the interdependency of markets, and economic welfare. They should also include macroeconomic issues, such as employment, national income, the balance of payments and the distribution of income, inflation, growth and business cycles, money and finance. The understanding should extend to economic policy at both the microeconomic and macroeconomic levels. In all these, students should show an understanding of analytical methods and model-based argument and should appreciate the existence of different methodological approaches. (ibid para 3.1) There are let-out clauses for individual institutions and for courses that may include some economics. It is recognised that, in both single honours degrees and in many degrees that involve a substantial amount of Economics, content will be adapted to suit the nature and objectives of the degree programme ……. In degrees that are not single honours Economics, not all the core elements … may be covered. It is also recognised that the forms of analysis chosen may differ and may be tailored to best serve the skills that students bring with them into their degree programme. It is neither the function nor the objective of this benchmarking document to prescribe what these forms of analysis might be; this is a matter for institutional choice and decision .(ibid para 3.2) The highlighted passages illustrate how, even in a fairly well defined subject such as Economics the subject benchmarking report allows course teams very wide latitude in deciding the actual content of their courses. It is clear that one implicit aims of the benchmark report is to ‘sell' the subject to employers and to future students and their advisers Along with all the other subject reports the Economics benchmarking statement is at pains to point out that: Some of the attributes that a graduate in Economics possesses are generic and not specific to the study of the subject. Their enhancement would be part of any degree programme. These would include general intellectual skills such as literary and information-processing skills, as well as interpersonal skills, such as communication. Economics degree programmes, therefore, provide a learning environment that facilitates and encourages the development and use of such skills.(ibid para 4.1)
Some brief comparisons with other subjects highlight variations in content and treatment. The Philosophy panel claims that its graduates will be expected to have acquired intellectual abilities which are readily transferable to other contexts including (inter alia) articulacy in identifying underlying issues in all kinds of debate, precision of thought and expression in the analysis and formulation of complex and controversial problems, clarity and rigour in the critical assessment of arguments, ability to abstract, analyse and construct sound arguments and to identify logical fallacies, ability to recognise methodological errors, rhetorical devices, unexamined conventional wisdom, unnoticed assumptions, vagueness and superficiality. Sociology graduates will have been ‘enabled to develop competence in' judging and evaluating evidence, appreciating the complexity and diversity of social situations, assessing the merits of competing theories and explanations, gathering, retrieving, and synthesising information, making reasoned arguments, interpreting evidence and texts, developing the ability to reflect on their own accumulation of knowledge. Apart from a wide range of subject specific skills the Music panel lists ‘a wide range of transferable skills,… many of which are applicable to issues of musical and non-musical origin'. There are ten ‘intellectual skills, nine ‘communication and interaction skills', eleven ‘skills of personal management' and four involving ‘enhanced powers of imagination/creativity'. Physics is more succinct and more self-confident about the transferable skills. A physics graduate may be expected to have acquired : problem-solving skills, investigative skills, communication skills, (Physics and the mathematics used in physics deal with surprising ideas and difficult concepts; good communication is essential.), analytical skills, IT skills and personal skills (ability to work independently, to use their initiative, to organise themselves to meet deadlines, and to interact constructively with other people). A simple word count of the 47 reports overall (see Appendix Table 1) makes it clear that ‘knowledge', ‘understanding' and ‘skills' are thought to be the most pervasive aims of most UK first degree programmes. There are, however, some interesting variations between subjects. ‘Knowledge' is mentioned ten times as frequently in Accountancy and Biomedical Studies as it is in ‘Religious education'. ‘Skills' are particularly likely to be mentioned in the Economics, Business and Management, Architecture, Accountancy, Dance and Drama, Music and Health Studies reports, but considerably less likely to be mentioned in Archaeology, Educational Studies and Religious Studies. A second batch of key words are critical and analysis, which appear about half as often as knowledge, understanding and skills. ‘Critical' is used at least twice as often in English as in most other subjects. ‘Analysis' is particularly favoured in Economics, Health Studies and Linguistics where it is used twice as often as in other subjects, but ‘analysis' is relatively rare in Librarianship, Medicine, Dentistry and Veterinary Science. Dance and Drama, Psychology, and Health Studies appear particularly likely to favour students doing independent work while Computing, Economics, Educational Studies, Materials and Town and Country Planning do not mention this at all. Employers or employment are mentioned rarely as are words that Yorke thinks ought to be included such as ‘original', ‘creative/creativity' and ‘synthesise'. As might be expected, creative or creativity appears frequently in Art, Music and Communication Studies but not at all in twelve other subjects, including Accountancy. Students of Education and those in Health Studies are particularly likely to be expected to ‘reflect' on what they are learning while the word is not mentioned by the accountants, the economists, the engineers, the physicists and six other subject areas. One of the entrenched features of first degrees in all UK universities is that degrees are classified – first class, upper second class, lower second class and third class are the usual categories used. In recent years upper second-class honours has become the modal category and third class the threshold. In its pilot study the History Panel confronted this issue directly and gave some indications of what it considered appropriate for first, second and third class honours degrees. Most of the reports in the main body were content with a distinction between minimum or ‘threshold' standards and the standards that the ‘modal' or ‘typical' or ‘focal' graduate may be expected to achieve. The Economics panel distinguishes between ‘modal' and ‘threshold' attainment. The threshold level. A graduate in Economics who has attained the threshold level should:
For the ‘modal' level of achievement ‘ understanding ' replaces ‘ knowledge ' and ‘ proficiency ' replaces ‘ awareness ' and the modal graduate will ‘know how to apply economic reasoning to policy issues in a critical manner. 'Knowledge' and 'understanding' are given the following meanings. Knowledge is the ability to reproduce theory and evidence as taught, understanding is a term applied to constructive and critical use and analysis of that material. (ibid page 5 footnote 4) In the most rigorous analysis of the subject benchmarks so far available Wisby (2002) made a detailed observation and interview based case study of the Sociology subject benchmarking group, and conducted in depth interviews with the chairs of twelve other benchmarking groups, selected to be indicative of the type of subject: humanities, social sciences and physical sciences and of the ‘market position' of subjects in terms of the pressure of applications to study in those areas and the perceived employability of their graduates. Arising out of these interviews she distinguishes a difference in attitudes toward benchmarking by three broad categories of subject: ‘established' subject areas that have a relatively long history in higher education; ‘ accredited' subjects, those disciplines that are subject to accreditation by one or a number of professional and statutory bodies (PSBs) and ‘new' subject areas that have arrived only relatively recently in the UK higher education sector and usually have a strong vocational emphasis. This last group tends ‘to be located predominantly in the post-1992 university sector', which may be significant because this sector has a much longer experience of external involvement in the content of its teaching. Wisby found that the established subject providers could be ambivalent towards the external regulation of provision, and that they were also likely to be attached to a ‘liberal' model of teaching and learning. These subject areas and their benchmarking groups tended to be suspicious of the benchmarking exercise. In the case of the accredited subjects the PSB play an important role in the current regulatory regime in protecting standards in professional and vocational education. Their remit typically incorporates all aspects of teaching provision – from entry standards and curriculum content to methods of teaching, learning and assessment, as well as resources to support learning. Notably, this activity often resembles the frameworks now being established through standards-based quality assurance – not least the provision of threshold standards. These disciplines, then, are obviously used to the external monitoring of provision and to working to external requirements. (Wisby 2002, p139) The most interesting category for Wisby were the ‘new' subject areas. They appeared to have more to gain from the benchmarking exercise than the others. For them the main priorities in the benchmarking exercise were using the benchmark statements to establish subjects as valid areas of study; to demarcate subject areas; and to address poorer provision at the margins in order to protect subjects' reputation or academic standing. This entailed a more strategic use of the benchmarking exercise . (ibid, p 140) One of Wisby's respondents from a new subject area claimed that benchmarking gave them an ‘opportunity to strengthen our subject' and ‘has ‘given us a little bit of credibility, it's given us some ammunition'. (ibid p 159)
IMPACT OF THE BENCHMARKS Subject benchmarks cannot be viewed in isolation. They need to be seen in the context of introspection about the aims and purposes of higher education and a significant cultural shift, inspired in large part by concerns about the implications of the sudden chaotic shift to mass higher education in the early 1990s. They are part of an interlocking network of quality assurance and enhancement measures imposed on United Kingdom higher education since 1990. A related contextual factor is pressures on academic departments to concentrate effort on research at the expense of undergraduate teaching in order to take advantage of the substantial financial rewards available to those university departments that are successful in the Research Assessment Exercises. A detailed blow-by-blow analysis of the ‘quality wars' of the 1990s is provided in Brown (2004). Among other developments is a national qualifications framework, recommended by the 1997 Dearing Committee, which attempts to ensure for the first time that all UK higher education institutions have similar structures of sub-degree programmes, first degree, taught postgraduate and research degree programmes. There are ‘institutional audits' of all universities and other degree awarding institutions, which monitor and assess the quality of the learning programmes and the standards of the awards in teaching departments and audit the institutions' ultimate responsibility for what is done in their names and through the exercise of their formal powers. There are specific subject based ‘academic reviews' in all higher education courses provided in the lower level further education institutions every six years. Reviewers test, by means of their own observations and analyses of the evidence provided by the college, the statements made in a self-evaluation. There is a ‘ Higher Education Academy ', which is concerned with improving teaching procedures at individual and departmental levels with a membership of individuals who have attended a recognised course on teaching in higher education. The original expectation was that the subject benchmarks would be an important input into Discipline Audit Trails (DAT) which were intended to be a significant component of the new round of institutional quality assurance reviews established by the QAA in 1999. Programmes of study were expected to be consistent with the benchmarks or the university would be required to provide a convincing explanation of why alternative content or approaches to teaching the subject were being adopted. In practice this intention was not achieved for two main reasons. One was that as a result of pressure from the higher education institutions, and in particular the powerful research led universities, but also because of changes in the leadership of the QAA, the external quality assurance reviews were modified and a so-called ‘light touch' procedure adopted whereby established universities needed merely to demonstrate that their own quality assurance procedures were effective: the DATs were effectively discontinued for these institutions. The other was that the benchmarking groups framed the subject benchmarks in ways that allowed course teams considerable leeway in interpreting them. There were also issues of the personalities and changing aims of senior politicians and the heads of the various agencies responsible for quality assurance in higher education. Such obstacles to the achievement of hard line regulatory interpretations of the subject benchmarks illustrate the point made by Adam Smith over two centuries ago: ‘All that such superiors can, however, force him (the university teacher) to do is to attend upon his pupils a certain number of hours, that is, to give a certain number of lectures in the week or in the year. What these lectures shall be must still depend on the diligence of the teacher … .' Rae (1895, p 248) Wisby, (op cit) makes frequent mention of the ‘developmental ‘offshoots'' of what had originally been conceived of as a regulatory exercise. These benefits were understood in terms of encouraging reflection on teaching practice, and generating discussion about teaching and learning issues – whether at the level of the individual academic, the department, or across institutions. The situation has changed somewhat since the 1770s. The individual university teacher is not quite so autonomous as he was then. But it remains the case that it is the academic profession that must, in the last analysis decide what is taught in universities, no other group has the knowledge or expertise and the subject benchmarking episode in British higher education makes this point quite clearly. One finding to emerge from Wisby's (op cit) fieldwork, ‘is the enduring collegialism within subject communities and the way in which this provides some protection from unwelcome external pressures'. She also notes a remark from one of the subject committee chairs that she interviewed ‘ The fact that the exercise to a large extent has been taken over by the academic community, means that academics themselves have, certainly in the Group that I chaired,…insisted that this should not be a...regulatory document, but that it should allow for academic freedom within it. The whole document was framed…in a way that really shifted the boundaries of the benchmarking exercise. When benchmarking started I think QAA had seen it as more prescriptive than in fact it has turned out to be . (op cit p 197) However, the subject benchmarks are having some impact on course content in many higher education institutions. In one college of London University the teaching of Geography was discontinued and the geographers transferred to a neighbouring university in order to meet the benchmark recommendations that a Geography degree programme should include both physical and human geography. More generally it is usual for new course proposals to be required by their institutions' course approvals committees to state what account they have taken of the relevant subject benchmarks. There are many cases of course documentations and assessment criteria being rewritten to take account of subject benchmark recommendations. Most departments appear to be aware of the need to ensure that either all their programme specifications are consistent with the relevant benchmark statement or statements, or to be able to provide a convincing explanation of why they do not. In future academic and institutional reviews by the Quality Assurance Agency, universities and colleges will be expected to show that they have conducted conduct regular internal quality assurance reviews of each programme, and that a programme specification has been published and used for each programme. These will not be required to follow subject benchmarks but it is likely that most will make some reference to them. In further education colleges, which are subject to more detailed appraisal by the QAA, there will need to explicit evidence of having taken account of the benchmarks. Subject benchmarks have not engendered serious hostility from within the academic profession, at least compared with many of the other activities of the Quality Assurance Agency. One reason for this is that the benchmarks have proved not to be external regulatory impositions on academic staff. They are seen as formative and developmental, representing the considered views of senior academics across the higher education sector about the aims and content of first-degree courses in each subject. Related to this is the crucial fact that unlike many of the quality assurance innovations of the past twenty years the benchmarks are subject based and, therefore are accepted as being at the heart of academic identity. Despite the modularisation and hybridisation of courses that accompanied and followed the expansion of the early 1990s most British academics still see themselves as being subject specialists. (Becher and Trowler, 2001; Henkel, 2000) and subject benchmarks are seen as having more direct relevance to their daily work than most of the other institutional management and process based evaluations of the Quality Assurance Agency. In the last analysis, however, developments since 2001 have, in effect enabled course teams in universities to disregard the subject benchmarks if they wish to do so. CONCLUDING COMMENTS In the UK each university is responsible for the quality of the degrees and diplomas it awards to students and it is a basic belief that any external system of quality evaluation should not undermine this responsibility. Nevertheless a fundamental change since 1990 is that it is now generally accepted that students and employer have a right to be assured that the university is fulfilling its responsibility to offer worthwhile qualifications. Worthwhile in this context is recognised as having two dimensions – fitness for purpose and fitness of purpose. The first allows each university to set the standards for its own degrees and leave it to the students and employment markets to differentiate between the programmes. The second involves some way of ensuring that all stakeholders, and especially students and employers of graduates, are not misled about the signals that possession of a degree sends out. At least until the late 1990s it was a firm belief in UK universities that all degrees should be at least broadly equivalent. A degree from one university should be equivalent to a degree in the same subject from another university. This has been most explicit in the postgraduate fellowships awarded by the research councils. The prime consideration in the award of a postgraduate fellowship has been the class and subject of the first degree: the institution where it was awarded was less explicitly considered. This requires that the content of a first degree programme and the levels of attainment expected are at least broadly comparable with those of other universities awarding the same qualification. Before the 1990s, when British higher education was by most international standards an ‘elite' system with a relatively small proportion of the population obtaining degrees, it was possible to maintain this belief through the informal system of external examiners which in broad terms ensured that standards of performance in all universities were comparable. Subject benchmarking can be seen as an attempt to maintain this comparability over a very much larger and more diversified system of higher education. However, any endeavour to create benchmarks to facilitate comparability, while at the same time not imposing rigid straitjackets, is beset by obvious problems. Wisby's distinction between ‘established', ‘accredited' and ‘new' subjects provides a useful analytical tool. The ‘established' subjects are, for the most part taught in established universities, and even where they are taught in less august institutions, the subject associations and other forms of informal authority influence the ways in which a subject is treated. Subject benchmarking for them was an interesting exercise in collectively exploring the nature of their subjects. None of the benchmarks attempts to impose specific content on individual degree programmes and within very broad limits universities can accept or ignore them according to the professional judgements of their own academic staff in the relevant areas. In most accredited subjects such as Law and Medicine, there is a distinction between acquisition of a degree and obtaining a licence to practise. Where the degree gives exemption from all or some of the courses needed to obtain professional qualifications it is the relevant professional body that validates the course and it is this professional validation that is the most important consideration for the course providers. This is made clear in the subject benchmark for Medicine: The benchmarks for medicine are but one of the external reference points for the undergraduate medical curriculum and must be considered together with the others, and in particular the recommendations of the Education Committee of the General Medical Council published in Tomorrow's Doctors 2002 . Benchmark Report for Medicine para 4) In such professional courses it is obviously important that graduates allowed to practise the profession are known in the market to have at least certain basic professional competences. This is the case in about half the subject benchmarks so far defined and in most of these, further professional qualifications are needed before the graduate is allowed to practice [6]. As already noted it is the ‘new' subjects where benchmarking has had most impact. It has made a useful contribution to the legitimation of subjects such as media studies and many branches of management, which began to appear to a significant extent in the British higher education curriculum only during the explosive expansion of the early 1990s. The committee chairs from these ‘new' areas interviewed by Wisby , ‘were preoccupied with the impact that further expansion might have on the status of their subject areas' In summary, the subject benchmarking process can be seen as an attempt to assure the higher education community itself and its stakeholders that after the rapid transition to mass higher education in the early 1990s there is still a real sense in which first degree graduates from one university are equivalent to those from any other university in the country. The influential Council for Industry and Higher Education has recently published a manual based on the Benchmarking reports summarising for employers the main attributes that they can expect graduates of various subjects to possess. (Kubler and Forbes, 2005). The intention of their report is to ‘help make explicit what has often been implicit' (p1). However, their study leaves implicit any opinions employers may have about whether the knowledge and skills of graduates from different institutions are equivalent. Benchmarking is also considered to be important in the burgeoning market for foreign students, which also grew at an extremely rapid rate in the late 20 th and early 21 st century. Universities are using them as authoritative statements of what knowledge and skills students can expect to acquire when studying for a first degree in particular subjects. Chandler , a member of the benchmarking Group for Sociology has noted that: ‘…benchmarks have implications for promotional literature. They may also assist graduate students to articulate skills…and promote themselves in the graduate market place. Hence there is an element of competition in the way in which subject groups benchmark their areas as they enable prospective students and potential employers to compare the learning outcomes of different disciplines' (Chandler, 2001, p. 56). The success of the subject benchmarking panels in persuading senior representatives from across the whole higher education sector to subscribe to common statements of the aims and broad content of nearly 50 subjects in higher education provides some evidence that there is still a sense of common purpose and standards in UK universities in a wide range of subjects, which students can use when choosing universities and employers can use when recruiting graduates. Whether the employment market for graduates or the student recruitment market will permanently accept this conclusion remains to be seen. Although there is much discussion of diversity of contemporary mass higher education in the UK there is very little formal consideration of differences between degrees from different institutions. This is left implicit. The benchmarks also, unsurprisingly, make no attempt to resolve the challenge of the equivalence of first degrees in different subjects. Clearly subject content is different and the substantive meaning of the most widely used words, knowledge, skills and understanding vary widely between the subjects. Apart from this the generic skills graduates are expected to have acquired vary considerably between different disciplines and subjects. In the last analysis what higher education and its myriad of individual courses offer continues to depend on the professionalism and integrity of individual teachers and course teams. The market, or the various markets that impinge on UK higher education will be the final arbiter between individual graduates, their courses and their almae matres . In the last analysis subject benchmarking has been a formative and developmental exercise and not, as many academics feared when they were initiated, primarily a regulatory mechanism. Benchmarks are a useful addition to the information available to external stakeholders such as potential students and employers of graduates. They are also useful for course designers, particularly in ‘new' subject areas in reminding them of what the general view of their peers is about what such a course should set out to do. Metaphorically a subject benchmark is a combination of menu, basic recipe book and public health manual. These can help to avoid disappointments, particularly important in an increasingly litigious society, but it is what the skills of the chef does with the basic recipe that makes the difference between whether the establishment can charge $10 or $100 for a dish. In a market driven higher education system, the national and international university and course rankings, which are appearing in the British media with growing frequency and detail will probably be more influential in the long run than standardised subject benchmarks. [7] RESOURCES General links
Examples of Honours degree benchmark statements Benchmark statements
Evaluation of the benchmarking process in Law, Chemistry and History
REFERENCES Becher, T. & Trowler, P. (2001) Academic Tribes and Territories: Intellectual Enquiry and the Cultures of Disciplines Open University Press, Buckingham UK Bekhradnia, B. (2003) 20 Years of Higher Education Policy: Looking back 10 years and Forward to the Next Decade (Higher Education Policy Institute, Oxford ) Brown, R. (1998) The Post-Dearing Agenda for Quality and Standards in Higher Education London : Institute of Education Brown, R. (2004) Quality Assurance in Higher Education: The UK Experience sine 1992 London : Routledge Falmer Chandler , J. (2001) ‘Benchmarking the Sociology discipline' in Harrison, E. & Mears, R. (eds.) Assessing Sociologists in Higher Education Aldershot : Ashgate pp. 41-60 Chemistry benchmarking report – see QAA website below Dearing (1997) Higher Education in the Learning Society Report of the National committee of Enquiry into Higher Education (HMSO, London ) Economics benchmarking report – see QAA website below General Medical Council (2002) Tomorrow's Doctors GMC, London HEFCE (1996) Subject Overview Reports http://www.niss.ac.uk/education/hefce/pub96/ (accessed 08 /11 /98). Henkel, M. (2000) Academic Identities and Policy Change in Higher Education Jessica Kingsley Publishers, London HEQC (1995) Graduateness: What Are Graduates? Clarifying The Attributes Of 'Graduateness' Higher Education Quality Council Quality Enhancement Group, London , (1995) available at http://www.city.londonmet.ac.uk/deliberations/graduates/starter.html History benchmarking report – see QAA website below Jackson N. (1998) Pilot benchmarking study of assessment practice in seven Engineering departments. In Jackson N, ed, Pilot studies in benchmarking assessment practice . Gloucester , Quality Assurance Agency for Higher Education Jackson , N. (1998b) Understanding standards-based quality assurance: Part I – Rationale and conceptual basis Quality Assurance in Education 6 (3) Jackson N. and D'Andrea V. (2000) Benchmarking the outcomes of learning In Jackson N. and Lund H., eds (2000) Benchmarking for higher education . Buckingham: SRHE and Open University Press Jary, D. (2002a) ‘Subject Benchmarking in the context of QAA processes' Paper presented at the C-SAP Annual Conference on Learning and Teaching – The new higher education? Teaching and learning in the ‘knowledge society' Aston University , Birmingham , 10-11th January Jary, D. (2002b) ‘Subject Benchmarking in the ‘adit society': Towards an evaluation of Subject Benchmarking' Paper presented at the BSA Annual Conference, University of Leicester, 25th-27th March Jary, D. (2002b) ‘Subject Benchmarking in the ‘adit society': Towards an evaluation of Subject Benchmarking' Paper presented at the BSA Annual Conference, University of Leicester, 25th-27th March Kubler, Bianca and Forbes, Peter Student Employability Profiles: A Guide for Employers (CIHE London March 2005) Law benchmarking report – see QAA website below QAA (website) Quality Assurance Agency Website http://www.qaa.ac.uk/academicinfrastructure/benchmark/honours/ QAA (1998a) Consultation: Developing the quality assurance and standards framework for UK higher education Higher Quality No. 3 March Gloucester: QAA pp. 14-26 QAA (1998b) Evaluation of the benchmarking process in Law, Chemistry and History Quality Assurance Agency, Cheltenham UK http://www.qaa.ac.uk/academicinfrastructure/benchmark/evaluation/overview.asp QAA (1998c) Developing a template for programme specification Higher Quality No. 3 March Gloucester: QAA pp. 14-17 QAA ( 1998f) The way ahead Higher Quality 4 (October) Gloucester : QAA pp. 1-2 Rae, J (1895) Life of Adam Smith Reprinted by Augustus M Kelley, New York 1965 Williams, G. (2004) ‘The Higher Education Market in the United Kingdom ' in Markets in Higher Education: Rhetoric or Reality? Ed: Teixeira,P; Jongbloed,B; Dill,D; Amaral,A ( Kluwer Academic Publishers, Dordrecht , Boston , London ) Wisby, E. (2002) Subject Benchmarking as a ‘new mode' of regulating teaching in higher education and its implications for academic autonomy – including a case study of the Sociology Benchmarking Group ( Sheffield University PhD) Yorke, M. (2001) Assessment issues arising from Subject Benchmarking Statements (Learning and Teaching Support Network, Bristol ) Young, M. (1961) The rise of the meritocracy (Penguin Books, London )
FOOTNOTES [1] This process came to fruition in 2004/5 when most of the remaining higher education institutions were transformed into teaching universities. [2] Art and Design, Biology, Business and Management, English, and Music and Drama. [3] Accountancy, Art and Design, Biological Sciences, Classics, Communication and Media Studies, Economics, English, European Studies, French, Geography, History, Hospitality Management, Law, and Philosophy. [4] In view of its central role in the development and use of subject benchmarks it is important to realise that the QAA is an independent body funded by subscriptions from UK higher education institutions, and through contracts with the main UK higher education funding bodies. It is owned by the higher education institutions through Universities UK (UUK) and the Standing College of Principals (SCOP), membership organisations whose members are heads of all the higher education institutions in the UK . However, legally and contractually it does have obligations to perform certain tasks on behalf of the government owned Higher Education Funding Councils. [5] An honours degree in UK higher education is essentially a basic first degree. There are many local variations and in a few cases ‘pass' degrees are still awarded. Most degrees are classified into first class, upper second class, lower second class, and third class. (This is currently under discussion and there are moves to substitute a Grade point average system). More than half of today's graduates obtain an upper second class degree so the modal threshold is somewhere in this range. [6] About 25 per cent of first degree graduates continue with further academic or professional study immediately after graduating. [7] However, in the Autumn of 2005, under pressure from the government a consultative document was circulated by UUK and SCOP (the bodies representing the heads of higher education institutions) making proposals for the classification of degrees and attempting to ensure that common principles are applied across the whole of higher education for the benefit of students choosing universities and employers recruiting graduates.
|
|
|
|
 |
|