Follow US:

Practice English Speaking&Listening with: Tim Oates - Assessing collaborative skills: OECD's interest in widening ideas of student performance

Difficulty: 0


a very fortunate to work damage

assessment it's like the CU p it's a non

non teaching department of cambridge

university about two-and-a-half thousand

people and some of you will will know

that it offers qualifications of

international repute around the world in

in over a hundred and seventy countries

so we have this this amazing privilege

of being our to get into schools do

observations work with people managing

education systems both administrators

and politicians and do a great deal of

international comparison we see a lot we

take theory to that observation in the

same way in which you heard Lucy did

yesterday and we look at the curriculum

issues which emerge around things like

collaborative skills I could talk about

many many things but I'm going to really

focus on collaborative skills in the

context of curriculum policy and in the

context of what is being said about the

balance of our curricula around the

world you will have heard phrases about

knowledge base curricula skills based

curriculum the importance of personal

skills the important importance of

focusing on personal qualities gonna

deal with all of that this is not a

presentation though for the next few

minutes on the detail of all of the

techniques for assessing collaborative

skills I'm gonna cover those off very

very quickly what I am gonna really

focus on is should we focus on

collaborative skills and how should we

locate them in the curriculum so that

all people get access to the benefits of

the development of an adequate level of

collaborative skills and they're

available to the individual to society

in the economy because I think it would

be easy to say look it's a key focus in

OECD it's a key focus and discourse

about curricula so just let's do it

let's assess it let's focus on its

development through that assessment and

then we'd forget that we would forego

some things we would forget that we

needed to question

we should do it and we would fail to

think about the range of ways in which

we can approach the development of

collaborative skills in education the

first the starting point is are they

important and and about halfway through

the presentation I will give you access

to some research which confirms

absolutely that they are so I'm not

questioning the the idea that these

collaborative skills should be developed

I'm going to explore what they are how

we can assess them how they should be

incorporated in curriculum how should we

assess them so I'm just going to talk

about that for a minute there are no

slides on this that we can do that we've

got loads of techniques we can set up

exciting activities in the classroom

based on project work which which ensure

all the children discuss their ideas

formulate solutions work one with

another we can observe that so that no

child is excluded from the opportunity

to develop collaborative skills most of

the techniques rely on observation

that's problematic and because it means

that teachers carry a big burden both in

terms of undertaking it and in being

consistent and with the growth of high

accountability around the world that

brings teachers into quite a conflicted

position and I'll explore that through

what we've done with science in this

country it's also important to think

about well what are collaborative skills

and what's the construct and do we want

to grade people on collaborative skills

do we want to put them on a scale so

that we can confirm absolutely that when

this child emerges from school they're

useless at collaboration and we're

completely content with that because

we've done all our measurements we

completed all our assessment grids and

we're sure that that's the

characteristics of that particular child

no it's not a trivial issue you measure

things for a purpose in education and

typically you measure them so that you

can do something about them to enhance

them or remediate the effects now we can

measure collaborative skills somewhat

indirectly through what we would loosely

term psychometrics you can measure

dispositions and characteristics of


which and don't seem to be directly

related to collaboration but nonetheless

predicted we can use those kind of tests

to sometimes it's quite responsible to

do that now bear all that in mind the

technology of how we assess

collaboration is actually not that

controversial the question is should we

do it and how should we do it and how

should we develop collaborative skills

and that's what I'll focus on okay so

today because of rapid economic and

social change schools have to prepare

students for jobs that are not yet being

created technologies which have not yet

been invented and problems we don't yet

know will arise and dress like her who I

know well and work with for many many

years decades in fact I don't agree with

that ok now I don't agree with it

because the words are ok and they're in

the right order and they convey a truth

about what's happening in society and in

our histories but it conveys the idea

that we haven't been doing that before

it conveys the idea that we will

constantly face extraordinary levels of

change which we've never experienced


well try the Industrial Revolution try

the Agricultural Revolution desert

plural that occurred in this country and

across Europe think about the history of

your own nations we have had this level

of change before think about the first

world war on the Second World War

massive change think about the success

of feminism as a political movement

transforming the position of women

across the globe ok it's not right in

conveying that it's something new that

we don't know how to handle it also

conveys the idea that it's a reinforces

other statements which say that you will

have many many many occupations and I

talked about this with Kate that's not

empirically true people are changing

jobs more frequently but they're not

changing occupations with much greater

frequency at all doctors are staying

doctors telephone technicians are


technicians and so on so what are we

preparing people for when we talk about

collaborative skills we've got to be

very clear about what that future is now

we did a review of 21st century skills

bad news is most of them on 21st century

most of them on skills look at the list

which typically emerges from these kind

of frameworks you know since when was

human collaboration a new skill I mean

it characterized forms of social

organization in Neolithic times we know

that from the trade patterns from the

archaeological evidence we know that

people can remember when the the Iceman

was sparked from the glacier in Austria

he was carrying 22 types of wood from

all the way around Europe in

collaboration is intrinsic to human

nature now I'm not saying we haven't

developed it optimally I'm not saying we

haven't assessed it with precision

because that can be improved but the

idea that their 21st century skills is

just a myth there can be 21st century

skill in which in the sense in which we

want to emphasize them so I look at that

how can we develop them how can we

understand them Anna ranker suto did

this very good paper 21st century skills

ancient yes they are ubiquitous yes they

permeate education enigmatic because

people fail to derive them with

precision and describe them with

precision it's a good report now the

problem is this that we've got

manufactured anxiety what that first

quote does is scare young people it

makes them frightened

it makes them worried about the future

not confident it makes it doesn't scare

them into engagement with education it

just gives them a general sense of

anxiety what we need to do is to signal

very clearly what the assets are that

they can gain from participation in

education it myth represents the history

of educational improvement because

actually we've done fantastic things to

improve education in the past now it's

not a new challenge

Finland improved its education system


during the 1950's and 1960's when did

Singapore improve its education system

there was almost nothing there in 1940

and now it's an extraordinary city-state

with no natural resources developing its

human capital through a fantastically

balanced and effective education system

and it's done that in an extremely short

time but we mustn't forget how they did

it history is important and the worrying

thing about that initial quote is it

says worry about the future

it doesn't say make sure we marshal all

of the assets from the quality of the

education systems which we have now it

miss reads the data on important

elements of education and that will be

the theme of my presentation

collaboration yes but don't suddenly

think that all education species

speeches should be about we are going to

improve education by injecting

collaborative skills into our curriculum

and focusing exclusively on them because

that's what a lot of speeches sound like

and that's inconsistent with the history

of the development of high quality

education and also I put this up through

policy analysis the desire to make

others responsible for complex

educational outcomes where does where

does collaboration come from in human

society if you make it entirely the

responsibility of individual teachers

they are carrying a very big burden

you've got to be very clear about where

you put responsibility for the

development of these kind of complex

human capitals school can't compensate

and carry it for everything else which

is going on and carries sole

responsibility for it okay what about

the plus side well labor market analysis

gives us very strong evidence on

priorities I don't listen to what

employers say they want I look at the

data on what they pay for okay because

there are differences between the types

of skills and knowledge which employers

attach a labor premium to now that

either indicates a shortage which sends

a signal back into schooling or

highlight the importance of something so

those are the data that we pay attention


a historical record gives us very clear

examples of success Singapore Finland

Alberta and Massachusetts and so on and

I think England now we've gone from

twelfth to eighth in reading in pearls

in a very short period of time since the

emphasis of reading on reading in 2010

and very very clear strategies

emphasizing certain approaches to

reading and just saying readings really

important for very young children

we've improved dramatically and kids are

reading more earlier and enjoying it so

we must look at the historical record

what has worked not some notion of

either futurologists the future is

uncertain be very frightened so listen

to me and what I think you need no no go

back to the historical record and say

what has worked and in what conditions

and that's the evidence I'll put up we

need balanced curriculum Theory informed

by evidence and he is emerging

I mean Estonia the National Curriculum

is astonishing ah new national

curriculum loads of press comment about

the problems of the assessment huge

discourse about qualifications the

National Curriculum has landed extremely

well and it's quite radical in many ways

radical in the true sense of radical

going back to roots really emphasizing

the core aspects of disciplines and

clarity for who is responsible for what

is improving around the world and in

countries like England the extent to

which the state should specify clearly a

series of constructs and then schools

should be responsible for designing

exciting broad and expansive programs of

learning to encourage the acquisition of

those things and I'll look at how they

should be stated and what what's the

purpose of all this why would you

encourage children to acquire

collaborative skills well anything that

you put into education costs money and

it forgot it you have forgone

opportunities if you focus on this you

won't be focusing on that so make sure

that when you decide to include

something on collaborative skills think

about what you're forgoing if you are so


and what benefit you're receiving from

it because what we need to do for

everything that we do in an education we

need to say what do we get from it what

do the children get from it the parents

and it can be neatly summarized and in a

very simple phrase that is nonetheless

very complex a balance of goods to the

economy society and the individual now

you can see instantly how certain

systems can get it wrong because they

can emphasize goods to the economy but

not to this not to society they could

emphasize you can you know about aspects

of Education which are good for the

child but also many not good for the

economy and so on so it's a balance of

these things in particular historical

times and settings what's what's going


well youth unemployment the level of

youth unemployed has reduced

dramatically since the 1970s across the

world that's really good education is

working duration in general education is

increased significantly from the 1960s

right their way across the globe we have

many more objectives that the large

transnational organizations like UN

actually outline as a necessity for us

in developing our education systems but

these are real real successes since the

1950s since the 1940s the still our

labor market shortages in occupations

with high cognitive demand that's

interesting collaborative skills but

actually many occupations it's it's high

cognitive demand where they're finding

it difficult to actually find people

think about all the immigration

discussion in respect of England the

debate we're having about brexit

suddenly we find out that a lot of the

immigrants are very highly qualified and

they're actually addressing labor

shortages interesting there are

continued regards that continuing

concerns regarding work readiness now

this is a load of old nonsense it gets

talked employers saying we have kids who

come from school and they're not work

ready whose job is it to make children

work ready

is it schools job

in Germany it's not the school's job to

make children work ready that's the job

of employers that's what the

apprenticeship system is for they expect

children to be well oriented towards

learning to have all of the outcomes

that we expect our general education

facility and language understanding a

scientific principle a high level of

literacy and understanding a

mathematical principle and then the

employers with the state take a

responsibility for the development of

the skills which make them work ready

and interestingly in terms of

collaboration collaboration is only

expressed very loosely in that framework

for the crew the curriculum the

vocational curriculum in Germany I mean

there's not a long list of standards

there's not a very very tight assessment

which goes on for hours there are no

grids actually putting people on a scale

saying you're great and you're terrible

at collaboration because they expect

collaborative skills to be gained

through immersion in real work processes

whilst being supported by learning

professionals in the context of an

apprenticeship they expect them to be

developed through some kind of

interesting osmosis learning

collaboration through being supported in

real settings and emerging at the end of

the apprenticeship program with skills

in collaboration and all the cognitive

outcomes of a vocational program to a

level which is worthwhile paying more

money for very interesting so all this

stuff about work readiness and the

importance of schools putting these

things into their curriculum to make

children work ready hmm I would say that

is a point which is well worth

discussing it is controversial I would

argue okay there are problems in

progression to a tree in numerous

subjects and those with high entry

demands goodness me we're just making

physics a five year course at Cambridge

it wasn't a five year course during the

70s because a levels were harder than Oh

people say yeah but very few people took

physics a-level no actually more people

took physics a level then about twice

the number

oh they say well they must have honey

had needed to get any very low marks to

get a high grade that's not what the

examiner reports in our archives say

they knew more physics and more people

knew more physics now this is a problem

because we we have put so many things

into our schools and given them so many

priorities the development of the really

tough stuff in disciplines which we

previously were doing in school

we now are doing in the first two years

of university who pays for that in many

systems families and individuals if that

was two of us in school I mean I know

that some systems in this room secondary

education is paid for but in many

systems like England it's not so if they

develop those skills and that knowledge

in school then the state is paying for

it that's quite a big issue okay so

where should we put work readiness

because we've got different forms of

vocational education and training

we've got school-based ve t we've got

initial ve T and a full time educational

setting employment base ve T like and in

Germany apprenticeship Focus VT for

comprehensive labor market preparation

many people to access a broad range of

jobs continuing VT for existing an adult

employed workers and VT for unemployed

adults now they all have very different

needs and the curriculum should be

different but many countries are trying

to implement single classes of

qualifications to meet all of those

needs VT for unemployed adults why are

they become unemployed very frequently

it's to do with the things which they

could have acquired at school but they

found it difficult to develop and they

haven't developed whilst they're in work

so we do need to support the development

of these broader competences in

schooling learning to learn orientation

to learn high-level critical thinking as

well as the COS of disciplines they're

all important interestingly the CBI

which is employer organization so that


basic literacy and use of English is a

problem for many people in the labour

market not a hint of collaborative

skills 21st century skills basic

literacy which is something we should

have cracked hundreds of years ago and

is still unevenly distributed in our

societies very difficult to be a high

order critical thinker if you ain't got

basic literacy if you don't understand

what if then counterfactual conditionals

which should be acquired children should

be exposed to by the age of two and

three and should be acquired

comprehensively by age of five and seven

undue emphasis on GCSE grades and

schooled league tables risk distracting

attention from the need to equip every

young person adequately with these

capabilities what that's what GCSE

English language is about it's about

basic literacy I mean and you have to be

motivated to get a high grade in a

qualification you have to be capable of

collaborating with others including a

teacher you have to be motivated so

qualifications are not just a measure of

the cognitive things which are listed

it's a measure of these other

capabilities as well if you get a high

grade and look almost all businesses

were satisfied with graduates IT skills

but added when it comes to numeracy

powers of analysis and use of English

skills of satisfaction start to fall no

yeah elsewhere employers are saying it's

all about 21st century skills here

they're saying it's about the cognitive

elements of disciplines so we've got to

pay a lot more attention to the evidence

sorry I wake everybody up a bit has

anybody seen Ken Robinson's you know the

one the cartoon one the RSA you know I

have it's all night something about 13

million views now it's dreadful it's

compelling it tells a great narrative

it's been it's had a ridiculous number

of views

and it uses this weird idea of the

factory model of Education designed to

feed the industrial economy well listen

to this that's inaccurate in terms of

the history of education because basic

literacy escalated in the Victorian era

okay an an accurate picture of that is

presented in Andy Greene's education and

state formation both the first in the

second edition because literacy levels

went up during the industrial revolution

despite the fact that the literacy needs

of factories was reducing because of the

form of industrial production so Ken

Robinson doesn't make sense he can't

explain all of the schools that were set

up by Victorian philanthropists who

gained their money through industrial

project production who were committed to

the children in those schools acquiring

levels of literacy far in excess of the

levels needed in their factories you

know take that Ken and rewrite the

script because you don't understand

why schools are where they came from how

they were formed and how that early

curriculum transmitted knowledge and

increased equality in society we do know

about effective education is a brilliant

study of preschool education by Kathy

silver and Pam Salmons and Brenda target

and inrun Suraj Blackford Blachford and

it's great and I commend it to you

brilliantly structured well funded

really empirical what they emphasize and

the thing I'm going to pull out of it is

the importance of a balance of emotional

social and cognitive development for

young people they couldn't find a

measurement instrument for the cognitive

development of children up to 5 they

could only find instruments for the

emotional development of children the

emotional and social development we can

measure all that stuff but there wasn't

anything for the cognitive but they had

to invent the Eckert's our measurement

to do that

and this balance between emotional

social and cognitive is so important you

see we do have categories for which we

have evidence cognitive resources and

they are concepts principles fundamental

operations like being auto right and

beyond to use at Bank the antem multiply

and divide and core knowledge those and

Bill Schmidt would agree those predict

outcomes for individuals and for

education systems how they're combined

as you go up through education and I'll

show you the evidence in a moment is

fundamental that does actually give us

access into what high level critical

thinking is very important and what

collaboration consists on and we need to

differentiate cognitive resources the

things people need to know and do the

way in which they combine them because

they describe levels of performance and

contexts and we frequently confuses I

find national curricular stuff full of

contexts because they want to make

National Curriculum relevant and

motivating what's motivating about

conservation of mass I mean it's just

wrong to say it's motivating or

demotivating everybody needs to

understand it what's motivating are the

contexts which teachers devise in

particular classrooms to enable it a

particular child or group of children to

understand the concept of conservation

of mass and chip and the teachers should

be in charge of that contextualization

in my view

I'll give you evidence without in a

moment it shouldn't be in the National

Curriculum National Curriculum shouldn't

be stocked full of motivating contexts

because then it forces a teacher to

teach those things that's what's got

Scotland's finding out to disastrous

effect they even decide contexts in the

National Curriculum and not the comp not

the cognitive resources they put at the

wrong way around and standards have been

going down dramatically and context

change they change all the time so you

don't need to change your national

curriculum since when did conservation

of mass guard of fashion since when is

gravity suddenly gonna have to come off

the national curriculum because it's


okay the fundamental things that should

be a natural curriculum it should be a

parsimonious listing of fundamental

elements with the contextualization left

to other levels of the education system

this is fascinating this is Kurt

Fisher's work and it shows that what you

do once you have learnt at a very early

age that the earth goes around the Sun

and not the other way around which was

previously the privilege of just two or

three people in 1640 and was considered

the absolute pinnacle of human knowledge

we now teach to three-year-olds that's

not in a level of hierarchy of

complexity of knowledge it's just an

idea and you can communicate it really

effectively really quickly along with

the plastic nature of glaciers the idea

of metaphor all the lovely things in

humanities which we know and love they

can all be communicated really easily

what differentiates the higher levels is

how you put them together to either big

bring new insights to you as a person or

entirely new principles for society the

top levels not a context insight it's

about stuff it's about cognitive

resources the contexts are about

application and how you learn and and

what coefficients stuff tells us is that

as you as you progress these building

blocks are put together in increasingly

complex ways to formulate new

understandings that's what's happening

at the level of the individual you

harness social learning and

collaboration to ensure that people are

comparing their representations

comparing their resources comparing how

well they're progressing in order to

acquire these things and learn how to

put them together you see all this stuff

I mean I think there's been a breakdown

of curriculum thinking to believe that

the signaling of important elements

should exclusively done by the content

and processes of assessment well that

means is we're interested in

collaboration so we must assess it to

death we must spend vast amounts of time

focusing on it and assessing it and

putting it on our assessment grids and

into our management systems

know there are many important things of

in education that we want to develop we

don't necessarily want to measure you

see there's been a breakdown of critical

thinking around the differences between

concept combination and context and

there's been a breakdown of the

curricular implementation have used

assessments as the only input to

pedagogy the wider processes of the

school and learning models you're

reducing the curriculum for the


so should we assess collaborative skills

I think marginally and I'll finish in a

few minutes on exactly that is

Singapore's fantastic progress England

making good progress with its new

national curriculum and and its emphasis

on reading and mathematics Estonia a

competent space curriculum hang on a

minute I read it I look at the aims and

it's full of collaboration application

context but then I read the statements

at 7 children have to memorize their

multiplication tables doesn't like

competence-based written to me not in

the kind of sense which we see around

the world emerging it sounds like to me

there are a whole bunch of really

important cognitive outcomes and

disciplines which are emphasized in the

lists of the content in the Estonian

national curriculum and then the aims

actually talk about how you combine them

in pedagogy and how teachers should use

the school curriculum the context which

they choose to develop those things in

fact the Estonian national curriculum is

as knowledge-based as the English

national curriculum which is

characterized as an exclusively

knowledge-based curriculum that upsets

the apple cart Finland going completely

the wrong direction okay so it's Finland

is now talking about phenomenon based

learning and I'll put and I'll talk

about that right at the end very old

stuff is it's not gonna stop their

decline in my view so England No I'm

going to talk about England and I'm not

saying you know this is what other

nations should do it's just to hold a

mirror up to your own system that's all

that's all I mean I could easily talk

about other systems States and so on or

or Kazakhstan we

I do a lot of work with but no I just

just talk a bit about England just just

to throw the issues into sharp relief we

have a huge dominance of assessment it's

led to a real weakness

in what we call key stage 3 11 to 14

education because the dominance of the

examinations at 16 and the dominance of

the assessment at the end of primary the

aims of the National Curriculum the new

National Curriculum are extraordinary

they're like Estonia they emphasize

application and rich pedagogic immersion

in processes and and contexts which

involve collaborative skills

unfortunately we remove the National

Curriculum as a requirement in some

schools and that class of schools

academies is increasing in number I

think that's a mistake I'm talking to

politicians about that we do have a

weakness in teacher training and that's

serious so when we talk about the

importance of the general goods of

Education like the importance for

children emerging with high levels of

collaborative skills teachers aren't

well prepared they do not know what the

priorities are and they often take the

signals exclusively from assessment so

that may make you sory if we want to

develop collaborative skills we better

assess it because then it will signal to

teachers that it's important and if

they've been poorly trained they come

into the profession with a low level of

expertise in designing a curriculum put

it in the assessment and they'll do it

that's true but dangerous

we got strengths of the models in

specific schools because we've opened up

school autonomy and we've opened up

school structures just as they did in

Sweden just as in heaven America both

countries Sweden America are suddenly

realizing they need a tighter national

curriculum a parsimonious listing of

important outcomes most states are

despite despite predictions have adopted

the common core standards you know in

some of those states hate Washington

yeah they really don't like federal

intervention in their state processes

but they've adopted the common core

standards this is very very interesting

historically Sweden a lot of discussion

in Sweden now on the fact that standards

were not well described in terms of the

national curriculum


so we've broken open the structure of

schooling and we've weakened the impact

of the National Curriculum on some

schools that could be problematic we've

had a very strong emphasis on early

literacy and boy has it worked it really

has worked and that the data from the

transnational surveys the improvement

correlates absolutely with the

intervention in terms of time we're

doing the same with mathematics we've

had love exchange work with Shanghai and

a lot of teacher observation work lesson

observation work are we imported a model

from Shanghai which is working in this

country through the National Center of

Excellence of teaching and mathematics I

advise very strongly about


but this transfer of expertise on

pedagogy has been undertaken through a

sophisticated policy learner and it's

working well we have new GCSEs and

a-levels and those the standards have

been elevated it's increased teacher and

pupil anxiety but they are now

internationally benchmarked and that's

good but we have a very weak vocational

route which we're trying to address

that's what we like in this country so

now let's just look at this okay we know

all of these things are important all of

them we've got evidence on all of them I

put I put some reference specific

references but there are long references

for all of them

discipline specific knowledge knowing

about the heliocentric movement of the

solar system that the earth goes around

the Sun in a broad range of disciplines

knowing what I Amba can't amateur is in

respective poetry be all too right

elaborated poetry and know where the

origins of that perfect form came from

orientation to learn weave Deacon quits

Crick's work that predicts outcomes very

important physical and mental mental

well-being we've emphasized that more in

our schools in the last two decades and

it's paid off our schools are full of

happier children

well oriented towards learning personal

and social identity very important

personal capitals John Boehner's word

basically if you are an externalizing

person things just happen to me I'm a

very unlucky person you're gonna have

really bad life outcomes we know that

through the

long toodle studies 958 1970 mm we

follow people right the way through

we've endlessly observed them it's

really bad for you and schooling can

change that it can empower you it can

give you a sense of agency that's really

important cultural capitals we know they

predict outcomes and certain households

with in in with lower income you have

poorer cultural capitals you know

Singapore fantastic in terms of social

mix they achieve in the schools by the

social mix that they enforce in the

local housing very interesting more

civic and political understanding very

important for social cohesion you know

these are all you can measure these all

off against what I've said in terms of

benefits to society the economy and

individuals if you emphasize all of

these you will have a balanced

curriculum but there's an awful lot and

there have a very different character

and I put facilitating technology

because that's important but it changes

we can measure all of those we can say

all of them with precision as constructs

which we need to do if we want to assess

them we need to see they're real like

externalizing behavior you need evidence

that they're real the EM we need to

state them as a constructs that they can

be the focus of the assessment that's

all standard stuff we can do it we can

measure all of these but where do we put

them there's an awful lot who should be

responsible for them no this is what the

OECD is saying is the draft paper and I

don't like it I'm involved in this

project so I can say this that's great

down there look I'll just it's not very

it's not very legible so I'm going to

blow up the three boxes that are on the

far left okay so there they are right

attitudes and values that's well stated

personal local societal global that's

nice they're discrete categories the

overlapping capabilities but it's nice

disciplinary knowledge right at the top

I can understand that interdisciplinary

knowledge how does that differ from

discipline or not

okay so interdiscipline orange is the

application in another disciplinary

field of disciplinary knowledge from

another field

oh do the same thing that's actually

about application not about content

epistemic epistemology is knowledge it

is the study of knowledge okay so maybe

it's knowledge about knowledge no

because that's Colin Colin that's

metacognition in the next box and

procedural knowledge is hang on knowing

how to actually manage if glass beaker

in the context of a titration is

disciplinary knowledge I don't get this

at all

so if you begin to articulate your

national curriculum using these

categories you're going to be in trouble

you're going to be assessing things over

and over and over that have been

expressed in a different form your

construct base will be confused an OECD

better get this right pretty quickly

because people are already using this

draft document to structure their

national curriculum and it's leading to

confusion both of the level of national

policy and on the ground so remember

these we've got evidence that they all

discriminate that they all predict

outcomes so where do we put them and a

couple of us here of an informing the

National Inspectorate here as to how

they can go into schools and look to see

where schools are putting these things

because if I want a child to understand

something about about democracy do I

teach them democracy on a Thursday

afternoon and assess them on it or do I

make the school a democratic institution

which they experience on a day to day


see where I put it it's important but

where I put it

collaboration is important where do I

put it drop it in the taught curriculum

in subjects or cross curriculum elements

or extra curriculum elements in the

taught curriculum because you can have

those two teachers doing stuff which

aren't part of the formal curriculum

because kids are particularly interested

in them expected activities outside the

school or extracurricular elements

guided or unguided you get the point I

can go through this endlessly you've got

these things that you think are

important you've got to decide where you

put them

some of them are in the it should be in

the ethos of the school if you wanted to

experience the sense of justice the

school has to be a just institution

otherwise you'll soon have trouble on

your hands the lived experience of the

institution is actually critical and I

call these some of the general goods of

schooling which some schools are really

very good at Shanghai schools are

brilliant at it they have lessons

focusing on the discipline content the

very life of the institution develops

collaborative skills develops these

wider competences and not a single

assessment of them in sight our aims

statements are like the Estonian ones we

say we have a very knowledge-based

curriculum this Estonian government says

we have a very competent space

curriculum they're virtually identical

and that's very very interesting

where does Estonia put all this stuff

about collaboration rich immersion in

broad and balanced experiences in the

aims of the National Curriculum not the

listing of outcomes and we do exactly

the same this is from England

mathematics is a creative and highly

interconnected discipline the ability to

reason mathematically an appreciation of

the beauty and power of mathematics

that's in our national curriculum that

is the law that's what schools need to

deliver it's a general good we now have

long listings of multiplying with

fractions quadratics at the right age

pre-algebra in primary course we do

that's in the listing of outcomes but

these are the aims fluent fluency in the

processes reasoning mathematically

solving problems it's in the aims not in

the listing of content interconnect

right let's repeat right and spoken

language orosi look at this this is

about collaboration we used to have

orosi as one theme in the subject

english-speaking this thing that was it

okay didn't appear throughout the

national curriculum

the Secretary of State said something

when I was revising the natural

curriculum fall and said speaking

listening in English ten recipe for an

idle chat in the classroom let's get rid

of it it was a deliberately provocative

statement I said no no no no and Robin

Alexander provided brilliant evidence no

policy should be in every subject in it

now is every subject in the national

curriculum science mathematics English

has this statement on spoken language

and if that's not collaboration I don't

know what it is and it's collaboration

of the most important thing exchanging

ideas and exchanging information about

what you can do this is not England this

is Estonia and is virtually

indistinguishable it includes stuff

about self-management and learning to

learn analyzing solution ideas and

testing accuracy justifying proof

understanding social cultural and

personal meaning of mathematics what's

interesting in this country people say

the national curriculum is really

prescriptive we don't tell people how

long to spend on teaching something they

do in Estonia that's quite interesting

and they accept this the National Creek

in school distinction in Estonia

National Curriculum parsimonious listing

school curriculum broaden balanced

really expansive because you have to

change the contexts in the particular

school to make sure that each child is

presented with something which in a way

that will enable them to understand it

the Confucian model every child capable

of learning anything depending on how

it's presented to them and the effort

they put into learning it again look now

this is SIL Estonia look at the

statements competence based what present

that a number is the sum of units tens

hundreds and thousands I mean come on we

we we are we are told you've got a

knowledge based curriculum it's not

competence based this is from Estonia

and that's the competence based

curriculum look how knowledge-based that

is find the numerical value of a letter

in equations by means of trying or on

the basis of analogy goodness me

you get the point and that's ours

represent a news number bonds and

related subtraction facts within 20

these are all vital outcomes the

cognitive outcomes of Education

reasoning mathematically now we're in

the whole area of critical thinking

kurt Fischer's combining those cognitive

components all in the context of a rich

oral community collaboration exchange

respect general goods of education being

delivered through the way in which the

conative elements are delivered Kuwait's

got problems because this is what

they've done they have actually these

are standards and they focused on

context I mean this is really unclear I

mean look at the statements and in

Scotland they emphasize the importance

of teachers giving rich immersive

experiences from which children will

draw the cognitive outcomes only they

don't and that's a real problem and

standards teachers are saying we don't

know what to teach we know how to design

these rich experiences but we don't know

what it is that we're trying to pull out

of them and it's very difficult to do so

Estonia England tightly our way around

be very clear about the constructs and

then leave the teachers to design rich

contexts it's their responsibility which

enable each child and every child to

access them this is the other way around

and it's leading to a real problem

textbooks I think by the way in C you

know see you please focus on materials

vital as mediating mechanisms here we go

you know this is the example and they're

trying to teach the examples in Kuwait

it's really hard and confused and

confusing think back to the OECD diagram

you've got to get your constructs right

Estonia has and the standards are

increasing Scotland has not and the

standards are going down this is a piece

writen and I've finished in just a

couple of minutes it's great isn't it I

mean it's you know re rich and or all

it's about problem solving and we know

from the diagram there's going to be

really complex but

look the sale is going off a quite a

difficult angle both in terms of the

linear direction of the vessel and where

the sail is it's really complicated so

now I've got to read all of that so

gotta get very high levels of literacy

it's about mathematics don't know why

there's such a high literacy demand it's

really complicated and oh when I get on

to it

it's about Pythagoras theorem it's not

complicated at all and what's the best

predictor of getting this right

knowing Pythagoras's theorem so you can

fool yourself you can say it's all about

complex problem solving but actually

it's about a fundamental bit of

mathematics which is actually relatively

low level though Smith's doing a lot to

correct that multiple choice questions

can be extremely complex this one is it

requires quite high levels of

problem-solving but boy do you have to

have cognitive resources to actually

manage and bring together to solve it

this one is wonderful this is from our

archive in Cambridge Lisa Jardine right

fellow here with Mark Warner leading

physicist at the Cavendish laboratories

Dave made these available on the


taken from our archive that's a great

physics question from when physics was

physics look it's from an examination in

1923 okay it's a great question and it

really encourages higher-order thinking

and you and we don't change the question

to make it easier we've given hints and

it's all available online and the way in

which the child is going through these

it can be observed outside online very

interesting kids love them it's gone to

150,000 hits a day at its peak not very

very good so I'll finish on these things

the interesting case of science

practicals we thought we would get more

practical work done and we get the

outcomes from practical work that we

want by taking the assessment of

practical work out of contributing to

the grade in the examination everybody

said that's appalling the level of

practical work will crash

what we knew is that teachers were under

pressure to maximize the kids marks from

the practicals we could see that in the

data because the the outcomes of the

practicals did not correlate with the

outcomes from the exam many more marks

on the practical in comparison to the

examination so teachers are under

ridiculous pressure to push up the marks

what were the practicals about

problem-solving collaboration no because

they had to do them as individuals you

wanted individual marks out of it so one

of the great assets of practical work

was not their collaboration discussion

exchange whilst you're doing it careful

that's dirty that will contaminate

things well you've got that measurement

wrong to the to two decimal places that

will be really problematic when you come

to do the next stage all of the learning

the rich learning that comes from

practical so we looked at the the the

pressures on the Assessors remember what

will happen if you try to assess

collaborative work teachers will need to

do it but as accountability increases

the conflicting professional pressures

will begin to distort the assessment

everybody said it will kill practical

work the latest evaluation shows it's

invigorated practical work because we

had to really screw it down to make it

really tie it down make it really boring

to make it a good assessment and

dependable and no matter how much we

screwed it down we were so leaving it to

teachers under terrible pressures so the

results were still drifting back up

again and it was breaking the but the

the relationship of trust between us and

examining boards we the new model is

that practical work is required it

doesn't contribute to grading we can

increase the complexity of the practical

tasks the teachers have to sign off that

an adequate number of experiments have

been done and the knowledge that you

gain from the practicals is assessed in

the exam so you know that you need to do

the practicals and it's been an

unmitigated success kids are learning

far more from failure in the practicals

that could never happen before they're

learning from success in the practicals

more practical work is being done so you

have to think very hard about whether

you want to assess these general goods

of education

because of the impact of assessing what

are the problems in assessment around

the world including and collaboration of

cognitive elements low validity a

failure to assess creed curriculum

content and if organizations like oacd

don't get the constructs right that'll

carry on happening poor measurement

characteristics of the assessment

instruments a lack of dependability in

the marking because of the complexity I

can tell you assessing collaborative

skills in context is complex attributing

collaboration to individuals rather than

groups complex we can do it but it takes

time and it takes time away from other

things poor face validity poor linkage

with learning operational problems and

cost these are the criteria that we

should be applying to good assessment

there in the Cambridge approach reliable

valid sound construct base use the the

outcomes of the use the uses of the

assessment the outcomes are are used

effectively and appropriately the impact

is good and it doesn't cost too much we

can use all these different approaches

objective items multiple choice short

answer extended response coursework

performance assessment evidence

accumulation but some of these like

coursework like performance assessment

like evidence accumulation take an awful

lot of time isn't it better to weave

collaboration into the learning

processes keep an eye on it make sure

it's being developed make sure children

are participating and not marginalized I

think it probably is we can boil all

these down to these three forms of

assessment banked items in a

multiple-choice awarding based

assessments like practicals

performance-based assessments some of

them are very very intensive this is the

final slide what I'm arguing is that not

that collaboration isn't important it is

I'm not arguing I'm arguing it certainly

isn't new I'm arguing that we've done it

brilliantly in the past in some schools

in some countries during some times and

we need to understand what they did

mostly they didn't talk about it they

put it in the general goods of education

and made sure it was happening

they didn't list it as an outcome to be

assessed to death we need absolute

clarity regarding well evidence

constructs we need a wide range of

constructs from discipline knowledge

right through to these personal capitals

but we need to locate constructs in the

correct place within this elaborate

model of the curriculum that I put up we

need to accept complexity in cost and

curriculum including assessment but we

need to understand that if we want to

assess things like collaboration they

are complex costly and could be

undependable in certain circumstances we

need to understand and deal with the

problems caused by high levels of

scrutiny accountability and the

distortions that can creep into our

assessment because of it and the final

thing is this Bill Schmidt is a very

wise old man and I love spending time

with him the international surveys

assess many many things most of which

are irrelevant to educational

performance but if you watch his lecture

from Cambridge they're in there because

they've been assessed before in the

national surveys they can be assessed

again and there for politicians can have

trend data which they love but the three

things which really predict individual

success and the success of educational

systems isn't anywhere near things

called 21st century skills its focus

rigour and coherence knowing what it is

that you want to assess knowing they're

real and knowing where to put them in

the curriculum thank you very much






The Description of Tim Oates - Assessing collaborative skills: OECD's interest in widening ideas of student performance