Arengu-uuringud
Arengu mõõtmine, hindamine, võrdlemine
Mati Heidmets
2013 sügissemester
Sotsiaalne mõõtmine ja hindamine
• Küsimus: kuidas suhtuda arenguhindamisse,
kuivõrd on see tõsiseltvõetav, mil määral äri,
mäng, PR?
• Mida tähendab mõõtmine ja hindamine inimese,
organisatsiooni, ühiskonna puhul? Mis on
mõõtmistulemused, kuidas neid seletada ja
mõtestada?
• Mõisteaparaat: mõõtmine, hindamine,
standardid, kontekst, võrdlus. Sotsiaalse
mõõtmise eripärad
• Kuidas mõõta Eestit!
Mõõtmine (measurement)
Mõõtmine: isiku või nähtuse suvalise tunnuse hindamine
kokkulepitud skaala suhtes. Tulemust üritatakse väljendada
numbriliselt. Rikkus, pikkus, tervis, tarkus, areng …
Mõõtmine – üldmõiste. Siia alla suvalise omaduse, tunnuse, ka
näit kvaliteedi hindamine väga erinevate protseduuride abil.
Vajalik mõõtmisinstrument – kaal, IQ test jne.
Measurement is the process of estimating the magnitude of some
attribute of an object, such as its length, weight, or depth relative to
some standard (unit of measurement), such as a meter or a
kilogram.
The act of measuring usually involves using a measuring instrument,
such as a ruler, weighing scale, thermometer, speedometer, or
voltmeter, which is calibrated to compare the measured attribute to a
measurement unit. Any kind of attributes can be measured.
Hindamine A (assessment)
Hindamine (assessment): hinnangu - numbrilise, sõnalise - andmine protsessile
või tulemusele. Koolihinne, hinnang MA tööle, projekti lõpphinnang,
arengutaseme hindamine …
Instrument sisaldab sageli subjektiivset momenti – kirjandi hindamine!
Hindamise tunnused: avalik või varjatud positiivne – negatiivne skaala, hindaja ja
hinnatav, varem teadaolevad protseduurid ja hindamiskriteeriumid
Assessment is the process of documenting, usually in measurable terms, knowledge,
skills, attitudes and beliefs.
There are two main types of assessment:
Summative assessment - Summative assessment is generally carried out at the end of a
course or project. In an educational setting, summative assessments are typically
used to assign students a course grade.
Formative assessment - Formative assessment is generally carried out throughout a
course or project. Formative assessment, also referred to as educative assessment,
is used to aid learning. In an educational setting, formative assessment might be a
teacher (or peer) or the learner, providing feedback on a student's work, and would
not necessarily be used for grading purposes.
Summative and formative assessment are referred to in a learning context as
"assessment of learning" and "assessment for learning" respectively
Hindamine B (evaluation)
Evaluation – väärtustav hindamine Asja, isiku või nähtuse väärtuse, kvaliteedi, olulisuse,
meeldivuse määratlemine. Subjektiivsem, kvalitatiivsem kui “assessment”.
Evaluation is systematic determination of merit, worth, significance of something or someone using
criteria against a set of standards. Evaluation often is used to characterize and appraise subjects
of interest in a wide range of human enterprises, including the arts, criminal justice, government,
health care, and other.
The American Evaluation Association has created a set of Guiding Principles for evaluators. The order
of these principles does not imply priority among them; priority will vary by situation and evaluator
role. The principles run as follows:
Systematic Inquiry: Evaluators conduct systematic, data-based inquiries about whatever is
being evaluated.
Competence: Evaluators provide competent performance to stakeholders.
Integrity / Honesty: Evaluators ensure the honesty and integrity of the entire evaluation
process.
Respect for People: Evaluators respect the security, dignity and self-worth of the respondents,
program participants, clients, and other stakeholders with whom they interact.
Responsibilities for General and Public Welfare: Evaluators articulate and take into account
the diversity of interests and values that may be related to the general and public welfare
Hindamine A ja hindamine B: hinde omistamine ja väärtustamine! Erinevus tinglik, eesti keeles
sama mõistega tähistatav tegevus
Sotsiaalse mõõtmise ja hindamise
elemendid
1-Mõõtja ja mõõdetav, hindaja ja hinnatav
2-Instrument (mille abil mõõtmine toimub - mõõdulint, skaala,
küsimustik)
3-Standard, kontekst (mille suhtes mõõtmistulemust mõtestatakse)
Eripärad:
– Mõõtja pole alati neutraalne
– Instrumente palju, skaalad kokkuleppelised
– Standardid väga erineval arenguastmel
Tulemus: sotsiaalse mõõtmise tulemused reeglina tõenäosuslikud, ka
numbriliselt väljendatud tulemus tähendab sageli (tugevalt
subjektiivset) hinnangut, mõõtmise ja hindamise piirid ähmased,
mõtestamine toimub võrdluse kaudu
Kas toimub avaliku arvamuse mõõtmine või hindamine, kas mõõtmine,
mille täpsusaste on +/- 3% on hindamine või mõõtmine?
Mõõtmine = protseduur, hindamine = tulemuste konteksti asetamine,
standardiga või teistega võrdlemine!?
Sotsiaalse hindamise vormid
Väljakujunenud hindamisskeemid organisatsioonis,
ühiskonnas:
Eelhinnang – tulevikuvõimaluste hinnang. Tööintervjuu
(kandidaadi sobivus ametikohale), sisseastumisvestlused,
turunõudluse prognoos
Enesehinnang – eneseanalüüsi raport, motivatsioonikiri
Tulemushindamine – riigieksamid, hinnang võistkonna x hooajale
Eksperthinnang – kollektiivse tarkuse kasutamine nähtuse x
hindamiseks
Auditeerimine – finantsaudit (protseduuride järgmine)
Akrediteerimine – näit õppekava akrediteerimine (nõuetele
vastavuse hindamine)
Kvaliteedihindamine - EFQM
Uurimistöö – isiksuse omaduste mõõtmine, organisatsioonikultuuri
hindamine, majandusarengu mõõtmine. IAI !
……………………………………………………………
Näiteid: auditeerimine
Auditeerimine: isiku, organisatsiooni, protsessi või tulemuse süsteemne
hindamine. Eesmärk – määratleda tegevuse nõuetele vastavuse ja
usaldusväärsus. Finantsaudit, keskkonnaaudit, kvaliteediaudit …
The most general definition of an audit is an evaluation of a person, organization,
system, process, project or product. Audits are performed to ascertain the validity
and reliability of information, and also provide an assessment of a system's internal
control. The goal of an audit is to express an opinion on the
person/organization/system etc. under evaluation based on work done on a test
basis. Due to practical constraints, an audit seeks to provide only reasonable
assurance that the statements are free from material error. Hence, statistical
sampling is often adopted in audits. In the case of financial audits, a set of financial
statements are said to be true and fair when they are free of material misstatements a concept influenced by both quantitative and qualitative factors.
Traditionally audits were mainly associated with gaining information about financial
systems and the financial records of a company or a business (see financial audit).
However recently auditing has begun to include other information about the system,
such as information about environmental performance. As a result there are now
professions that conduct environmental audits.
Sise- ja välisaudit
Internal auditors are employees of a company hired to assess and evaluate its system
of internal control. To maintain independence, they present their reports directly to
the Board of Directors or to Top Management. They provide functional operation to
the concern. Internal auditors are employees of the company so that they can easily
find out the frauds and any mishappening.
External auditors are independent staff assigned by an auditing firm to assess and
evaluate financial statements of their clients or to perform other agreed upon
evaluations. Most external auditors are employed by accounting firms for annual
engagements. They are called upon from the out side of the company
An audit is an educational term in the United States for the completion of a course of
study for which no assessment is made or grade awarded. The grade of "audit" is
awarded to those who have elected not to receive a letter grade for a course in which
they are typically awarded.
In this case, 'audit' indicates that the individual has merely received teaching, rather than
achieved a given standard. The term 'audit' is Latin, translating as 'he hears'; in other
words, the student has experienced the course but not been assessed.
This technique is often employed by individuals who wish to take a specific course
without the risk of under-performance resulting in a poor or failing grade. This can be
helpful when reviewing a long-unstudied subject, when first beginning the study of a
discipline wherein one has little experience or confidence, when taking a course
merely for enjoyment with no need or desire of academic credit, et cetera.
Akrediteerimine
Akrediteerimine: kehtestatud nõuetele vastavuse hindamine, sellekohaste
tunnistuste väljastamine
Accreditation is a process in which is certification of competency, authority, or credibility
is presented.
Organizations that issue credentials or certify third parties against official standards are
themselves formally accredited by accreditation bodies (such as ); hence they are
sometimes known as "accredited certification bodies". The accreditation process
ensures that their certification practices are acceptable, typically meaning that they
are competent to test and certify third parties, behave ethically, and employ suitable
quality assurance.
One example of accreditation is the accreditation of testing laboratories and certification
specialists that are permitted to issue official certificates of compliance with
established standards, such as physical, chemical, forensic, quality, and security
standards
Educational accreditation is a type of quality assurance process under which services
and operations of an educational institution or program are evaluated by an external
body to determine if applicable standards are met. Should standards be met,
accredited status is granted by the agency
Erijuht: Diplomatic accreditation is the process in which an ambassador is certified as
one country's official representative to another
Kvaliteedihindamine
Kvaliteedihindamine: hinnang organisatsiooni, protsessi või produkti omaduste vastavuse
kohta kehtestatud (kvaliteedi)nõuetele
Quality assurance, or QA for short, refers to planned and systematic production processes that
provide confidence in a product's suitability for its intended purpose. It is a set of activities
intended to ensure that products (goods and/or services) satisfy customer requirements in a
systematic, reliable fashion. QA cannot absolutely guarantee the production of quality products,
unfortunately, but makes this more likely.
•
Two key principles characterise QA: "fit for purpose" (the product should be suitable for the
intended purpose) and "right first time" (mistakes should be eliminated). QA includes regulation of
the quality of raw materials, assemblies, products and components; services related to production;
and management, production and inspection processes.
•
It is important to realize also that quality is determined by the intended users, clients or customers,
not by society in general: it is not the same as 'expensive' or 'high quality'. Even lowly bottom-ofthe-range goods can be considered quality items if they meet a market need.
•
Whereas Quality Control emphasises testing and blocking the release of defective products,
Quality Assurance is about improving and stabilizing production and associated processes to
avoid or at least minimize issues that led to the defects in the first place. However, QA does not
necessarily eliminate the need for QC: some product parameters are so critical that testing is still
necessary just in case QC fails.
Standard, norm
Standard – kokkuleppeline norm, nõuete kogum, hindamiskriteerium. See,
mille suhtes asja, isikut või nähtust hinnatakse. Formaliseeritud,
kirjapandud, avalikustatud. Tehnilised standardid, kullastandard,
kvaliteedistandardid, sotisaalabi standard …
Standard is an established norm or requirement. It is usually a formal document that establishes
uniform engineering or technical criteria, methods, processes and practices. The standardization
process may be by edict or may involve the formal consensus experts
Standard sotsiaalse mõõtmise puhul:
– Täpselt määratletud tase, omadus, tunnus (punktide arv autokoolist
läbisaamiseks, lävend ülikooli sissesaamiseks)
– Kehtestatud normid ja nende järgimine (kvaliteediaudit)
– Tunnuse jaotus populatsioonis (standardiseeritud test, keskmine eluiga)
– Kontekst (võrdlus seniste hindamistulemustega, uuringutega)
– Sisemine standard (eksperdi hinnang korruptsioonitasemele)
Sotsiaalse hindamise standard vs tehniline standard!
Võrdlus
• Sotsiaalse hindamise päästerõngas –
hindamistulemused konteksti, võrdlusse. Võrdlus
kui orientiir, võrdlemine kui mõtestamine
• Võrdluses “standardiks” teised analoogilised
asjad, isikud, protsessid …
• Hea õpilane (vaid võrdluses teistega!), arenenud
riik, pikk eluiga, tark laps, tubli Eesti …
• Võrdluspõhine MA töö!
Mõõtmise/hindamise eesmärk –
usalduse loomine
Otsime: kindlust, püsivaid teadmisi, püsivaid seoseid. Kardame
määramatust ja ettearvamatust.
Kuidas seda teha sotsiaalses mõõtmises, mis olemuslikult tõenäosuslik
ja mitmetähenduslik. Mida saame uskuda, kus on tõde?
Sotsiaalses maailmas on tõde konstrueeritud, kokku lepitud, ajutine
usalduse objekt?!
Social constructivism holds that truth is constructed by social
processes, is historically and culturally specific, and that it is in part
shaped through the power struggles within a community.
Constructivism views all of our knowledge as "constructed," because
it does not reflect any external "transcendent" realities (as a pure
correspondence theory might hold).
Consensus theory holds that truth is whatever is agreed upon, or in
some versions, might come to be agreed upon, by some specified
group. Such a group might include all human beings, or a subset
thereof consisting of more than one person.
Sotsiaalse mõõtmise
usaldusväärsus
Usaldus - teadmine “tõe” tekkimise protseduuride kohta. Kuigi lõplikku
tõde pole olemas, toodetakse seda iga päev juurde.
Usaldusväärsed tootjad on enesekriitilised ja refleksiivsed. Välja
mõelnud protseduurid sotsiaalse mõõtmise usaldusväärsuse
suurendamiseks
Valiidsus – kas mõõdame, seda, mida arvame mõõtvat. Konstruktid ja
operatsionaliseeringud
Reliaablus – kuivõrd püsiv (nähtust püsivalt kirjeldav) on
mõõtmistulemus?
The considerations of validity and reliability typically are viewed
as essential elements for determining the quality of any
assessment. However, professional and practitioner associations
frequently have placed these concerns within broader contexts when
developing standards and making overall judgments about the
quality of any assessment as a whole within a given context
Sotsiaalse mõõtmise ja hindamise
loogika
• Mõõdame/hindame konstrukti ehk kindlas
populatsioonis käibivat ja mõistetavat nähtust
(organisatsioonikultuuri, pühendumus, hoiak)
• Mõõtmisinstrument kokkuleppeline ja üks
paljudest
• Tulemused mõtestatud suhetes standardiga
või võrdluses
• Nimetamise võim, klassifitseerimise võim!
(sooline ebavõrdsus, vaesuspiir, eliitkoolid)
• Iga tulemus on vihje asjade tegeliku sisu kohta