See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/372651190 Legitimate Subjective Observation [LSO] & The Evaluation of Soft Skills in the Workplace Conference Paper · May 2002 CITATIONS READS 7 28 1 author: Glenn Peter Costin Deakin University 16 PUBLICATIONS 235 CITATIONS SEE PROFILE All content following this page was uploaded by Glenn Peter Costin on 27 July 2023. The user has requested enhancement of the downloaded file. Legitimate Subjective Observation [LSO] & The Evaluation of Soft Skills in the Workplace A Concurrent Session Briefing Paper Spotlight on the Provider The National Training Framework - Training Partnerships and Regional Development Albury Convention Centre and Performing Arts Centre May 2002 Glenn P. Costin Introduction Legitimate Subjective Observation [LSO] was developed in response to what, at the time, was felt to be a missed opportunity. By the end of the year 2000 a project initiated by the Dusseldorp Skills Forum and WorldSkills Australia [formerly WorkSkill] known as Quick Hut 2000 had been completed with significant evidence of soft skill development, yet only a cursory exploration into the evaluation of these skills had been made. In truth, we did not know at that time how to conduct such an assessment, nor did we hold a very clear interpretation of what soft skills were. This, coupled with the drive by both the Australian National Training Authority [ANTA] and WorldSkills to include the assessment of soft skills within National Training Packages and hence WorldSkills challenges, made it clear that some attempt needed to be made, both to define the skills and to find a means by which they could be evaluated. This paper sets out to describe the process of LSO as it has been used in recent WorldSkills Australia skills challenges. In addition, it explores the possible application of the process to the workplace. To do so however some background to the activities of WorldSkills Australia leading up to the initial deployment of LSO must first be given, along with a definition of soft skills and a discussion of their contextual complexity. WorldSkills Australia Over the past 18 years WorldSkills Australia has developed an Australia wide system of skill challenges which recognise competency attainments at regional, national and international levels in over 50 occupational categories. A skills challenge being an almost sports like event whereby young trades people compete against one another to demonstrate who is the best in their particular field. In its most basic form a skills challenge is 2 or more [generally eight or more] same trade competitors working independently on identical projects with identical time and material constraints and being 'measured' by an 'objective' marking scale to national competencies. This marking scale is used to 'rank' competitors in order of I those who demonstrated the most skills to those who demonstrated the least, such performance being reflected in the awarding of medals - gold, silver and bronze. Through ANTA, WorldSkills has recently begun to generate more formal links with the mainstream Vocational Education and Training [VET] sector. Such links have led to several alterations in the design of skills challenges that have brought them in line with competency assessment and closer to contemporary Australian and regional industry practice. With this shift came the need to assess a loosely defined skills grouping referred to as 'soft' skills: skills such as planning, managing, communication, skills transfer and problem solving. As the National and Regional designer of WorldSkills challenges in the field of carpentry for the past 7 years, I was already exploring the possibilities of assessing the 'soft' skills of our competitors as against, or alongside, the 'hard' skills that we would normally have focused upon. This exploration was duly encouraged and at the WorldSkills National Skills Challenge held in Adelaide in 2001 the first carpentry team skills challenge was held and a process for the 'evaluation' of soft skills was employed. Based upon the principles of ethnography and naturalistic inquiry [see Lincoln & Guba 1985], and within the theoretically framework of constructivism and situated learning [see Lave & Wenger 1991, Wenger 1998], this process has become known as Legitimate Subjective Observation or LSO. Skills and Context To go any way towards developing an understanding of this process, its application, and its possible value to the evaluation of soft skills in the workplace, much territory must be covered, not the least being the definition of soft skills and their relationship to context. However, prior to doing so, an anecdote drawn from the experience of the Quick Hut 2000 project may help in developing an understanding of the value both of soft skills and in being able to evaluate them. After observing the competitors over such an extended period of time, it had become clear to many of the organisers that of all the competitors one was significantly skilled at team bonding. The success of the Quick Hut project had led to one of the teams representing Australia at the 1st APEC Youth Skills Camp held in Korea 2000. The hut that this team had designed and constructed as part of the Quick Hut event had been shipped over to Korea and was being reassembled at the camp using a multinational team, few of whom could speak to each other in any common tongue. Three of the Australian team aided in the reassembly. Two trades people and the third, actually a university student studying Early Childhood development. Whilst the trades people helped in deciding what went where and how - it was the seemingly odd fit of the non trades person that brought the team together. It was her skills as a 'peoples' person, a team builder, that bonded the confusion of languages, both verbal and non verbal, into co-operation and understanding. The significance of the above anecdote is two fold, firstly in that here is an obvious demonstration of the value of soft skills to the workplace [albeit a transposed one] and the transferability of that skill from context to context. What is also significant however, and the more relevant in this case, is the capacity of the organisers to identify and then strategically and successfully employ the individual when and where needed based upon what can at best be described as subjective evaluation. II It was however a trustworthy one, one based upon multiple observations of performance in context. This question of what or who can provide a trustworthy evaluation of competence has produced much debate over recent years [Hager 1995; Jones 1999; Blackmore 1999; Bolton et.al. 1999; Mulcahy 2000]. This has been particularly so with regard to soft skills and the generic or 'key' competencies. Crowley, Garrick & Hager [2002 p.1] have noted for example that despite having determined that generic competencies are "…relevant to industry improvement…", "…industry has yet to develop systems and mechanisms for facilitating the optimum transferability and recognition of such skills". Whilst this paper defines soft skills as differing from generic competencies, it will be noted that soft skills significantly underpin the entire generic competency grouping. Thus the process and underlying theoretical framework of LSO, while informing us of soft skill performance and development levels, goes some way also to answering [the individual rather than corporate or industrial aspects of] Crowley, Garrick and Hager's 'Phase Three' question: that which asks why "Experience and/or expertise in workplace reform do not necessarily transfer easily from job to job…" [Crowley et al. 2002, p. 1]. In setting out to inform about the process of LSO it should be acknowledged from the outset that LSO in itself is not a complete evaluation of skilled performance [though it does cover three of Hagar's (1995) evidence gathering methods and to a limited extent the other two also: that is 'Simulations', 'Skills tests', 'Direct observation', and to a lesser degree 'Questioning techniques' and 'Evidence of prior learning']. However coupled with other more conventional measurement and assessment processes it is argued that LSO will lead to a clearer picture of this performance. It being proposed that such an evaluation process offers a significant, and generally missed, portion of the evidence required to develop a trustworthy and transferable interpretation of workplace competence. That is to say, an interpretation of competence that is perhaps more indicative of skilled performance in other, future, and alternate contexts. At this point it will have become apparent that the soft skill grouping is much larger than the rather loose definition given at the outset suggests. It is therefore relevant to offer a clearer distinction between hard skills, soft skills and generic competencies. This will be followed by a brief discussion as to why the concept of evaluation holds more relevance to soft skill performance than does assessment. From these descriptions an outline of LSO shall be developed, providing a picture of what the process must do, as well as what it is. Soft and Hard Skills - The basic dichotomy Soft Skills Tend to be: Skills by which the individual interacts with, interprets, structures, co-ordinates or otherwise informs the social and physical environments within which physical, societal and or personal product may be generated. Soft Skill Examples - Skills of process and community Planning, preparing, organising, communicating, observing, describing, identifying, empathising, learning, intuition, sense of timing, attitude, tool development, skill III transfer, process development, creativity, ingenuity, design, sense of aesthetics, endurance… Hard Skills Tend to be: Skills by which the individual interacts physically with technology during the generation of physical product. Hard Skill Examples - Skills of product and individual Tool use, formula use, text use, measuring, marking, strength, fitness, endurance… Note: The term ‘product’ thereby means more than a physical item. The deployment of hard skills is invariably with the aim of producing a physical product or item of some kind, acknowledging that this may be no more than an increase or decrease in human tissue [i.e. weight gain/loss programs]. The deployment of soft skills however, whilst often used to aid the efficient use of hard skills and hence the production of a physical item, are equally often used to reach all manner of social, personal and interpersonal goals. Goals such as: personal well being; peer well being; group consensus; community spirit and the like. They are not confined to economic ends. Having provided a dichotomy, one must be wary of this Anglo-Saxon, European or ‘Western’ penchant [to dichotomise]. With skills, as with so much of human endeavour and behaviour, the dichotomy approach has some functional value for limited explanation. A far richer understanding however is had by surveying the areas of union or melding rather seeking an often indefinable point of fissure or boundary. Diagram A below gives a clearer picture of the flow from what may be termed ‘high end’ soft skills to ‘high end’ hard skills and the skill types that begin to show the blurring of the distinction [and one reason why dichotomising is a dangerous act] 'High end' Soft Skills Soft Skills Ingenuity, sense of aesthetics, empathising, learning, creativity, intuition, sense of timing, communication problem solving, attitude, tool development skills transfer, process development, Designing, describing, identifying Planning, preparing, organising Endurance… Strength, fitness 'High end' Hard Skills text use, formula use Tool use Hard Skills Diagram 2. The Soft skill - Hard skill continuum IV A further reason for being wary of the dichotomy is that its rare to find a hard skill not underpinned by one or more soft skills. Hard skill performance is hence often dependent upon soft skill capacity. Learning itself is a soft skill. Skill transfer is a soft skill. It may be argued therefore that hard skills are developed through the application of soft skills. The following examples demonstrate this point [soft skills in bold] Attitude and perception of quality: endurance [why bother], product finish [near enough], tool use [rough], formula selection [simplest], choice of method [easiest] Problem solving, planning, preparing, organising: output, available time to perform hard skill, material availability, product quality, hard skill acquisition & development Tool development, skills transfer, process development: tool use, tool choice, quality of finish, time to perform hard skills, hard skill acquisition & development Designing, describing, identifying ingenuity, aesthetics, learning, creativity, intuition, sense of timing: Choice of formula, text use, tool selection; available time for hard skill performance. Hard skill acquisition & Development Empathy, communication: tool use reliant upon teams or groups, product outcome when multiple hard skills held by multiple practitioners are required. Hard skill acquisition & development Finally, and as previously noted, soft skills are integral to all the generic or 'key' competencies as depicted by the Finn [1991] Mayer [1992] and Carmichael [1992] reports. These competencies or skill clusters are the obligatory core of all Australian National training packages. As skill clusters, generic competencies are in themselves combinations of hard and soft skills with only the competency ‘using technology’ as being definitively more hard than soft. However as argued previously, the appropriate and skillful application of hard skills is soft skill dependent and hence as a competency ‘using technology’ is not the application of hard skills alone. Likewise, to perform the other 'softer' key competencies adequately, one must often become skillful in the use of technology appropriate to those tasks. For example, communicating ideas and information involves language [arguably a technology], information transfer systems of some form [even the humble quill is a piece of technology], and a variety of formula for structuring the information in a manner best suited to the target audience. Generic Competencies Skill groupings [s=soft, h=hard] Collecting, analysing and organising information s/h Communicating ideas and information s/h Planning and organising activities s/h Working with others and in teams s Using mathematical ideas and techniques s/h Solving problems s Using technology h/s Using cultural understandings s V Transferability With soft skills underpinning all hard and generic skill activity, it is argued that a trustworthy evaluation of soft skill level will be far more indicative of future performance than any measure of hard skill alone. Measurements and assessments of hard skill tend to focus upon the 'explicit' or observable, and hence upon physical product. Transferability however is dependent more upon the skills that support that product production, the skills of communication, interpretation, problem solving, team work and so on. Significantly, transferability depends upon how close the soft skill levels demonstrated in one context, relate to those required for competent 'over all' [hard and soft] skilled performance in another [Lincoln & Guba 1985]. Evaluation or Assessment As Lovat and Smith [1991, p. 160] state evaluation is the "…overarching concept which both depends upon measurement and assessment, and brings together a number of measurements, and assessments to make a composite judgement or decision.". As shall be argued, a trustworthy appraisal of soft skill performance is only possible when such an appraisal is taken in context. Such being the case, a variety of measures and assessments must be conducted on this context covering a raft of intertwined skills before a judgement may be made about any particular one of them. LSO, it will demonstrated, is conducted whilst a subject, or more correctly a group of subjects, under go a variety of measurement and assessment processes in context. What is taking place therefore is more correctly termed an evaluation of performance, rather than a single assessment of performance. Indeed, given this contextually embedded and mutually intertwined nature of the skills, it may be argued that soft skills may only ever be evaluated, and that any attempt at independent assessment has limited trustworthiness either to the context of performance or contexts of the future [this latter point being for Lincoln & Guba (1985, p. 297) the issue of 'transferability' between 'sending' and 'receiving' contexts]. Evaluating Soft Skills Having described what soft skills are, the issue of the complexity of evaluation becomes apparent. Following the description of above, soft skills are skills derived and developed from the subjective interpretation of the context in which the performer is embedded, by that performer and or their peers. They are intensely human and tend to defy objective analysis, particularly when this analysis comes from outside the context in which they are performed [as against hard skills that are frequently mechanised and the 'product' of which may be physically measured]. Their performance at a level demonstrably competent in one context, is not always indicative of skilful performance in another. Any attempt to measure the performance of a single soft skill in isolation from the whole person and their context is thus doomed to failure as such does not take into account this human and contextually dependent nature. VI Any process deployed to inform about soft skill performance must therefore provide judgements in relation to the context as a whole. Any 'instrument' employed must be able to decipher a context and make a determination, informed during the demonstration of a whole raft of skills, that a particular skill is or is not being performed competently to that particular setting. Or that the further development of a particular skill would enhance this overall skill performance. To do so, this 'instrument' must be familiar not only with the skills under perusal, but also the context of their performance, and to some degree at least, probable future performance contexts. Currently there is only one such 'instrument' available to us, that being a person or persons familiar with the context, experienced in the skills being assessed, and skilled in observation. They must know where to look, what to look for, and how to look. Following on from the above it will also have become apparent that soft skills are particularly difficult to perceive as individual skills. Deriving from the context in which they are performed, they form part of that context and as such are embedded in the overall skill performance. Assuming therefore, by a measure of end product alone, that if a given task has been completed on time and to an acceptable standard then the requisite soft skills underpinning that overall skill performance are likewise at an acceptable standard is a dangerous assumption. The supporting soft skills, such as planning, management or skill communication for example, may well have been 'adequate' to bring about a successful conclusion in that given context. However it is also possible that success came about due to 'luck', or that there where no distractions, or that it suited the mood of the performer to strive on this occasion, when on another it may not have [a poor attitude stimulated by the external motivation of the assessment for example]. Soft skills hence have low objective perceptibility - they are difficult to perceive [observe within the context of an individual carrying out independent tasks] and do not relate to objective criteria [one cannot for example state that 3 team meetings, 5 minutes long and one hour apart, should be held as indicative of competent planning – context dictates number, duration and timing]. Subjectivity is hence an obvious issue in any measurement of soft skill performance [and arguably skilled performance generally]. Subjectivity of course has not been on common ground with the positivistic principles of modern, formal, Vocational Education and Training [VET] for some time, it has most certainly been regarded as suspect within the WorldSkills challenge environment. Yet it has long been acknowledged that subjectivity permeates all human interaction including modern science and all other forms of educational practice, measurement and assessment [Lovat & Smith, 1991, Lincoln & Guba 1985]. Any human decision is based upon context and paradigm. We subjectively chose what to measure 'objectively', we subjectively determine what shall be studied and what shall not be, we subjectively receive funding for research in one direction, but not in another. As Lovat and Smith [1991, p. 154] state when discussing assessment, "…all measurement is itself subjective and rests upon interpretative judgement". Borrowing from Lincoln and Guba [1985] objectivity and subjectivity are thus best seen as the two alternate ends of a continuum. It is the paradigm dominant in a particular context that lends legitimacy to any particular set of measures, and in so doing provides the 'guise' of objectivity to that which by another paradigm would be questionable. The issue in workplace assessment is therefore not a question of "is VII a process objective?", but "in lending legitimacy to the subjective, how trustworthy is the information derived from the process as a whole?". Given that there are various paradigms from which legitimacy may be lent, one paradigm not necessarily acknowledging the legitimacy of another, it is the issue of trustworthiness that becomes paramount, and, given the purpose of workplace assessment, this ultimately infers transferability. Legitimate Subjective Observation: A Description Having outlined issues that brought the practice of LSO into being, and the complexities of that which it has been designed to evaluate, a description of LSO itself is in order. LSO is very much that which its title expresses: the legitimising of observations that would otherwise be deemed subjective and illegitimate under the paradigm of 'modern' training. Legitimacy With regard to the observer, legitimacy is offered on strict 'grounding' rules, rules that demand that the observer be not just familiar, but intimate with the context within which the observations are made. That is, the observer should be grounded in the context [therefore a part of it] without the pretence of being objectively removed from it. They must be skilled in the 'hard' skills being performed when the 'product' [see note to soft and hard skill definitions] of the context is predominantly derived from such skill. As stated previously, for the observer to hold legitimacy they must also be proficient in the practice of soft skill observation [that is they must be knowledgeable of behaviour indicative of the various soft skills being monitored]. Observation Observations of skilled performance should be made when such performance is conducted in the normal context of that performance. That is, carpenters doing carpentry on a construction or building site, hairdressers doing hairdressing in the salon atmosphere and so on. Observations conducted outside of the normal context of performance hold limited legitimacy, such legitimacy decreasing the further removed from this 'normal context' the observations are made. Any 'offsite' observations [taken during formal training conducted by an RTO for example] would hence have to be taken within a context contrived to closely mimic the normal context of performance. LSO: The WorldSkills Experience As outlined earlier the conventional skills challenge as conducted by WorldSkills Australia did not greatly resemble the 'normal context of performance'. In addition, the standard assessment system employed was almost aggressively 'objective' in design. The 1999 National carpentry challenge was a prime example of this format, with several hundreds of objectively measured criterion used to determine performance [a system that took several hours to produce a result]. Only the product was assessed. However as stated earlier, in the year 2000 a project known as Quick Hut 2000 was conducted by WorldSkills [then WorkSkill] and the Dusseldorp Skills Forum. This project gave significant insights on a range of VIII training and skill development issues, particularly regarding soft skills and the need for trustworthy process for their evaluation. The 2001 WorldSkills Nationals held in Adelaide 2001 then saw the introduction of a skills challenge developed from the information derived from the Quick Hut 2000 project. This was also the first explorative application of LSO. The new skills challenge, in the form of a carpentry team event, sought to match legitimate observers with a legitimate context in which to make their observations. The legitimate context was developed by 'destructuring', a process by which the formal constraining structure of the event was decreased to allow for more informal interaction and team based problem solving. In a 'destructured' event, instead of constructing someone else's solution to a problem, competitors are asked to confront the problem, and then construct their own solution. They must do this in a context impacted upon by the same economic, legislative [Australian standards, OH&S and the like], physical and time constraints as they would in industry. Destructuring served three purposes. The first and primary, being that it provided a context closely replicating that of normal performance. Secondly and significantly, it made soft skills more explicit or observable [being team based, competitors had to communicate their ideas and solutions to others, and processes had to be explicit for team members to follow]. Finally, it created a context in which soft skill development was encouraged. This latter being through peer interaction combined with the competitive problem based nature of the challenge itself. Destructuring therefore allowed for this contrived, off site, context to relate more honestly to work place reality. Observations taken within it could therefore be held to be more trustworthy and transferable with regard to future workplace activity. That is, performance in the WorldSkills carpentry challenge as an off site context became more indicative of performance on site. Competitors - the Observed Individuals worked within a team setting to produce a combined result. The task required individual competitors to complete sections or components by themselves, as well as sections as a team. Teams and individuals had to do all the things a competent carpenter did on site, that is: design, engineer, make aesthetic and economic decisions, order and store materials, work to time lines, interact with other trades, clients and supervisors, comply with OH&S regulations. The Judges - the Observers Judges were deemed legitimate in that they were expert in the trade area [context] being observed. Legitimate in that they were trained in the observation of soft skills [to recognise behaviour indicative of sound soft skill performance relevant to that context], albeit this training was minimal due to time constraints. Legitimate in that they were part of the context [competitors needing to interact with them when 'ordering' materials or if seeking specialist or expert advice]. Competitors were observed as they constructed a small building [garden shed] to a given design criteria. Judges observed competitors interacting whilst planning, budgeting, designing, problem solving and constructing over a 3 day period. One judge per team, judges observing each team [and therefore individual] for an equal period of time by rotating shifts. Judges responded to soft skill criteria documents IX for both teams and individuals in an openly 'subjective' manner. Results were averaged across all three judges. Outcome The results were significantly informative at two levels. The one being with regard to the LSO process, the other with regard to the value of the information such a process may offer industry. As determined by combining the LSO results with the 'objective' measures of hard skill performance, the winning team held both the Gold and Bronze [first and third] medal winners. The winning team however was effectively lead [as attested to by all judges and the team members themselves] by neither medal winner but by a competitor ranked 6th. The gold medal winner and the No. 6 competitor however held the same overall soft skill scores [the two top soft skill rankings]. A breakdown of these scores showed however that No. 6 had the higher leadership level. Hence the results could inform industry both of who was the better all round carpenter, and who the better choice for foreperson, team leader or supervisor. With regard to the process of LSO, the results showed that greater accuracy could be developed through more targeted criteria specific to a whole raft of differing soft skills as a given context deemed relevant. Applying LSO to the work place In applying the LSO process to the workplace, trustworthiness would arguably increase due to the closer contextual relevance of the observations obtained. Trainees would be observed performing tasks in the context of their general performance. Evidence of 'competent' performance both of soft and hard skills would be built up over an extended period of time. Evidence would be compiled by legitimate observers selected from within the workplace and 'overseen' by an external workplace assessor. Evidence being derived from multiple sources/observers whenever possible. Selection the observers: In larger firms it is envisaged that forepersons, team leaders, trainee supervisors would be the most appropriate. In smaller firms, the managers or Registered Training Organisations [RTOs] using destructured team tasks. The ultimate evaluation would be made by the external workplace assessor based upon a combination of all observations, giving the greatest weight to those closest to context of performance. Observer support systems: As LSO is designed to supplement rather than supplant the existing workplace assessor training programs, assessor would need to be supported in the observational practice by a number of means. Firstly an outline of observational best practice would need to be established [overt observational practices that reduce negative influences, observational reporting techniques and the like]. Secondly soft skill performance criteria would need to be identified, written in a manner that allowed for contextual interpretation, yet retaining a national framework. Observers would need to be provided with examples of indicative behaviour and settings in which this behaviour is becomes most explicit. Determinations would also have to be made on the ultimate X relationship between soft skills observed and the generic competencies as currently defined. LSO and soft skill development [skill gap training] LSO provides a framework for the development of soft skills within both the context of employment and destructured challenges [as conducted by WorldSkills or by RTOs]. This is achieved by LSOs offering guidance to trainers, employers and the employees [trainees] themselves as to soft skill gaps. Future training and experiences can therefore be maximised through targeting individual needs. LSOs also mean that soft skills, and their relevance to workplace performance, become more explicit to both the performer and the observer. It being a given that the observer shall always influence the observed to some degree, by adhering to overt practices of positive critic this influence is potentially a motivational influence for skill development rather than against. Hence like the WorldSkills model depicted earlier, a trainee's soft skill development is being supported simply by that trainee being knowingly involved in the process. Summary The process of LSO as modelled in the WorldSkills carpentry skills challenge can support the existing workplace assessor program by providing insights into those soft skills that underpin all levels of competent performance. It has been argued that as the underpinnings of competence, any assessment of skilled performance that does not include an evaluation of soft skills through some legitimate means has limited trustworthiness with regard to the context of performance, and limited transferability with regard to future performance. Soft skills being developed in and defined by the context of their performance, legitimacy may only be offered when the means by which evaluations are made are contextually embedded. LSOs cover this criterion. They are taken by human observers legitimately using their experience and skill history to make judgements of performance as they see its relevance and applicability to specific vocational contexts. As such LSOs provide much needed insights into skilled performance that no 'objective' assessment is able to. They can perhaps provide the difference between competency and work readiness. XI References: Blackmore, P. [1999] A Categorisation of Approaches to Occupational Analysis, Journal of Vocational Education and Training, 51, 1, pp. 61-76 Bolton, A., Brown, R., McCartney, S. [1999] The Capacity Spiral: four weddings and a funeral, , Journal of Vocational Education and Training 52, 2, pp 585604. Carmichael, L. [1992]. Australian vocational certificate training system. Canberra: Australian Government Printing Service. Crowley, S. Garrick, J. Hager, P. [2002] Construction Work: The Hidden Impact of Generic Competencies in the Australian Construction Industry, Conference paper presented at The fifth Australian VET Research Association Conference - Making a world of difference? Innovation, internationalisation, new technologies and VET 21-22 March 2002. Finn, B. [1991]. Young people's participation in post-compulsory education and training. Canberra: Australian Government Printing Service. Hager, P. [1995] Competency Standards - a Help or a Hindrance? An Australian Perspective, The Vocational Aspect of Education, 47, 2, pp. 141-151. Jones, A. [1999] The place of Judgement in Competency-based Assessment, Journal of Vocational Education and Training, 51, 1, pp.145-160. Mayer, E. [1992]. Putting general education to work: The key competencies report. Melbourne: Australian Education Council. Mulcahy, D. [2000] Turning Contradictions of Competence: competency-based training and beyond, Journal of Vocational Education and Training 52, 2, pp 259-282 Lincoln, Y. & Guba, E. [1985] Naturalistic Inquiry. Newbury Park: Sage. Lave, J. & Wenger, E. [1991] Situated Learning: Legitimate Peripheral Participation. Cambridge University Press. Wenger, E. [1998] Communities of Practice: learning, meaning, and identity. Cambridge University Press. XII View publication stats