tri hanggono-output komponen 2

8/3/2019 Tri Hanggono-Output Komponen 2

http://slidepdf.com/reader/full/tri-hanggono-output-komponen-2 1/34

Component 2 The Certification of Graduates Using

a National Competency-basedExamination

Tri Hanggono Achmad2nd HPEQ Conference

Denpasar, 4 December 2011



Outline of presentation

Overview of Component 2

Report and key performance indicator

Lesson learned

Challenges and Future plan



Overview of Component 2



Health Professions EducationQuality (HPEQ) ProjectRationale: Indonesia’s current health indicators,the current status of Indonesia’s health professionaleducation programs, and on the increasing demandfor larger numbers of and more highly qualified

health professionals.Goals: strengthen quality assurance policiesgoverning the education of health professionals inIndonesia through

1. rationalizing and assuring competency focusedaccreditation of public and private health professionaltraining institutions,

2. developing national competency standards and testingprocedures for certification and licensing of healthprofessionals, and

3. building institutional capacity to employ results based



Objectives

v Sub-component 2.1: Establishing anIndependent National Agency for CompetenceExamination of Health Professionals

v Sub-component 2.2: Improving theMethodology and Management of theNational Competency-based Examination

v Sub-component 2.3: Developing an ItemBank Networking System to Support theNational Competence Examination



Main Program 2010 2011 2012 2013 2014

2.1 National Agency kick off mid-termevaluation

finalevaluation

take off sustainable

2.2.a Computer-based Testing (CBT)

Paper-based,Preparation and

Try outImplementation

Implementatio

n

Implementatio

n

Implementatio

n

Paper-based, CBTPreparation and

Try Out

Paper-based,CBT

Preparation and Try Out

Implementation

Implementation

Implementation

PreparationBenchmarking

TestPreparationand Try Out

Implementation

Implementation

2.2.b ObjectiveStructured Clinical

Examination (OSCE)

Preparation Try-out Implementation Implementation Implementation

Preparation Try out

Implementation

Implementation

Implementation

Preparation Preparation Preparation Try outImplementatio

n

2.3 National Item BankNetworking for

Assessment (NIBNA)

Preparation Establishment

Enhancementfor Pre-clinical

Formative Test

Enhancementfor ClinicalFormative

Test

Enhancement

for Progress Test

Preparation Establishment


Formative Test


Formative Test

Enhancementfor ClinicalFormative

Test

Preparation Preparation Preparation Preparation Establishment

Medicine Dentistry Nursing/Midwi



HPEQ +

Pharmacy

Nutrition

Public Health

Other health professionals related to MTKI



KPI PROJECT per YEAR



Key Performance IndicatorSuccessfull rates on the first take of National Competence Examination

HealthProfessiona

l

Baseline(2009)

Target Value

2010 2011 2012 2013 2014

Medicine71,67 % 71,67 % 73,25 % 75,50 % 79,60 % 84 %

Dental 79 % 80 % 80 % 81 % 82 % 83 %

Nursing 60 % 65 %

Midwifery 60 % 65 %



Achievement of Component 2



Objectives






Subcomponent 2.1

Lembaga Pengembangan Uji Kompetensi(LPUK)

An independent national agency that

responsible in developing assessmentstrategy, methodology, and tools toevaluate competency of graduationfrom health professional education

institution (medicine, dentist, nurse,midwife)

Function:

1. Assessment service: as resource

● E● E● E● E



Organization

Tim Task Force 1300a/E3-HPEQ/SK/08.11

Director: Iwan Dwi Prahasto

Secretary: Yulherina

Member:

Advisory board

National

Regional

Examination

Center

● Exam

inati

on Di

visio

n

● Exam

inati

on Di

visio

n

● Exec

utive

secr

etar y

● Exec

utive

secr

etar y



The Milestone LPUK

Before 2010:First phase of development

2010 – 2014:Program Acceleration and Expantion

After 2014:Sustainability Phase

Establishment of KBUKDI

& KDGI

Development of KNUKP

Variousworkshop initemdevelopment &review, standardsetting

Implementationof UKDI & UKDGI,and competenceexamination insome MTKPs

Academicpaper &variousguidelines forworkshop &implementation of examination

ResourceSharing

InternationalBenchmarking

Humanresource

development

Regional &Internationalcollaboratio

n

Recertification &assessment of professionalbehaviourResearch

grant &scientificpublicati

onResource &ICT supportforassessmentEstablishme

nt of LPUK &qualitystandard

Collaboration w/ otherHP



Subcomponent 2.1

Benchmarking to National BoardExamination for medicine, nurse & dentist

Technical assistance from Prof. Gordon Page

from Univ. British Columbia – CanadaAgreement with Ministry of Health

MoH Regulation Number1796/Menkes/Per/VIII/2011 about Registration of

health professional

Guidelines to conduct various workshopon assessment

Item development for CBT



Subcomponent 2.2

Medicine & Dentist:

Standard setting with Modified Angoff

Try out & CBT Implementation

Try out OSCE

Nurse & Midwife:

Try out CBT in November 2011



Try out & CBT

ImplementationActivity Number of CBT Center

Medicine Dentist Nurse Midwife

Try Out 1 6 1 12 (809) 13 (610)

Try Out 2 10 6

Try Out 3 12 11

Try Out 4 14 13

Try Out 21CBT 1 10



Try Out OSCE

Medicine

OSCECenter

Candidates

Try Out1

1 14

Try Out2

12 160

Dentist OSCE

Center

Candidate

s Try Out1

1 9

Try Out

2

13 274



Standard setting

StandardSetting

Medicine Dentist

Number of judges

Cut Score Number of judges

Cut Score

I 52 53 21 49

II 52 58 21 49

III 46 58

IV 48 58

V 50 58

VI 66 61

Judges: from health professions institutionsImplementation of standard setting in some institutions

um er o en er



Total= 32 CBT Center Total workstations= 2892PC/laptop

(842 : 2140)

um er o en erWorkstations



Average result of UKDI & UKDGIin 2010 – 2011

01.05

2.1

3.15

4.2

5.25

6.3

7.358.4

9.45

-0.05

0.05

0.15

0.25

0.35

0.450.55

0.65

0.75

0.85



Comparison of UKDI result basedon Accreditation

ANOVA p=0,000



Objectives






Subcomponent 2.3

Establishment of NIBNAItem development for knowledge basedassessment – CBT bank masing2 profesi

Item development for skill based assessment -OSCE

Development of IT support system for studentassessment

Technical assistance for Item Bank

Administration:Prof. Clarke Hazlet from Univ. Alberta –

Canada



Number of workshops &participants in 2011

No. Workshop Frequency Participant Institution

1 Item Review andDevelopment (CBT)

6 (4 M, 2 D)8 (4 N, 4 Mid)2 (other HP)

389 (M)45 (D)168 (N)132 (Mid)

65 (M)26 (D)168 (N)129(Mid)

2 CBT Coordinator 2 26 (M)

8 (D)

26 (M)

8 (D)3 Standard Setting 7 65 (M)

26 (D)65 (M)26 (D)

4 OSCE Examiner 3 (2 M, 1 D) 131 (M)28 (D)

65 (M)26 (D)

5 Item Review andDevelopment (OSCE) 3 (2 M, 1 D) 130 (M)53 (D) 65 (M)26 (D)

6 OSCE Coordinator 2 12 (M) 12 (M)

7 Standardized Patient 2 65 (M)26 (D)

65 (M)26 (D)

Medicine; D: Dentist; N: Nurse; Mid: Midwife; HP: Health Professions



Item Bank

0

2

4

6

8

10

12Number of MCQ Item in Banks

OSCE:Medicine: 12 stations

Dentist: 9 stations



Jumping Hurdles

SuccessfullyDeveloping adequate quantity of high quality items: achievable target

If development is collaboratively produced & organized, realistic timelines forachieving a “large-enough” bank are

- 5 yrs for Medicine (given 72 medical schools & 40,000 items req’d)

- 10 yrs in Dentistry (if 26 dental schools & 30,000 (?) items req’d)

- 4 yrs in Nursing (if 57 nursing schools & 25,000 (?) items req’d)

- 3 yrs in Midwifery (if 60 midwifery schools & 20,000 (?) items req’d)

SIPENA D l t R d



SIPENA Development RoadMap



Lesson learned

Building awareness of the role of assessment inimproving the quality of life (health) and driving inprioritizing quality

Support and Commitment from, and collaboration

among various stakeholdersMore potencies being identified

Resources sharing (human, infrastructure, items,system) efficiency & close the gap

NCE support the improvement of assessment method& learning process in institutions assessment drives

learning (standard of competence)

Assessment as a learning / education materials and

management process



Challenges & Future plan

The dynamic of the project continuity of the

program based on the road map with possiblemodification

Maintaing quality in bigger scope & quantity of

project

prioritizing & synchronize & coordinationamong Project Components & stakeholders

resources sharing among health professionals

Closing the gap of development & implementationparalel implementation of assessment development& learning response (case : OSCE & Clinicalteaching)

Customers friendly item banking & evidence based



This Project Can be & Will be

Collaboration is Project’s Mode of Operation

Since collaboration among health care educators also involves endorsement &participation of government & professional bodies in health care

Successful !



Terima Kasih

[email protected]

[email protected]

tri hanggono-output komponen 2

Documents