OBE Awareness

PM Ir. Dr. Mohd Shukry bin Abdul Majid

Universiti Malaysia Perlis

Contents

Introductory remark

OBE awareness

Programme Objectives (PEO)

Programme Outcomes (PO)

Introductory remarks

In December 2005, the Malaysian Cabinet decided to merge the National

Accreditation Board (Lembaga Akreditasi Negara, LAN) and the Quality

Assurance Division (QAD) of the MOHE

The Malaysian Qualifications Agency (MQA) created, the single quality

assurance agency in the country, whose scope now covers both the public

and private Higher Education Providers (HEP).

The MQA responsibilities are:

to implement the Malaysian Qualifications Framework (MQF) as a

reference point for Malaysian qualifications;

to develop, with the co-operation of stakeholders, standards, criteria and

instruments as a national reference for the conferment of awards;

to quality assure higher education providers and programmes;

to accredit programmes that fulfil a set of criteria and standards;

to facilitate the recognition and articulation of qualifications;

to establish and maintain the Malaysian Qualifications Register (MQR); and

to advise the Minister on any matter relating to quality assurance in

higher education.

6

New 2017 EAC Manual requirement

MQA requirement from CoPPA

9

What Is Outcome Based Engineering Education?

IT’S NOT WHAT WE TEACH,

IT’S WHAT YOU (student) LEARN

11

Outcome-Based Education

• OBE is an educational process.

• Directed/focussed at achieving certain specified outcomes in terms of individual

student learning.

• Outcomes - key things students should understand and be able to do or the

qualities they should develop.

• Both structures and curricula are designed to achieve those capabilities or qualities.

• Educational structures and curriculum are regarded as means not ends. If they

do not do the job they are rethought (Continuous Quality Improvement (CQI)).

Planning

Implementation Assessment

Corrective action

12

Outcome-Based Education

13

Say what you do

Do what you say

Prove it

Improve it

Outcome-Based Education

Outcome-Based Education

Roles of lecturers Review PEOs, POs, course structures and syllabi.

Teach the relevant engineering, maths, and other relevant subjects.

Conduct relevant tutorials and laboratory practical sessions.

Give appropriate guide on assignments and projects.

Conduct empirical measurements of POs.

Prepare the required documentation.

Obtain and maintain accreditation from MQA through Continuous Quality Improvement (CQI).

Roles of students

Know the required Programme Outcomes and Programme Objectives - make it available from first lecture.

Be more proactive in the learning process to acquire the Learning Outcomes of subjects.

Give constructive feedbacks -- Online Teaching Evaluation, Academic Advisory System, dialog sessions with Dean, buddy system, etc.

17

OBE addresses the following key questions:

• What do we want the students to have or be able to do? - planning

• How can we best help students achieve it? - delivery

• How will we know whether they students have achieved it? - assessment

• How do we close the loop for further improvement (Continuous Quality

Improvement (CQI))? - CQI

Outcome-Based Education

EAC’s Outcomes Expectation

Vision & Mission of

the Institution/Faculty

Section

3.0 EAC

Manual

Expected achievement of

graduates/alumni 4 -5 years after

graduation

Graduates abilities/competencies by the

Min. 12 Programme Outcomes Competencies

(POs)

or Section

4.0 EAC

Manual time of/upon graduation

Students abilities/

competencies after

completing

courses/subjects

Teaching & Learning Processes (Academic Curriculum –

structure, content, course outcomes, delivery,

assessments)

Programme Objectives or Early

Graduates Career Goals (PEOs)

Program

Outcomes Program

Objectives

Missions

Visions

Stakeholders

Course

Outcomes

Advisory committee

Outcome-Based Education implementation

(Closing the Loops)

Alumni Assessment

Assessment

Assessment

CQI

CQI

CQI

An

aly

sis

An

aly

sis

An

aly

sis

Quality Assurance

System JKJITU

Delivery

Programme

Committee /

Stakeholders

School Board

JITU Assessment

Analysis & CQI

MPPS

EXCO

LPU

JKIT Exam Board

University Senate

Outcome-Based Education implementation

(Closing the Loops) - μe

OBE – Delivery modes

Lecture

Guest Speaker

Delivery Modes

Face-to-

Face

Active

Cooperative/ Collaborative

Technology-Based Delivery

Experiential Method

Work-Based

Learning (WBL)

• Presentation

• Tutorial

• Seminar

• Colloquium

• Problem-Based

• Case Study

• Project-Based

• Online learning

• Teleconference

• Simulation

• Practical

• Industrial Training

• Work Attachment

• Field Trip

• Visit

• Lab

• Demonstration

• Workshop

How can we best help students achieve it? -

delivery

OBE - delivery

Effective Course Delivery

• Responsibility of Lecturers:

– carefully planned – NOT ADHOC

– teaching strategy in accordance with lesson

– incorporated new delivery strategies to address specific problem of the

students.

• Responsibility of Students:

- must be responsible for their own learning.

- define clearly student Learning Time (SLT)

Formative Assessment

Summative Assessment

Course Assessment

Programme Assessment

Assessment Tools

Direct and Indirect Assessments

How will you know what they have

achieved it? - delivery

How will you know what they have

achieved it? - Assessment

Example of direct assessments

30

Samples of individual student work Pre-test and post-test evaluations Standardized tests Performance on licensure exams Blind scored essay tests Internal or external juried review of student work Case study/problems Capstone papers, projects or presentations Project or course imbedded assessment Documented observation and analysis of student behavior/performance Externally reviewed internship or practicum Collections of work (portfolios) of individual students Activity logs Performances Interviews (including videotaped)

Examples of indirect assessments

31

Questionnaires and Surveys

Students

Graduating Seniors

Alumni

Employers

Syllabus and curriculum analysis

Transcript analysis

Student support

OBE – Problems in the house

OBE – Problems in the house

Do you disclose the CO at the beginning of lecture?

No monitoring – CW marked at the end of semester

No good planning in the delivery

Assessment did not help the student achieving CO

No feedback

Mismatch assessments

Ineffective lectures/one way communication

OBE – Problems in the house

Immature assessment

suitability of the taxonomy level of the assessment

lesser assessments results “covers up” more significant

assessments

Only grading correct answer rather than rewarding

understanding

Moderation to ensure attainment – suppressing the correct

CQI

EAC’s Outcomes Expectation

Vision & Mission of

the Institution/Faculty

Section

3.0 EAC

Manual

Expected achievement of

graduates/alumni 4 -5 years after

graduation

Graduates abilities/competencies by the

Min. 12 Programme Outcomes Competencies

(POs)

or Section

4.0 EAC

Manual time of/upon graduation

Students abilities/

competencies after

completing

courses/subjects

Teaching & Learning Processes (Academic Curriculum –

structure, content, course outcomes, delivery,

assessments)

Programme Objectives or Early

Graduates Career Goals (PEOs)

Programme Objectives

Programme Objectives (PEO) are specific goals consistent with the mission and vision of the IHL, are responsive to the expressed interest of programme stakeholders, and describe the expected achievements of graduates in their career and professional life a few years after graduation.

Programme objectives are specific statements on what a learner is expected to learn to achieve the programme aims – CoPPA.

What do we expect our graduate to accomplish in broader society as a result of Programme’s education, 3-5 years after graduation.

Broad statements – long term target (3-5 years after graduation)

Consistent with vision & mission of IHL

Number of statements should be limited and

manageable

Should not be simply restatement of programme

outcomes

Reviewed, revised & updated continually

Has clear link to the programme outcomes &

curriculum design

Characteristics of Good PEO Statement

Characteristics of Good PEO Statement

42

Programme Objectives

43

Programme Objectives

44

Programme Objectives

PEO Statement

PEO1 Become electrical engineers who are competent, innovative

and productive in addressing customer needs

PEO2 Grow professionally with proficient soft skills

PEO3 Demonstrate high standard of ethical conduct, positive

attitude and societal responsibilities

PEO for Electrical Engineering

PEO for Mechanical Engineering

Programme

PEO Statement

PEO1 Graduates who are competent, responsible and practice

professionalism in the global context

PEO2 Graduates who are knowledgeable and capable to apply the

evolving technology in mechanical engineering field

PEO for Mechanical Engineering

Programme

PEO Statement

PEO1 Demonstrate leadership in professional activities such as

research, experimental studies, and industrial projects

PEO2 Be capable of working effectively in cross-functional teams,

communicating effectively and participating in the practice of

mechanical engineering

PEO3 Contribute to professional societies and demonstrate ethical

conduct

PEO for Mechanical Engineering

Programme

PEO Statement

PEO1 Graduates who are competent and adaptable in multi industries by

practicing the knowledge in mechanical engineering that relevant to

professional engineering practice.

PEO2 Graduates who will pursue study in graduate works and others

professional education.

PEO3 Graduates who will be leader at note-worthy level and provide

solutions that benefited the respective organization, society and

nation.

On-going process, covers all cohorts which have graduated from the Programme.

Assessment are performed after each cohort been in the industry for 3-5 years.

The assessment of the attainments of the PEOs utilized the performance indicators as

given earlier.

Types of survey:

Alumni Survey

Employer Survey

Methods of survey:

Printed forms

Online forms

Telephone interviews

49

PEO Attainment Assessment

Performance Criteria, PC

PEO needs performance criteria or indicator as it is general

Criteria should be specific and measurable

Target should be achievable

PEO- PI and target

Programme Objectives (PO)

53

Doctor of Philosophy (PhD)

PO1

1.Graduates who are skilled and have extended knowledge that is profound in areas of specific expertise.

PO2

1.Graduates who think critically and capable of analyzing the results of research in their purpose field of interest.

PO3

Graduates who are highly knowledgeable and capable of knowledge and innovation toward new products and services at national/international level.

Programme Objectives (PO)

54

Master of Science (MSc)

PO1

1.Graduates who are skilled and have extended knowledge in areas of specific expertise.

PO2

1.Graduates who are capable of conducting research in their purpose field of interest.

PO3

Graduates who are capable of contributing knowledge and innovation toward new products and services at national/international level.

PEO - The big picture

55

Points panels look for;

Is the PEO statement reflective of the programme expectation of the

graduates 3 –5 years after graduation?

Are they well define, measureable & achievable?

Where are PEOs published and publicized?

Are the PEOs supportive of the mission & vision of the IHL?

Is there a clear link between the PEOs and POs

Is there an established process for assessing and evaluating achievement

of PEOs? Please explain how the IHL do this!

Explain how the measurement is done and what do the IHL do with the

outcome of assessment and used in the CQI of the programme?

Is there a high degree involvement of stakeholder in the process? Please

deliberate

Learning Outcomes (LO)

59

Doctor of Philosophy (PhD)

Show high degree of scholarship in the field of research

Contribute to original research that broadens the boundaries of knowledge through thesis writing or dissertation

Conduct critical analysis and evaluate along with synthesize of new ideas

Plan and perform independent research professionally, ethically and responsibly and able to lead / supervise research projects

Report research findings to peers at a level appropriate for publication in the international stage

Recognize the need for continuing professional development

Learning Outcomes (LO)

60

Master of Science (MSc)

Show advanced and continuous knowledge and has the ability to continue to develop or use this knowledge in new situations or in the context of a multidisciplinary

Analyze and evaluate critically problems in disciplines in situations with limited information and to provide solutions through the application of appropriate instruments and techniques

Assess existing information and research evidence and apply it in the context of engineering

Plan and implement research tasks professionally, ethically and responsibly

Report technical findings in writing and verbal forms

Recognize the need for continuing professional development

PO Attainment for PhD

61 PD: Proposal Defense

PR: Progress Report

Pub: Publication

LO vs Assessment Components

Lear

ning

Out

com

es (L

O)

Assessment

PD

PR

Pub

Viva

/The

sis

1 Show high degree of scholarship in the field of research /

2 Contribute to original research that broadens the boundaries of knowledge through thesis writing or dissertation

/ /

3 Conduct critical analysis and evaluate along with synthesize of new ideas

/

4 Plan and perform independent research professionally, ethically and responsibly and able to lead / supervise research projects

/ /

5 Report research findings to peers at a level appropriate for publication in the international stage

/

6 Recognize the need for continuing professional development

/ /

PO Attainment for MSc

62 PD: Proposal Defense

PR: Progress Report

Pub: Publication

Lear

nin

g O

utc

om

es

(LO

)

Assessment

PD

PR

Pu

b

Viv

a /

Thes

is

1 Show advanced and continuous knowledge and has the ability to

continue to develop or use this knowledge in new situations or in the

context of a multidisciplinary

/ /

2 Analyse and evaluate critically problems in disciplines in situations with

limited information and to provide solutions through the application of

appropriate instruments and techniques

/ /

3 Assess existing information and research evidence and apply it in the

context of engineering / /

4 Plan and implement research tasks professionally, ethically and

responsibly / /

5 Report technical findings in writing and verbal forms / /

6 Recognize the need for continuing professional development / /

LO vs Assessment Components

Programme Outcomes (PO)

Statements that describe what students are

expected to know and able to do/perform or attain

upon graduation

These relate to the skills, knowledge, and

behaviours that student acquire through the

programme

Cont.

Directly map to relevant courses and support the

PEOs

Explicitly map to the COs of the related courses

Performance Indicators or targets (PI)

Process put in place to measure the PO

Measurement must be realistic and reflective of the

performance of the class

An established PO model

A CQI process instituted for PO at the programme

level

Process flow of PO review in UniMAP

66

Quality Assurance

System JKJITU

Delivery

Programme

Committee /

Stakeholders

School Board

JITU Assessment

Analysis & CQI

MPPS

EXCO

LPU

JKIT Exam Board

University Senate

CQI process for non-attainment of PO

68

PO Assessment

Programme Outcomes Report (PER) for cohort/student

PO attainment evaluation

>50%

Non-attainment analysis

Finish

System/process

No

No

Yes

Yes

Finishing school/Value added

program

Finish

New HEA01 /

HEA03

START

Student

Characteristics of Good PO Statement

Course outcomes (CO)

Learning outcomes are statements of what a learner is expected

to know, understand and/or be able to demonstrate after

completion of a process of learning.

These statements are course specific and relate to the course

content

COs are linked to specific POs and thus serve as direct

evidence of the achievement of PO

COs are assessed and evaluated through various measurement

tools

CQI for PO

Points EAC panels look for;

Is the mapping between courses and POs realistic?

Is there any issue with constructive alignment i.e. outcomes-assessment-activities?

Is there a process instituted at the programme level for the measurement of COs and POs? Please deliberate!

Is the programme measuring all the 12 ETAC POs and do the results demonstrate the achievement of the said PO?

What is their PO assessment model? Please explain clearly how they do this!

Is there Performance Indicators set for the outcome assessment? Is the target realistic?

Explain how do they use the outcome assessment results in the CQI at the course level and programme level

Cont.

How do the final PO achievement of the graduating

students are determine?

Are the students informed of their PO results?

Who are the stakeholders involve in the PO process and

assessment –please deliberate! … for example, industrial

partner involved in the assessment of specific POs etc.

Is the IAP involve in any part of the POs i.e. at the

formulation, measurement, assessment or CQI stage?

Is there evidence of strong participation in the delivery of

the OBE curriculum in the programme from the academic

staff