Information Management in Higher Education...

of 65/65
Der folgende Text wird über DuEPublico, den Dokumenten- und Publikationsserver der Universität Duisburg-Essen, zur Verfügung gestellt. Diese auf DuEPublico veröffentlichte Version der E-Publikation kann von einer eventuell ebenfalls veröffentlichten Verlagsversion abweichen. Randhahn, Solveig: Information Management in Higher Education Institutions - Training on Internal Quality Assurance Series | Module 4 In: Training on Internal Quality Assurance Series DOI: http://dx.doi.org/10.17185/duepublico/43225 URN: urn:nbn:de:hbz:464-20170215-102130-9 Link: http://duepublico.uni-duisburg-essen.de/servlets/DocumentServlet?id=43225 Lizenz: Dieses Werk kann unter einer Creative Commons Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International Lizenz genutzt werden.
  • date post

    14-Sep-2018
  • Category

    Documents

  • view

    216
  • download

    0

Embed Size (px)

Transcript of Information Management in Higher Education...

  • Der folgende Text wird ber DuEPublico, den Dokumenten- und Publikationsserver der UniversittDuisburg-Essen, zur Verfgung gestellt.

    Diese auf DuEPublico verffentlichte Version der E-Publikation kann von einer eventuell ebenfallsverffentlichten Verlagsversion abweichen.

    Randhahn, Solveig:

    Information Management in Higher Education Institutions - Training on Internal QualityAssurance Series | Module 4

    In: Training on Internal Quality Assurance Series

    DOI: http://dx.doi.org/10.17185/duepublico/43225

    URN: urn:nbn:de:hbz:464-20170215-102130-9

    Link: http://duepublico.uni-duisburg-essen.de/servlets/DocumentServlet?id=43225

    Lizenz:

    Dieses Werk kann unter einer Creative Commons Namensnennung - Nicht kommerziell - KeineBearbeitungen 4.0 International Lizenz genutzt werden.

    http://dx.doi.org/10.17185/duepublico/43225http://nbn-resolving.org/urn:nbn:de:hbz:464-20170215-102130-9http://duepublico.uni-duisburg-essen.de/servlets/DocumentServlet?id=43225http://creativecommons.org/licenses/by-nc-nd/4.0/http://creativecommons.org/licenses/by-nc-nd/4.0/http://creativecommons.org/licenses/by-nc-nd/4.0/

  • Information Management in Higher Education Institutions

    Solveig Randhahn

    Training on Internal Quality Assurance Series | Module 4Solveig Randhahn and Frank Niedermeier (Eds.)

  • With financial support from the

    Imprint

    This e-publication is part of the Training on Internal Quality Assurance Series which is also published as paperback (ISBN: 978-3-7345-7691-1) and distribut-ed in book shops worldwide. More information is available at http://www.trainiqa.org

    Author: Solveig Randhahn

    Editors: Solveig Randhahn and Frank Niedermeier

    Reviewers: Karl-Heinz Stammen, Frank Niedermeier

    Edition: First edition

    Layout: Nikolaj Sokolowski, Randi Ramme

    Publisher: DuEPublico, Duisburg/Essen, Germany

    DOI: 10.17185/duepublico/43225

    Copyright 2017 Solveig Randhahn

    This book is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). https://creativecommons.org/licenses/by-nc-nd/4.0/

    Please cite the use of our course book series in presentations, trainings, papers etc. according to scientific standards. You can cite this book as:

    Randhahn, S. (2017). Information Management in Higher Education Institutions. Module 4. In Randhahn, S. & Niedermeier, F. (Eds.) Training on Internal Quality Assurance Series. Duisburg/Essen: DuEPublico. Retrieved from: http://dx.doi.org/10.17185/duepublico/43225

    http://www.trainiqa.orghttps://creativecommons.org/licenses/by-nc-nd/4.0/

  • Acknowledgment

    Our modules and books have been prepared and written in a joint effort of the University of Duisburg-Essen

    and the University of Potsdam under the DIES (Dialogue on Innovative Higher Education Strategies) Programme

    conducted by the German Academic Exchange Service (DAAD) and the German Rectors Conference (HRK) with

    funds from the Federal Ministry for Economic Cooperation and Development (BMZ). We take this opportunity

    to thank the DIES Programme and all the partners from Africa, Europe and Southeast Asia who were involved

    in the development process and express our deepest gratitude for the received support, without which the

    modules and course books would not have been possible to realise.

    We want to further express our sincere gratitude for the most valuable support and contributions from the

    involved partners

    Autorit Nationale dAssurance Qualit de lEnseignement Suprieur (ANAQ-Sup), Sngal,

    ASEAN Quality Assurance Network (AQAN),

    ASEAN University Network (AUN),

    Association of African Universities (AAU),

    Conseil Africain et Malgache pour lEnseignement Suprieur (CAMES),

    European Association for Quality Assurance in Higher Education (ENQA),

    National Accreditation Board (NAB), Ghana,

    National Council for Tertiary Education (NCTE), Ghana,

    National Universities Commission (NUC), Nigeria,

    Southeast Asian Ministers of Education Organization Regional Centre for Higher Education and Develop-

    ment (SEAMEO RIHED),

    United Nations Educational, Scientific and Culutral Organization (UNESCO),

    University of Professional Studies Accra (UPSA), Ghana

    and especially from Prof. Dr. Shahrir Abdullah, Richard Adjei, Prof. Dr. Goski Bortiorkor Alabi, Prof. Dr. Bassey

    Antia, Prof. Dr. Arnulfo Azcarraga, Gudrun Chazotte, Assoc. Prof. Dr. Tan Kay Chuan, Kwame Dattey, Prof. Dr.

    Ong Duu Sheng, Prof. Zita Mohd. Fahmi, Mae Fastner, Assoc. Prof. Dr. Nantana Gajaseni, Robina Geupel, Josep

    Grifoll, Juliane Hauschulz, Dr. Pascal Hoba, Dato Syed Hussein, Benjamin Jung, Prof. Abdel Karim Koumare,

    Dr. Vipat Kuruchittham, Prof. Dr. Chiedu Mafiana, Prof. Dr. Duwiejua Mahama, Barbara Michalk, Prof. Dr. Le

    Quang Minh, Nguyen My Ngoc, Johnson Ong Chee Bin, Concepcion V. Pijano, Prof. Dr. Philipp Pohlenz, Sonja

    Pohlmann, Dr. Suleiman Ramon-Yusuf, Dr. Sylvia Ruschin, Dr. Chantavit Sujatanond, Dr. Oliver Vettori and Marc

    Wilde.

    The authors

  • Dr. Solveig RandhahnFaculty of Social Sciences

    University of Duisburg-Essen, Germany

    [email protected]

    https://www.uni-due.de/gesellschaftswissenschaften/

    Dr. Solveig Randhahn is Managing Director of the Faculty of

    Social Sciences at the University of Duisburg-Essen (UDE) in

    Germany. She studied Political Science, Spanish Philology and

    Economic Policies at the University of Mnster. She received

    her PhD in Political Science, doing research on education and

    social policy in Germany. Furthermore, she is a certified expert

    in Education and Science Management.

    Dr. Solveig Randhahn shows a wide range of work experience in

    the field of quality management at higher education institutions.

    She was responsible for the Service and Information Centre at

    the Institute of Political Science and worked at the Department of

    Quality Development in Teaching and Learning at the University

    of Mnster. Afterwards, she was employed at the University

    of Applied Sciences in Aachen, coordinating the accreditation

    processes of the University and advising the University leadership

    in terms of higher educationpolicies in teaching and learning.

    In January 2014, Dr. Randhahn took over the responsibility as

    manager of the TrainIQA project (Training on Internal Quality

    Assurance in West Africa), coordinated by the Centre for Higher

    Education Development and Quality Enhancement (CHEDQE) at

    UDE. The project aimed at developing capacity in the field of

    internal quality assurance (IQA) in higher education institutions

    by providing hands-on workshops for quality assurance officers

    from higher education institutions in the West African region.

    In March 2016, Dr. Randhahn switched to the Faculty of Social

    Sciences as Managing Director. In addition, she was elected as

    Vice-Dean for teaching and learning at the faculty.

    https://www.uni-due.de/gesellschaftswissenschaften/

  • List of Abbreviations

    AAA

    BA

    BSC

    CB

    CEUS

    CHE

    EUNIS

    HEI

    IUCEA

    MA

    MSRE

    PDCA

    PhD

    QA

    Annual Academic Achievements

    Bachelor

    Balanced Scorecard

    Course Book

    Computerbasiertes Entscheidungsuntersttzungssystem fr die Hoch-

    schulen in Bayern (computer-based management tool for the institu-

    tions of higher education in Bavaria, Germany)

    Centre for Higher Education

    European University Information Systems

    Higher Education Institutions

    Inter-University Council for East AfricaUniversity of Duisburg-Essen

    Master

    Federal Ministry of Science, Research and Economy

    Plan-Do-Check-Act or Plan-Do-Check-Adjust

    Doctor of Philosophy

    Quality Assurance

  • 8

    Table of ContentsIntroduction to the Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    Introduction to Information Management at Higher Education Insitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

    1 Introduction to Information Management at Higher Education Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    1.1 Why Should Higher Education Institutions Collect Data? . . . . . . . . . . . . . . . . . . 13

    1.2 Characteristics of an Information Management System . . . . . . . . . . . . . . . . . . . . 16

    Translation of Higher Education Objectives Into Numbers . . . . . . . . 30

    2 Translation of Higher Education Objectives Into Numbers: Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . . . . . . . 31

    2.1 Meaning and Function of Quantitative and Qualitative Indicators . . . . . . . . . . . 31

    2.2 Determination and Operationalisation of Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    2.3 Using Indicators Key Aspects to be Considered . . . . . . . . . . . . . . . . . . . . . . . . . 34

    2.4 Challenges of Using Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . 39

    Reporting: Presentation and Communication of Data and Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    3 Reporting: Presentation and Communication of Data and Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    3.1 Definition of Reporting Objectives for Different Target Groups . . . . . . . . . . . . . . 45

    3.2 Content of Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    3.3 Organisational Conditions for Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

  • 9

    Elaborated Information Systems - Examples for Data Sharing . . . . 52

    4 Elaborated Information Systems Examples for Data Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

    4.1 Case Study of the ETH Zurich: Annual Academic Achievements Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

    4.2 Case Study of the University of Vienna: Course Controlling . . . . . . . . . . . . . . . . 55

    4.3 Unidata Facts and Figures at the Push of a Button A Case Study from Austria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

    References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

    List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

    List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

  • 10

    Preface

    Introduction to the Module

    Prerequisites for the Module Learnershaveabasicunderstandingandknowledgeofdifferentqualitymanagementapproaches(e.g.pro-

    cess-andevaluation-based)inthehighereducationcontext(seecoursematerialModule1),

    theyareabletousethePDCA-cycleasasystematicapproachtomanagingquality(seecoursematerial

    Modules1and2),

    theyhavebasic theoreticalknowledgeof thenewpublicmanagementapproachand its challenges for

    highereducationinstitutions(HEI)(seecoursematerialModule1).

    Intentions of the Module Establishingsystematicqualityassurancestructuresathighereducation institutionsrequiresawiderange

    ofdecision-makingprocessesbydifferentstakeholders.Toimplementthederivingmeasuresandactivities

    effectivelyandefficientlyandaccordingtothequalityobjectivesofthehighereducationinstitution,dataand

    informationandtheirappropriatecirculationarenecessary.

    Thismodulegivesanintroductiontothebasicdiscussionaboutinformationmanagementsystemsathigher

    educationinstitutions.Itanalysesthequestionwhyuniversitiescollectdataanddiscusseskeycharacteristics

    of informationmanagementsystemsatuniversities.Basedonthis, thecoursebook introducestheuseof

    quantitativeandqualitativeindicatorsasameansofmeasuringandassessingobjectives.Itexplainshowto

    determineandoperationaliseindicators,howtocriticallyreflectonthemandhowtousetheminaresponsi-

    bleandappropriateway.ItpresentstheBalancedScorecardasamethodicalmanagementapproachtodeal

    withindicatorsathighereducationinstitutions.

    Furthermore,thecoursebookgivesan introductiononhowtoestablishadata-basedreportingsystemat

    highereducationinstitutions.Itdealswiththeobjectivesofdifferentstakeholdergroupsandassesseshowto

    considertheseappropriatelyinareportingsystem.Itgivesaninsightonthekeyconditionstobeconsidered

    whengeneratingreports.

    Finally,thecoursebookpresentsvariousexamplesofhowhighereducationinstitutionsdealwithinformation

    byestablishingdifferent(technical)structuresandproceduresofcampus-widedatasharingandreporting..

  • 11

    dealwith informationthat is relevant forplanningandcontrollingwithregardtoqualitydevelopment/

    assurance/management,

    developinternaldataandinformationchannels,consideringtherespectivetechnicalandstructuralframe-

    workofhighereducationinstitutions,

    defineandoperationalisequantitativeandqualitativeindicatorsathighereducationinstitutions,

    recogniseandconsideropportunitiesandlimitationsofquantitativeandqualitativeindicatorsasmeasures

    forqualityassuranceofprocessesathighereducationinstitutions,

    developandmanagereportingsystemsfordifferenttargetgroupsbasedonatransparentsetofinternal/

    externalcriteria.

    On successful completion of the module, you should be able to

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    12

    1 IntroductiontoInformationManagementat HigherEducationInstitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    1.1 WhyShouldHigherEducationInstitutionsCollectData? . . . . . . . . . . . . . . . . . . 13

    1.2 CharacteristicsofanInformationManagementSystem . . . . . . . . . . . . . . . . . . . . 16

    identifythereasonsforhighereducationinstitutionstocollectdata,

    recogniseanddifferentiatethelinkagesbetweeninformationmanagementandcontrollingprocessessuch

    asplanning,managingormonitoring,

    identifyelementarycharacteristicsofaninformationmanagementsystemandtodeducesystematicsteps

    todealwithinformationatHEI(e.g.gatheringandacquisitionofinformationneeds,processingandstorage

    ofinformationaswellascommunicationchannelsofinformation).

    On successful completion of this chapter, you should be able to

    Chapter 1

    Introduction to Information Management at Higher Education Insitutions

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    13

    1 Introduction to Information Management at Higher Education Institutions

    1.1 Why Should Higher Education Institutions Collect Data?

    Establishingsystematicinstitutionalqualityassurancestructuresrequiresabroadvarietyofinformationthat

    isfundamentaltoenabledecision-making,communicationandorganisationalprocessesbetweendifferent

    stakeholdersandtherealisationofactivities.

    Informationcanbedefinedaspurposefuldatathat isrelatedtoaproblemandthat isusedtoachievean

    objective(Wittmann1980).Wecantalkaboutknowledgewhenpeoplestarttoputinformationintoamean-

    ingfulcontext(Gladen2003,2).

    InformationisnecessaryforallorganisationalconcernsandobjectivesofaHEI:foreasingandoptimisingdeci-

    sion-makingprocesses,forplanninganddevelopingrealisticsettings,forreportingandqualitydevelopment,

    andwithitenhancingtheinstitutionalefficiencyandeffectiveness(Saupe1981).Highereducationresearcher

    J.FrederickVolkweinsystemisesthesestrategicandoperativeobjectivesintofivefundamentalconcernsof

    highereducationinstitutions(Volkwein1999):

    1. Expensesforhighereducation(shortageoffinancialfunding)

    2. Requiringanefficientmanagementandincreasingproductivityatthesametime

    3. Effectivenessandsurplusvalueofhighereducationinstitutions(competitionandrighttoexistwithout

    thenecessitytoproduceoutputwithregardtocontents)

    4. Accesstohighereducationinstitutions(increasingnumberofstudentsasajustificationforadditional

    funding)

    5. Reporting

    These fundamental concerns go alongwith various, constantly changingwhile simultaneously increasing

    demandsforinformation.Thequestionishowhighereducationinstitutionscanrecognise,determine,pro-

    ceedand,finally,copewiththeseinformationdemandsefficientlyandeffectivelyinthelightofavailablestaff,

    materialandtechnical resources.Forexample, todeterminetheavailablecapacitiesofyour institutionto

    establishanotherstudyprogramme,youhavetoconsiderandcalculatetheplannednumberofstudents,the

    numberoflecturerswhoareavailable(intermsofworkinghours),aswellastheresultingcostsforstaffand

    infrastructure.

    Inorder tobeable todealwith such informationdemands,highereducation institutionshave started to

    establishintegrateddata-basedinformationsystems.Thesearebaseduponexistingeconomicalapproaches

    Information aspurpose- fulknowledge (Wittmann 1980)

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    14

    forbusinessstrategiesandmanagementconcepts.Usingup-to-datedataand informationtechnologiesat

    highereducationinstitutionsshouldcontributetoeffectiveandefficientprocessesinthehighereducation

    organisation.

    Inthiscontext,data canbedefinedasasetofqualitativeand/orquantitativevariablesthatbecomeinfor-

    mationbyinterpretation.Dataarearesultofmeasurementsandcanbevisualisedbyusingtables,graphsor

    images.Hence,datacanbeunderstoodasanabstractconceptfromwhichinformationandthenknowledge

    arederived(BostonUniversity2015;DWBI2014;seealsoModule2).

    Methodicalinformationmanagementservesaccountabilityandreportingpurposesintheinternalandexter-

    nalcontextofhighereducation.Itcreatesperformanceandcosttransparencyandthereforeprovidesacen-

    tralcontributionforqualityassuranceinresearch,teachingandsupportingservices:awell-establishedinfor-

    mationsystemserves the formulationof institutionalobjectivesandthereforethe facilitationandoptimi-

    sationofdecision-makingprocesses fora sustainable strategicplanning inhighereducation (Saupe1981;

    Kpper,Friedl,Hofmann,Hofmann&Pedell2013).

    An information system can be understood as a coordinated arrangement of staff, organisational

    and technical elements that provides decision-makers with purposeful knowledge for their task

    fulfilment.

    (Eberhardt, 67 in Frese 1992)

    Thekeypurposesofinformationmanagementincludeacloselinkagetomanagerialaccountingprocessesat

    HEI.

    Asaprimaltaskofmanagerialaccounting,wecanconsidertheoverallcoordinationofthemanagementsys-

    temofahighereducationinstitution:

    Management must deal with the dynamics of change and provide coordination for the overall

    system.

    (Kast & Rosenzweig 1974, 620 in Horvth 2011, 8)

    AccordingtoHorvththemanagementsystemconsistsoffivesubsystems:planning,accounting,information

    supply,organisationandhumanresourcemanagement(Hrvath2011,8;Kpperetal.2013,636).Concern-

    ingtheinformationsupply,managerialaccountinghastocoordinateandaligntheaforementionedsubsys-

    temswithregardtotheinformationneedsofdecision-makers.Ontheonehand,thisincludesthecoordina-

    tionwithintheinformationsystemthecollectionofnecessarydata,itssystematisation,storageand,finally,

    itsallocation.Ontheotherhand,thisincludesthetransmissionofdatatotheaforementionedsubsystemsof

    themanagementsystembysuitablereportingsystems.

    Datacreate suitable

    information fordecision-

    making processes

    Linkage between

    information management

    andmanagerial accounting

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    15

    Figure 1 Layer model for higher education institutions (Tropp 2002, 2)

    Thedesignofinformationsystemsisorientedtowardstworeferencelevels.Theverticallevelreferstosuch

    levelsathighereducationinstitutions,wheredecisionsaremadeandtasksarecarriedout,i.e.thetopman-

    agement,faculties,institutesandchairs.Thehorizontallevelreferstothecoreprocessesofhighereducation,

    i.e. research, teachingand services. These includevarious informationneeds thatgoalongwithdifferent

    requirementsregardingthewayofsystematisationandallocationofinformation.Dependingonthelevelof

    centralisedandde-centraliseddecision-makingprocessesbetweenthetopmanagement,faculties,institutes

    andchairs,multi-dimensionalinformationsystemsareneeded(Kpperetal.2013,636).

    Theincreasingcomplexityanddiversityofinformationleadtoverydifferentscopesofperformanceofthese

    informationsystemsamonghighereducationinstitutions.Coreprocessesofthesocalledstudentlifecycle1,

    thatarefrequentlymanagedthroughprofessionalinformationtechnologies,aresuchasthefollowing:

    application,assessmentandadmissionprocesses

    studentadministration

    planningandmanagementoflectures(universitywidecourseschedule,generalandindividualcourse

    schemes,registrationandderegistrationofstudentsfromcourses/exams)

    managementoflecturehallbooking

    examinationmanagement(e.g.examregistrationandderegistration,transcriptofrecords,recognition,

    archivalstorageoffinalexamination)

    managementoforganisationaldata(buildingandlecturehallplans,e-mailandphoneindex)

    1Thestudent-life-cycleincludesallrelevantactivitiesandfieldsforstudents,lecturersandadministratorsthathavetobecon- sideredduringtheacademiceducationprocess:e.g.application-->admission-->teachingandlearning-->assessment--> graduation-->alumni.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    16

    Higher education institutions have started to integrate thiswidely ramified IT-landscape in complex data

    warehousesystems.

    Comingbacktothequalitymanager,wecanaskwhichareasofsuchacomplexdatasystemarerelevantto

    her/him.Focussingonteachingandlearning,wecanthinkofaprofessionaldatamanagementofprocesses

    suchasinternalandexternalevaluationsonfacultylevelorthehighereducationinstitutioningeneral,tracers

    studies,oralsostaffdevelopmentinteaching.

    Further Reading

    Taylor,J.(2014).Informingordistracting?Guidingordriving?Theuseofperformanceindicatorsin

    highereducation.InMenon,M.,Terkla,D.,Gibbs,P.(Ed.),Using data to improve higher education.

    Research, policy and practice.Rotterdam:SensePublishers.

    HigherEducationFundingCouncilforEngland(HEFCE)(2011).Performance indicators in higher ed-

    ucation. First report of the performance indicators steering group (PISG).London:HEFCE.

    Balasubramanian,K.(2009).ICTs for higher education. Background paper from the commonwealth

    of learning.Paris:UNESCO;WorldConferenceonHigherEducation;CommonwealthofLearning.

    1.2 Characteristics of an Information Management System

    Whyshouldqualitymanagerscareaboutinformationmanagement?Basically,qualitymanagershaveacon-

    sultativefunctionwithregardtodifferentdecision-makingprocessesathighereducationinstitutions,beiton

    managementlevel,onorganisational/administrationleveloronfacultylevel.Therefore,theyneedtobeable

    togatherinformationrequirementscorrectlyandanalyseandevaluatethecollecteddataandinformation

    accurately.

    Examplesoftargetsinaninformationmanagementsystem,forwhichqualitymanagerscanplayakeysup-

    portingrolecanbethefollowing:

    Definition of Data Warehouse

    Adatawarehouseisacopyoftransactiondataspecificallystructuredforqueryingandreporting.

    Source: (Kimball 2002)

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    17

    Defineandestimateinformationneedsrequiredforcertaindecision-makingprocesses.

    Prepareunderstandableandinterpretabledatafortherespectivetargetgroupsandavoidcontradictions.

    Interfacefunctionwithregardtoinformationdistribution,inordertohelptoclosecommunicationand

    informationgapsamongsendersandaddressees.Thatmeans,theycanexplainandclarifywhichinfor-

    mationisavailableforwhichissues,orwhoneedswhichpartoftheexistingdataandinformation.

    Supportforreading,analysingandinterpretingdatamaterial,consideringtherespectiveparticularcon-

    text.

    Contributetodevelopingmoretransparencyabouthowtheinformationflowsofahighereducationinsti-

    tutionworkaccordingtodefinedqualitycriteria.

    Insomehighereducationinstitutionsthesetargetscanbecloselyrelatedtomanagerialaccounting.2Toavoid

    overlappingactivitiesbutachieveaneffectivetargetallocation,youshoulddefineandcoordinatetherespec-

    tiveresponsibilitiesbetweenaqualitymanagerandaunitformanagerialaccountingclearly.

    Takingthisintoconsideration,thewholefieldofinformationmanagementcontainsenoughquestionstobe

    discussedinapropertrainingcourse.Thisiswhyinthiscoursebookwehavetolimitourfocusonsomepar-

    ticularaspects.Inshort,wewillfocusonthelinkagesbetweeninformationandqualitymanagementandthe

    roleofqualitymanagers.

    Thecoursebookgivesanintroductiontomanagementrelevantdataandinformationwhichahigheredu-

    cationinstitutionneedsforimproving,assuringandmanagingqualityinthecoreprocessesofteachingand

    learning,researchandservices.Therefore,itgivesanoverviewonthekeycharacteristicsofinformationman-

    agementsystemsanddiscussesthecriteriathatarenecessarytodevelopasystematiccollection,analysisand

    interpretationofdataandinformationaccordingtotheneedsandrequirementsofspecifictargetgroups.3

    Basedonthis,yougettoknowthemostimportantessentialstoassessandjudgeinhowfarstrategicand

    operativeobjectivesofqualityassurancehavebeenreached.Youwill learnaboutthechallengesofdefin-

    ingquantitativeandqualitative(keyperformance)indicators(Chapter2.1),howtocollectandanalysethem

    (Chapter2.2),aswellashowtodealwithresistanceagainstdataandinformationandtoachieveacceptance

    (Chapter2.3and2 .4).

    AccordingtoHorvthamethodicalinformationmanagementsystemcanbestructuredintothethreefollow-

    ingphases(Hrvath2011,308etseq.)

    I. IdentifyinginformationneedsandgatheringrawmaterialatHEI

    II. Datacollection,processingandanalysis

    III.Datadissemination(workflowsbetweendisseminatorandreceiver)

    2ForfurtherinformationonmanagerialaccountingandtherelationtoinformationanalysisseeDemski(198,2008). 3 MoreinformationonthisissuecanbefoundinCB2aswell.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    18

    I. Identifying information requirements and gathering raw material at Higher Education Institutions

    Tobeabletogather,classify,processandreportdataandinformationinaninformationsystem,firstofallyou

    havetofindoutabouttherespectiveinformationrequirements.Decision-makersathighereducationinstitu-

    tionshavedifferentinformationneedsaccordingtotheirrespectivestrategicobjectivesandtargets(seeTable

    1).Theseinformationneedshavetobedefinedclearlyandunambiguouslytobeabletodeducesystematic

    andeffectivedatacollectionanddistribution.

    Information requirements can be defined as the type, amount and quality of informationwhich a deci-

    sion-makerneedstofulfilher/histargets4(Koreimann1976,6;Gladen2003,4).

    Wecandifferbetweenobjectiveandsubjectiveinformationneeds.Objectiveinformationrequirementsrefer

    totheamountofinformationwhichissetinafactualcontexttosolveaproblem.Subjectiveneedsarethe

    informationwhichadecision-makerconsiderstoberelevantforher/histargets(Kpper2013,218).

    Basedonthis,aconcreteinformationdemandgenerallyincludesbothsubjectiveandobjectiveinformation

    requirements.Veryoftendecision-makersarenotsufficientlyawareoftheirsubjectiveinformationneedsor

    cannotformulatethemappropriately.Itmayalsohappenthattheyevenwanttohidetheirrealinformation

    requirements(Nusselein2002,3).

    Figure 2 Gathering information based on needs, supply and demand (translated based on Picot & Frank 1988, 608 in Hrvath 2011, 311)

    4OwntranslationfromGermanintoEnglish.

    Objective andsubjective

    information requirements

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    19

    Thefollowingtableillustratespossibleinformationneedsofdifferentstakeholders,referringtostructuralcon-

    ditions,resourcesorprocessesinteachingandlearningwhichcancomeupwhenestablishingqualityassur-

    ancestructuresatanhighereducationinstitution.

    Subject-matter Examples for information sources Examples for information requirements

    Structuralframeofresearchandteaching

    (National)lawonhighereduca-tion

    StrategicplansofaHEI

    Strategicplansoffaculties

    HEIconstitutionandregulations

    Examinationregulations

    Regulationsfordoctoral

    degreesandhabilitation

    IstherearegulatoryobligationtoestablishaQA-unit?Ifso,whichrequirementshavetobefulfilled?

    WhichobjectivesshallbeachievedwiththeQA-unit?(E.g.annualevaluationofstudypro-grammes;establishmentandcoordinationofqualitycyclesinteachingandlearning)

    Whichinformationhastobedocumentedinanexaminationregulationtocomplywithinternal/externalqualitystandards?

    ResourcesofaHEI(staff,facilities)

    Dataonavailableresourcesandcashflows

    Staffingperprofessor Third-partyfundsperprofessor Overviewonavailablestaffandresourcesatfaculties

    Whoprovideswhichamountoffinancialresourc-esfortheset-upofaQA-unitandforwhatperi-od?Forwhichpurposescantheseresourcesbeused?(E.g.facilities,staff,IT)

    WhatisthenumberofqualifiedstaffavailablefortheQA-unit,andforwhatperiod?

    Whichadditionalqualityassuranceactivitiescanberealisedbasedonthird-partyfunds(e.g.additionallectures,tutorials,mentoringpro-grammes)?

    Processman-agementofteachingandlearning

    Input/outputdataofthepro-cessteachingandlearning(aggregationonprogrammelevel)

    Dataoninternationalisation Qualityofgraduates Detaileddataonteachingandlearning(e.g.coursescheme,assessment,mentoring)

    Capacitiesofprofessorshipinteachingandlearning

    Whichdataisavailableonthenumberofappli-cationsperplaceinaprogramme,thenumberofstudents/graduatesperprogramme,thedrop-outratioetc.?Isthisdataconsistentwithinter-nal/externalqualityrequirements?Whichaddi-tionaldatamightbenecessary?

    Howmanyincomingandoutgoingstudentsarethereonfaculty/programmelevel?

    Isthereanyinformationavailableonthegradu-atesandtheircareerpaths?

    Whichinterdisciplinarycoursesdowehave? Scopeofregularcoursesofferedperpro-gramme?Numberofparticipantsperlecture?

    Numberofprofessorsperprogramme?Mentor-ingratioperprogramme?

    Table 1 Information sources and requirements from different stakeholders (adapted from Nusselein 2002)

    According to thedifferent subject-mattersmentioned in the table, theprioritiesof the listed information

    requirementsdifferdependingontherespectivetargetgroup.Focussingonthestrategicframeinresearch

    and teaching, forexample,a vice-chancellorneedsother information thanadeanoradeanof students.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    20

    Theformerisespeciallyinterestedinstrategicplanningofthewholehighereducationinstitutionandconsid-

    ersinformationaboutstrategicplanningonfacultylevel.Adeanofstudentshowever,isresponsibleforteach-

    ingandlearning,focussingespeciallyonexaminationandprogrammeregulations.Yetforadean,information

    onregulationsofthedoctorateorpost-doctorallecturequalificationsmightbemorerelevant.

    Informationthatreferstothefinancial resourcesandcashflowsareespeciallyrelevantforthechancellor

    (understoodasheadofadministration)whoisresponsibleforthebudgetofahighereducationinstitution.

    However,theinformationrequirementsofthevice-chancellororthesenatemightfocusondataaboutstaff-

    ingorthird-partyfundsperprofessorwhichcanbeusedasquantitativeindicatorsforresearchperformance.

    Amongothers,theyneedthis informationforprofessorialappointmentprocedures.Afacultyneedsmore

    detailedindicatorssuchastheavailablestaffingorfinancialresourcesofthefaculty.

    Focussingontheprocessofteachingandlearning,thetopmanagementisusuallyinterestedininput/output

    dataonprogrammelevel(e.g.numberofapplication,students,graduates,drop-outratioperprogramme).

    Furthermore,dataoninternationalisationandthequalityofthegraduatesisrelevantinordertoanalyseand

    interpretthesuccessofastudyprogramme.Deansofstudentsneedinformationthatdifferentiatesinmore

    detailbetweenthewholestudyprocesses(e.g.dataontheorganisationofassessment,courses,andproce-

    duresofrecognition).Finally,achancellorneedsdatatobeabletodeterminetherequiredresources(capac-

    ities)inteachingandlearning.

    Qualitymanagersshouldknowallthesedifferentperspectivesandtherespectiveinformationrequirements.

    Basedonthis,theycancontributetothedistributionofinformationtothosewhoeffectivelyneedthem,but

    alsosupportdecision-makingprocessesondifferentinstitutionallevels.

    Questions & Assignments

    1. Pleasestudythetableandthementionedexamplesofinformationrequirementsagain.Lookingat

    yourown institution,whichof these informationrequirementsdothedecision-makersprioritise

    andwhy?

    2. Arethereanyadditionalinformationrequirementswithregardtoqualityimprovementinteaching

    andlearningatyourhighereducationinstitution?Forwhomandwhy?Pleasegiveexamples.

    Howcanqualitymanagersfindoutaboutthesedifferentinformationneedswithoutonlyraisingassumptions

    orhypotheses?Therearedifferentwaysofgatheringinformationrequirements,whichcanbeseparatedinto

    inductiveanddeductiveprocedures (Kpper2001,145). Inductivemethods focusontheconditionsofan

    organisationasthefundamentforinformationrequirements.Basedonthis,youparticularlyidentifyinforma-

    tionsupplyaswellassubjectiveinformationneeds.Examplesformethodologicalapproachesaresuchasthe

    analysesoforganisationaldocumentsanddataorananalysisoftheorganisationorasurveybasedoninter-

    viewsorquestionnaires.Deductivemethodsidentifyinformationinasystematicway:Basedonthestrategic

    objectivesofanorganisation,theytrytofindoutabouttheobjectiveinformationneeds(Kpperetal.2013,

    222;Nusselein2002,3).

    Determination ofinformation requirements

    basedon inductiveand

    deductive procedures

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    21

    Togainamorecomprehensivepictureoftheinformationneedsthatisbothobjectiveandsubjectiveinfor-

    mationneedsitisrecommendedtocombineboththeinductiveanddeductiveapproach.Thefollowing

    procedureofananalysisofinformationneedsmayserveasanexample:

    Integrated concept of an information needs analysis

    Figure 3 Based on the project Computer-based management tool for the institutions of higher education in Bavaria (CEUS) (Nusselein 2002, 4)

    Description of Figure 3:

    Theorganisationanalysis focusesontherespectiveunitsofanorganisationunddeterminestargetsand

    decision-makingcompetencesoftherespectivedecision-makers(inthecaseofhighereducationinstitu-

    tionssuchas(vice)chancellor,highereducationboard,senate,chancellor,dean,deanofstudents).5

    Theresultsoftheorganisationalanalysesarethebasisforthesubsequentinterviewswiththeabove-men-

    tioneddecision-makers.Theinterviewshavetwopurposesinparticular:First,theycompletetheobjective

    targetprofilebyaddingsubjectivelyconsideredtargets(seeabove,No.1);second,theygiveinformation

    aboutthesubjectivelyconsideredinformationrequirementsforthedefinedsetoftargets.

    Thedeductiveanalysisgathersobjective informationrequirementsandwith it completes thesubjective

    informationneedsgainedbytheinterviews.

    Followingthis,theresultsaretestedwithanothersurveybytheabove-mentioneddecision-makers.Based

    onaquestionnaire,theyshallevaluateandnarrowdowntheinformationrequirementsaccordingtoprior-

    ities(Kpper1997,133).IntheprojectCEUS,theoutlineofthequestionnairewasbasedontheaforemen-

    tionedsubject-matters:a)structuralconditions,b)resources,c)processplanninginteachingandlearning,

    d)processplanninginresearch(Nusselein2002,5).

    Inaconcludingworkshopthesurveyresultsarediscussedwiththedecision-makersagain.Ifnecessary,fur-

    theradaptionsoftheinformationneedsaretobeeffected.6

    5 Thetypesofdecision-makersmaydifferdependingontheorganisationalstructureandhavetobeadaptedaccordingly.6 IntheCEUSprojectthismethodofgatheringinformationneedswasrealisedatseveralhighereducationinstitutions.Basedonthis itwaspossibletoachieveasufficientandcomparabledatabasis.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    22

    Thedescribedwayofgatheringinformationrequirementsexemplifiestheprocedureatvarioushighereduca-

    tioninstitutionsinGermany.Itisimportanttokeepinmindthatduetodifferentstructuralconditionsindiffer-

    entcountriesandinstitutions,thedescribedmethodtoanalyseinformationrequirementshastobeadjusted,

    dependingontheinternalandexternalparticularitiesofahighereducationinstitution.

    Dependingonthepurposesoftheinformationistobeused,thecollectionofdatahastobemoreaggregated

    ormoredetailed.Consideringtheabove-mentionedexamplesofinformationrequirementsofa(vice)chan-

    cellor,achancellororrepresentativesfromfaculties,itcanbeconcludedthatthedetail-leveloftheprovid-

    edinformationincreaseswithadecreasinghierarchylevel.Viceversa,theaggregationlevelofinformation

    increasesfromthelowesttothetophierarchylevel.Inordertoprovidecomparabledataandinformationon

    alllevels,theaggregationofinformationshouldalwaysrefertoacommonandstandardiseddata-basis(Eber-

    hardt2003,73).

    Furthermore,itcanbeconcludedthatingeneralitisnotpossibletocoverallinformationneeds.Establish-

    inganddevelopingastructuredinformationsystemathighereducationinstitutionscanhelptocloseorat

    leasttoreducethesegaps.Therefore,oneofthekeychallengesisnotknowingexactlywhichunitsofahigher

    educationinstitutionprovidepromisinginformationsourcesandhowtoconnectandusetheseinformation

    sourcesforthewholeinstitution.Sometimesthatisbecausetherespectiveinvolvedpartiesdonotwishsuch

    connections.Sometimes,collectingspecificinformationneedsisjustnotpossible,beitbecauseofalackof

    time,beitduetotechnicalrestraints,orbecausethereisnotenoughstafffortheprocessing.

    Consideringthis,acontrollerwhoisresponsibleforgatheringinformation,firstofallhastoanswerthefol-

    lowingquestions:

    Doesmyinstitutionprovidetheinformationneeded?

    Whichpossibilitiestogatherinformationtheinstitutiondoesnotyetprovideexist?

    Howmuchtimeandeffortdoesittaketoprovidethisinformationandwhocandoit?

    Whichqualitycriteriacanbeguaranteedfortheinformationtobeneededwithregardtobeingcomplete,

    timely,comparableetc.(seeTable2)

    Questions & Assignments

    1. Howdoyouproceedwhengatheringinformationatyourinstitution?Whoisresponsibleforthis

    task?

    2. Howfardoestheprovidedinformationmeettheneedsofthetargetgroups?

    3.Whichchallengesareyouconfrontedwithwhencollectingdataatyourinstitution?

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    23

    II. Data collection, processing and analysis

    Havingcollectedthenecessarydatafortherespectiveinformationneeds,thisdatanowhastobeevaluated

    andanalysedinatransparentandunderstandableway.Generallythisisdonebystafflocatedataunitfor

    managerialaccounting.Butwithregardstodataanalysisaccordingtodefinedqualitycriteria(seeTable2),it

    isrecommendabletoinvolvethequalitymanageraswell.Additionally,she/hecanhelptoillustratethetech-

    nicaldatainsuchawaythattherespectivetargetgroupisabletoread,understandandinterpretitcorrectly.

    Themaintaskforqualitymanagerswhoareresponsiblefortheevaluationandanalysesofdataandinforma-

    tionistocheckwhichcharacteristicsanidealinformationshouldhavetosatisfythedesiredinformationneeds

    asmuchaspossible(Hrvath2011,298etseqq.).Thisprocessofevaluationandanalysesincludesvarious

    challenges.

    Averycommonproblemis, forexample,thatdata isnotcurrent,butretrospective,that it istoodetailed

    andextensive,orthatitisinconsistentandcontradictory.Basedontheserestrictions,thedatadoesnotgive

    enoughsignificantinformationontherespectiverequirements.

    Workingagainsttheserestrictionsandachievinggreaterprecisionofthecollecteddatawithregardtothe

    respectiveinformationneeds,somecriteriaofsuccessshouldbeevaluated.

    Thefollowingtableshowsexamplesofkeycriteriaofsuccesswhenevaluatingandanalysingdataandinfor-

    mation.Itincludessomeimportantquestionsthatshouldbeansweredwhencheckingthesecriteria.

    Criteria of success for data collection

    Questions to be clarified Phrase to memorise

    Typeofdata Isitquantitativeorqualitativedata? Whichinformationdoesthedatagive? Isthedatasignificantlyvaluable?

    Thedataiscategorisedclearlyintoquantitativeorqualitativecategories.Thesignificanceofthedataisclearandcanbenamed.

    Degreeofcom-pression

    Arethereanyduplicationsthatcanbereduced? Howtoaggregateandsummarisedata?

    Asmuchdataasnecessary,aslittledataaspossible.

    Timelinessofdata Isthedataup-to-date? Istheperiodofdatacollectionandthereportingperiodcongruenttotherespectiveissueofinter-est?

    Theperiodofdatacollectionreferstotherelatedissueofinterest.Theperiodofdatacollectionmatchesthereportingperiod.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    24

    Criteria of success for data collection

    Questions to be clarified Phrase to memorise

    Layout Whichlayoutisappropriateforthetargetgroup?(e.g.writtenreport;tablesummary;graphic/visualisedlayout)

    Doesthelayouttransfertheneededinformation? Doesthelayoutincludeasystematicandreada-bleoutline?

    Thelayoutisappropriatefortheneedsofthetargetgroup.

    Problem-solvingrelevance

    Whichinformationvaluedoesthedatahaveforthetargetgroup?

    Whichindicatorprovesthisvalueandwhodecidesaboutthisindicator?

    Thecollecteddataisvaluablewithregardtotheissueofinterest.

    Priorityandcol-lectionfrequency

    Whenisthedataneededandwho/whatdecidesaboutthistimeframe?

    Whatisthefrequencyofdatacollectionandreporting?

    Whichconsequenceshavetobeconsideredwithregardtothescopeofdataevaluationandanaly-sesresultingfromshort-termorlong-terminfor-mationneeds?

    Istheperiodofdatacollectioncoordinatedwiththedateofprovision?

    Whichcontrolmechanismscanbeconsideredrespectivetotheavailabletime?

    Isthecollectionfrequencysufficienttoachievesignificantinformationfromthedata?

    Theperiodofdatacollectioniscoordinatedwiththedateofprovision.Thefrequencyofdatacollec-tionissufficienttoproducesignificantinformation.

    Purposeofuse Isthedataonlyusedforonepurposeordoesitservevariouspurposes?

    Doesthepurposerequireaspecialformofdataevaluationandanalyses?

    Checkifdatacanbeusedfordifferentpurposes.

    Amount Whichdataisrequiredtodelivertheinforma-tionneededfromtherespectivetargetgroupandwhichnot?

    Howdetailedshoulddatabetodelivercertaininformation?

    Thelevelofdetailandtheamountofdatamatchestheissuesofinterestandinforma-tionneededfromthetargetgroup.Basedonfiltering,comprehen-sionandcanalisationofdata,youshouldproducesignificantandunderstandableinforma-tion.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    25

    Criteria of success for data collection

    Questions to be clarified Phrase to memorise

    Accuracy Whatisthelevelofaccuracyofthecollecteddata?

    Doesthedatadelivercoherentandconsistentinformationordoesitincludecontradictoryordifferingpossibilitiesofinterpretation?Ifso,howfardoesthisreducethevalueofthegainedinfor-mation?

    Reducecontradictoryformsofinterpretation,butproduceclearandunambiguousinfor-mationfromthedata.

    Reliability Whatisthedatasource?Isthedatasourcereli-ablewithregardtotransparency,methodologyandmeasurability?

    Thecollecteddataisobtainedfromareliabledatasource.

    Measurability/plausibility

    Whichcriteriahavebeendefinedtomeasurethedata?

    Arethesecriteriatransparentandunderstanda-ble?

    Defineclearandunderstanda-blecriteriaofmeasurability.

    Costs Whichfinancial,stafformaterialcostsresultfromcollecting,analysingandreportingdata?

    Clarifythecostsfordatacollec-tion,analysesandreporting.

    Data-protection Whataretheproceduresofdocumentingandsavingdata?

    Whichdataprotectionruleshavetobeconsid-eredwithregardtodataaccess?

    Clarifyregulationsandpro-ceduresofdocumentingandsavingdata.

    Communicationprocesses

    Whichcommunicationflowsarenecessaryforcollecting,analysingandusingdata?

    Whoisinvolvedindatacollectionandanalyses? Whohastobeinformedaboutthedatacollec-tionandanalysesandhow?

    Arethesecommunicationflowsclearandtrans-parenttoallinvolvedstakeholders,andtowhatextentaretheyputintopractice?

    Coordinateanddefinecom-municationchannelstocollect,analyseandusedata.

    Table 2 Criteria of success for data collection

    Questions & Assignments

    Theseniormanagementofyourinstitutionwantsallfacultiestohandinareportaboutthecurrent

    successoftheirstudyprogrammes.

    1.Howdoyoureportthesuccessofstudyprogrammesatyourinstitution?

    2.Whichinformationneedsdoyouconsidertoberelevantinthisregard?

    3.Whichcriteriaofsuccessareimportanttobeconsideredinthedatacollectionprocess?

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    26

    III. Distribution of information

    Aftercollectingandanalysingthedatathegainedinformationistobedistributedtotherespectiveaddress-

    eesviareportingsystems.Thedesignofthesereportingsystemscanvarydependingonthetypeandthe

    amountofinformation,aswellasthetargetgroupanditsobjectives.InChapter4wewilllearnaboutthe

    reporting-issueinmoredetail.Therefore,thischapterwillonlygiveanoverviewontherequirementsofan

    informationmanagementsystemwithregardtodistribution.

    The key element of distributing information is the relation between the sender of information and the

    addresseeandthequestionofhowtotransfertherelevantinformationappropriately.Thismeans,aninfor-

    mationsenderhastoknowtowhomshe/hehastodelivertheinformationandinwhichform.Atthesame

    time,addresseesofinformationshouldknowhowtoread,understandandusethereceivedinformationfor

    thearticulatedneeds(Hrvath2011,354etseq.).

    Suchcoordinationisnoteasytoachieveinpracticebutincludesvariouschallenges.Forexample,producers

    ofinformationoftendonotknowsufficientlywhotheaddresseeofthecollecteddataisandwhatthedatais

    neededfor.Ontheotherhand,forinformationusersitmightbeunclearwhichinformationcanbeprovided,

    howtoreadandanalysecollecteddataconsideringtherespectivecontext.

    Figure 4 Types of interferences during the process of information distribution (adapted from Kpper et al. 2013, 241)

    Dealingwiththesechallenges,qualitymanagerscanplayanimportantrolebybeingacommunicationlinkage

    betweenthedifferentstakeholdersandunitsofahighereducationinstitution.Theycanrevealcommunica-

    tionandinformationgapsbetweensendersandaddresseesofinformationandreducethembyclarifyingthe

    contentofthespecificdatainanunderstandablewayforthetargetgroups.Indoingsotheycontributeto

    achievingmoretransparencyandworkinginformationflowsathighereducationinstitutions.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    27

    Incidence Case: Students Newsletter

    TheresultofasurveyattheAfrican-Universitywasthatthestudentsfeel insufficiently informed,

    beitwithregardtoorganisationalproceduresandrelevantdeadlinesoftheirstudiesorwithregard

    tocurrentdevelopmentsinresearch.Thevice-chancellorforacademicsaskedthequalitymanagerin

    chargetodevelopanewsletter.Thepurposeofthisnewsletterwastoinformregularly(e.g.quarter-

    ly)aboutrelevantorganisationalissues,deadlinesandfixeddates,newservices,orotherissuesthat

    mightbeofinterest.Sinceanewsletteriscloselyrelatedtothetargetsofthedepartmentforpublic

    relations, thequalitymanager informedthedepartmentabout thiswork task. Indoingso,healso

    wantedtofindouthowfarthepublicrelationscolleagueswereabletosupporthimwithregardto

    developinganddistributingthenewsletter.Aftertalkingtoeachother,thequalitymanagerdecided

    topublishthenewsletterbothasaprintversionandasanonlinepdf-versionontheuniversityhome-

    pagetoreachasmanyuniversitymembersaspossible.Thepublicrelationscolleaguesofferedtocare

    fortheplacementofthedocumentonthewebsiteandtosendasufficientnumberofprintedcopies

    toeachfacultyandunit.Furthermore,thequalitymanageraskedacolleaguefromthedepartmentof

    dataandinformationmanagementtocreateamailinglist.Inthefuture,interesteduniversitymem-

    berscansubscribetothismailinglistandwillreceivethenewsletterautomatically.

    Concerningthecontentdesignofthenewsletter,thequalitymanagerwantstoproceedaccordingto

    thefollowingoutline:

    1. Didyoualreadyknowabout?

    Informationaboutinterestingevents

    Importantdatesanddeadlines

    Currentresearchprojectsattheuniversity

    Miscellaneous

    2. LibraryServices

    3. ICTServices

    4. Haveyoualreadyread?Newpublicationsfromresearchersoftheuniversity

    5. Portraitofauniversitymember(shortinterviewwith5-6questions)

    Thequalitymanagerisveryenthusiasticabouthisprojectactionplanforthepublicationofthenews-

    letter and already very excited about feedback from the students and the other universitymem-

    bers.

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    28

    Further Reading

    Alter,S.(1996).Information systems: A management perspective(2ndedition).MenloPark:

    BenjaminCummingsPub.Co.

    Questions & Assignments

    Your vice-chancellor of academics asks you to develop a newsletter for the lecturers at your

    university.

    1.Whatmightbeinterestingandrelevantinformationforlecturers?Howandbywhomcouldyou

    gathertheseinformationneeds?

    2.Whichstepsdoyouhavetoconsidertodesignanddistributethisnewsletter?

    Whichchallengesshouldbeconsideredinthisregard?Whichcriteriaofsuccessareimportanttobe

    consideredinthedatacollectionprocess?

  • Chapter 1: Introduction to Information Management at Higher Education Insitutions

    29

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    30

    2 TranslationofHigherEducationObjectivesInto Numbers:QuantitativeandQualitativeIndicators . . . 31

    2.1 MeaningandFunctionofQuantitativeandQualitativeIndicators . . . . . . . . . . . 31

    2.2 DeterminationandOperationalisationof QuantitativeandQualitativeIndicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    2.3 UsingIndicatorsKeyAspectstoBeConsidered . . . . . . . . . . . . . . . . . . . . . . . . . 34

    2.3.1 RequirementstoDefineIndicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

    2.3.2 TheBalanced-ScorecardAnInstrumenttoMonitorIndicators . . . . . . . . . . . . . 36

    2.4 ChallengesofUsingQuantitativeandQualitativeIndicators . . . . . . . . . . . . . . . . 39

    differentiatekeyfunctionsofusingquantitativeandqualitativeindicators,

    determineandoperationalisequantitativeandqualitativeindicatorsbydeterminingcentralparameters

    suchasthesample,thereferenceperiodorthenumericalvalue,

    considerkeyconditionswhenusingquantitativeandqualitativeindicators(e.g.trade-offsbetweenrele-

    vantandnon-relevantdata,validityofdata,sensitisationofthetargetgroup,expenditureincostandtime,

    dataprotection),

    dealwiththeconceptoftheacademicbalancedscorecard.Basedonthis,participantsareabletotranslate

    HEIstrategiesintoobjectivesandfindsuitableindicatorstomeasureaperformanceleveltobereachedin

    adefinedperiod.

    On successful completion of this chapter, you should be able to

    Chapter 2

    Translation of Higher Education Objectives Into Numbers

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    31

    2 Translation of Higher Education Objectives Into Numbers: Quantitative and Qualitative Indicators

    2.1 Meaning and Function of Quantitative and Qualitative Indicators

    Inthepreviouschaptersyouhavelearnedthatthepurposesofinformationsystemsareto

    supportdecision-makingprocesses,

    achievetransparencyonstructuralprocesses,

    increaseefficiencyandeffectivenessoftheprocessesathighereducationinstitutions.

    Indicatorsplayanimportantroletoreachtheseobjectives.Theirtaskistosummariseaquantitativemeasur-

    ablesituationandtoidentifyrelevantfactsandcorrelationsinasimpleandcondensedform.(Kpper2013,

    476).

    Focussingonhighereducationinstitutionsmeansmakinganyactivitiesreferringtodecision-making,organi-

    sationalorplanningprocessestransparent.Theygiveaquantitativeoverviewaboutthestatusquoatahigher

    educationinstitution.Indicatorsreducecomplexityandaggregateinformation,whichmeansthattheyinform

    aspreciselyandbrieflyaspossibleaboutperformances.Indoingso,theyhelptoachieveanadequateinfor-

    mationsupplyforhighereducationmanagement:theyallowanalysingthestatusquoaswellastoevaluating

    theoutcomesofthespecificcoursesofactions.Fromaninternalperspectivetheyareafundamentalbasisof

    managementandrelateddecision-makingprocesses.Fromanexternalperspective,highereducationinsti-

    tutionscanbemeasured,compared(e.g.rankings)andevenmanaged(e.g.targetagreementswiththemin-

    istry)basedonperformanceindicators.Basedonthis,indicatorsarealsocloselyrelatedtothequalityassur-

    ancesystemofahighereducationinstitution.

    If indicatorsareusedtodescribeperformancesorthesuccessofdefinedobjectivesofahighereducation

    institution,weoftenusethetermkeyperformanceindicatorsorperformanceindicators.Accordingto

    theAnalyticQualityGlossary,

    Performance indicators are data, usually quantitative in form, that provide a measure of some

    aspect of an individuals or organisations performance against which changes in performance or

    the performance of others can be compared.

    (Harvey 2004-14)

    It shouldbeconsidered, thatalthoughperformance indicatorshavea relativelyprecisemeaning, there is

    atendencytousethistermforanystatisticaldatarelatedtotheactivitiesofhighereducationinstitutions,

    whetherornotitreallyreferstoperformanceorsuccess(Harvey2004-14).

    Performance Indicators

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    32

    Consideringthis,qualitymanagersshouldbeabletounderstandmeaningandfunctionof(performance)indi-

    cators,usethemcorrectlyandexplainthemappropriatelytotherespectivetargetgroups.

    AccordingtoGladen(2003,11)keyfunctionsofindicatorscan...

    describecomplexandoperationalissues,structuresandprocessesinarathersimpleway,

    guarantyacomprisingandquickoverview,

    serveleadershipforspecificanalyses,

    serveleadershipforcurrentplanning,decision-makingandmanagerialaccounting,

    enableinformationreleasebyaggregationandselection,

    describecriticalfactorsofsuccessandshortagesinthemanagementsystem.

    2.2 Determination and Operationalisation of Quantitative and Qualitative Indicators

    Indicatorscanbedescribedwiththreekeyparameters:

    1. Theobject/target,theyaredescribing(what?).

    2. Thetimeframe,whichtheyreferto(dateorperiod?).

    3. Adefinednumericalvalueforquantification(howmuch?).

    Indicatorscanbedifferentiatedintoquantitativeandqualitativeindicators.Quantitativeindicatorsdescribe

    issuesandsituationswithaclearlydefinednumber.Basedonthereductiontothesubstantialsignificance,

    existingindividualinformationiscondensedtoanobservableandmeasurablematteroffact(Gladen2003,

    12).

    Examplesincludeavailablethird-partyfundsofafaculty,numberofstudentsinacertainprogramme,number

    ofPhDstudentsperprofessor,availableacademicstaffofafaculty,drop-outstudentsratiosetc.

    Qualitativeindicatorsareproxyparameters,whosecharacterorvaryingvaluehelpstoconcludethechar-

    acterorvaryingvalueofanotherimportantparameter(translatedfromGladen2003,15).Thatmeansthat

    theydonotdescribedirectlymeasurablevariables,buttheyserveasasubstitutewhichiseasiertobemeas-

    ured.Basedonthiswecananalyseperformancesthatcannotbequantifiedormeasureddirectly.Forexam-

    ple,ifafacultyorachairwantstodescribeitsresearchperformancelevel,theyconsidervariousquantitative

    indicatorssuchasnumberofpublications,patents,successfuldoctoratesortheamountofraisedthird-party

    funds.Thesumoftheseindicatorsissupposedtohelpratingtheresearchperformance.

    Theproblemofusingqualitative indicators is that theyonlyhavea limitedvalidity,becausethecause-ef-

    fectrelationshipbetweentheoriginalandthesubstitutingindicatorisonlybasedonassumptions,butnot

    Quantitative Indicators

    Qualitative indicators

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    33

    onexactdescriptions.Thismeansthatcause-effectrelationshipscanbebiasedormono-causalandwithit

    incomplete(Kpper2013,480).7Thiscanprovokecontradictionswithregardtotheanalysisandinterpreta-

    tionoftherespectivedata,asisshowninthefollowingexample:

    Theseniormanagementofahighereducation institutionwantstoknowwhichthemostsuccessfulstudy

    programmesof their faculties are. Therefore, theydefine thequantitative indicator numberof achieved

    degrees.Viewedinisolation,thisindicatorisdefinitelyvalidsinceitdescribeswhatitismeanttodescribe

    thesuccessofstudyprogrammes,whichismirroredintherespectivenumberofdegrees.Nevertheless,

    ifnotusedadequately,this indicatorcanentailwrongincentivesorundesiredside-effects.Forexample,a

    target-settingbasedonthisindicatorcouldinducefacultiestoneglectexistingcriteriatopassfinalexamsin

    ordertobeabletoachieveasmanysuccessfuldegreesaspossible.

    Theexampleshowsthatwehavetobecarefulandmustdefineindicatorsdeliberatelywhenusingthemfor

    managementpurposes(alsoconsiderChapter2.3.2focussingontheBalancedScorecard(BSC)).

    Ifasuccessfulstudyprogrammeisnotonlydefinedbythenumberofgraduatesbutalsobyfulfillingprevi-

    ouslydefinedminimumrequirementsinteachingandlearning,thismeansdifferentiatingandconcretising

    consideredparametersinamorequalitativeway.Forexample,todescribeasuccessfulstudyprogrammewe

    canconsiderevenmorequantitativeindicatorsthataresummarisedtoaqualitativeindicator(e.g.mentoring

    studentsratio,drop-outstudentsratio,numberofrepetitionoffinalexamsortheaveragetimeneededto

    completeadegree).

    Similarly,wecanrefertosuccessfulresearch:Thesuccessofascientificexperimentdependsonvariousinflu-

    encingparameters,whicharesearcheroftenisnotabletocontrol.Thatmeans,weneedindicatorsthatare

    abletoreduceinformationasymmetriesinsuchawaythattheaddressee(e.g.theseniormanagement)is

    abletoconcludeonthefactualresearchactivitiesoftheresearcher.

    Therefore, data cannot only be analysed quantitatively, but their qualitative characteristics and possible

    resultingeffectshavetobeconsideredaswell.

    Further Reading

    Dealingwithnationalteachingperformanceindicatorsthefollowingarticlegivesanexamplefrom

    Australia:

    Barrie,S.,&Ginns,P.(2007).Thelinkingofnationalteachingperformanceindicatorstoimprove-

    mentsinteachingandlearninginclassrooms. Quality in Higher Education,13(3),205-286.

    7YoufindmoreinformationonhowtodealwiththeissuevalidityinModule2,Chapter5.4.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    34

    Qualitymanagerscanplayanimportantroleinthiscontext.Theycanuncovercontradictionswhenusingindi-

    cators,theycanmaketransparentandunderstandablecause-effectrelationships,andtheycanshowdeci-

    sion-makerswaysofdealingwiththemappropriately.Todoso,qualitymanagersshouldknowandbeableto

    dealwiththekeyrequirementsofindicators.Therefore,thefollowingchaptergivesanintroduction.

    2.3 Using Indicators Key Aspects to Be ConsideredThischapterdescribeskeyrequirementstobeconsideredtodefinevaluableindicators.Furthermore,wewill

    gettoknowtheBalancedScorecardasanexampleofaninstrumenttouseanddealwithindicators.Thechap-

    tertiesinwiththediscussionaboutthemethodologicalrealisationofsurveysinModule2.

    2.3.1 Requirements to Define IndicatorsThefollowingfactorsshouldbeconsideredwhenaimingatdefiningpreciseindicators(Hrvath2011,542et

    seq.;Tropp2002,57etseqq.)

    1. Each indicator needs a concrete purpose

    Tobesignificantanindicatorneedsaconcretepurposeandoneorseveral(butnotarbitraryselected)

    addressees.

    Tobeabletouseindicatorsforseveralpurposes,theyhavetobedefinedanddifferentiatedexactly.

    Datacollection,thatisnecessarytodefineanindicator,hastoberelatedappropriatelytothepurposeof

    theindicator.

    Formalrequirements(e.g.law/politicalrequirements),whicharerelevantfordefininganindicator,have

    tobeconsidered.

    Key questions to be answered:

    Whatisthesignificanceoftheindicator?

    Whichnumericalvaluetranslatesthissignificance?

    Whichinformationdoesthisnumericalvaluetakeintoaccountandwhichnot?

    Whichformalrequirementshavetobeconsidered?

    2. Validity of data: No quantitative data without additional qualitative information

    Indicatorshavetobecontrolledwithregardtotheirvaliditytoavoidwrongincentivesorunexpected/

    undesirableside-effects(seeexampleonsuccessfuldegrees).

    Key questions to be answered:

    Whatarethecontinualdatasourcesandwhocollectsthemtodefineanindicator?

    Whataresuitablereferencevalues(benchmarks)tocontrolthevalidityofanindicator?

    3. Trade-off between relevant and non-relevant data and information

    Providedhigh-qualityvalidity,thescopeofdatatodefineindicatorsshouldbereducedasmuchaspossi-

    ble.Anoverloadedlevelofdetailcanevenhinderstrategicmanagement.

    Reductionofdatacollectionthatisnotrelevantforthedefinitionofindicatorsandwithitavoiddata

    graveyards.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    35

    Key questions to be answered:

    Whichdataisnecessarytodefineacertainindicatorandwhichisnot?

    Arethereanyirrelevantdatathatkeepbeingconsideredunnecessarily?

    4. Considering feedback

    Thenumericaldatashouldbealignedtotherealityoftheaffectedstakeholdersandevaluatedwith

    regardtocontradictions.

    Atthesametime,theaffectedstakeholderscanbeprovidedwiththeevaluatedandanalyseddatatobe

    consideredforfurtheractionsanddevelopments.

    Key questions to be answered:

    Doesthecollecteddatareflectreality?

    Arethereanyconstraints?

    Dotheselectedindicatorsprovideanyadditionalbenefitsforimprovementandenhancement?

    5. No isolated measurements

    Whencollecting,analysinganddocumentingdata,itshouldnotbedoneinisolationbutcomparable

    parametersshouldbeconsidered(e.g.descriptionofabsolute,relativeandaccumulatednumbers).

    Datatobeusedtodefineindicatorsshouldbecollectedcontinuouslyoveralongerperiodinsteadofonly

    onceandinisolation.Byconsideringalongerperiodthesignificanceofindicatorsincreasesanditfacili-

    tatesamoreexactjudgementofaverageperformancelevels.

    Key questions to be answered:

    Whatisthedateofreferenceandtheperiodofreferenceforthedefinedindicator?

    Inwhichintervalshouldtheindicatorsbelookedat?

    6. Expenditure in cost and time

    Collecting,analysingandpublishingdataandinformationrequiresfinancial,staffandalsomaterial

    recourseswhichhavetobecalculatedintime.

    Timeneededtogatherinformationistobecalculatedintimeandtobecoordinatedwithpossibledead-

    lineswhichhavetobeconsidered.

    Key questions to be answered:

    Whichexpendituresonresources(staff,finances,IT-system,material)havetobeconsidered?

    Whatisthetimeframetosubmittherequireddataandinformation?

    Whatisthecost/benefit-ratiowithregardtoexpenditureofresourcesandtimeandtheadditional

    benefitoftheprovidedinformation?

    7. Data protection

    Thecollectedandanalyseddataaretreatedresponsiblyandaccordingtogivendataprotectionguide-

    lines.

    Key questions to be answered:

    Dodataandinformationcomplywiththerespectivedataprotectionguidelinesinforce?

    Whathastobedonetomeetpersonaldataprotectionrightsandtoavoidmisuse?

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    36

    8. Sensitisation of the target group to use edited data reports

    Informingthegroupofaddresseesabouthowtointerpretindicatorsandwhattousethemfor.

    Key questions to be answered:

    Istheinformationoftheindicatortransparenttothegroupofaddressees?

    Whichinformationdoesthegroupofaddresseesneedtobeabletousetheindicatorsappropriately?

    Further Reading

    Chalmers,D.(2008).Teaching and learning quality indicators in australian universities. Outcomes of

    higher education: Quality relevance and impact.Paris:ProgrammeonInstitutionalManagementin

    HigherEducation.

    2.3.2 The Balanced-Scorecard An Instrument to Monitor Indicators

    Indicatorsthataredefinedunderstandablyandcomprehensivelycancontributetoreduceinformationasym-

    metriesbetweendifferenttargetgroups.Theyspecifytherespectivedefinedobjectivesandthusfacilitate

    thecoordinationofnecessaryprocessestoreachtheseobjectives(Kpper2013,500).Thiscanbecarriedout

    eitherverticallyacrossthedifferenthierarchicallevelsofahighereducationinstitution,aimingatmanaging

    itsmultipleunits (e.g.with targetperformanceagreements),orhorizontally tomanagedifferentdomains

    basedondefinedtargetsforthesedomains(e.g.orientationofstudyprogrammesoninternationalstudents).

    Oneexampleofaninstrumenttomonitorindicatorsathighereducationinstitutionareindicatorsystems.An

    indicatorsystemis

    an arrangement of indicators in a systematic way, which means that the individual indicators

    are linked in a meaningful way, that they complement each other, and that they are aligned to an

    overriding common objective.

    (translated from Tropp 2002, 3 et seq.)

    Questions & Assignments

    1. Whichparticular conditionsdoesyour institutionhave toconsiderwhendealingwithdataand

    information?Whichchallengesdosuchconditionscomewith?

    Indicator system

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    37

    Besides indicator systems, another strategicmanagement instrument is theBalanced Scorecard,which is

    increasinginpopularityathighereducationinstitutions.

    ABSCfacilitatesthelinkbetweenstrategicplanningandoperationalprocessestorenderperformanceassess-

    ment.Otherthanindicatorsystems,aBSCisnotbasedonapredefinedsetofindicators,butenablesamore

    precisechoiceofindicatorsfortherespectiveobjectiveswhicharetobeoperationalised.Therefore,aBSC

    isveryusefulinmonitoringcomplexitiesandorganisationalparticularitiesofahighereducationinstitution,

    suchasuncleartechnologiesofperformanceassessment,ambiguousandcomplextargetstructures,differing

    memberships,staffexpertise,hierarchiesororganisationbasedonknowledge(Scheytt2007).

    ABSCcancontributesignificantlytoachievemoretransparencyandclarityaboutthestrategicobjectivesof

    ahighereducationinstitution.Basedonthis,suitableorganisationalprocessescanbedevelopedinorderto

    reachthesedefinedobjectivescanbedeveloped(Rbken2003,4).

    Thetermbalancedsignifiesthattheperspectivesthatarerelevanttorealiseastrategyareequallyweighted

    inthescorecard(Kaplan/Norten,inRbken2003).AccordingtoKaplanandNorten,typicalperspectivestobe

    consideredinaBSCarethefollowingfour8:

    1.customer

    2.learningandgrowth(humanresourcesandorganisationaldevelopment)

    3.financial

    4.internalprocesses

    Consideringtheseperspectives,wecandefineindicatorsforthestrategicobjectivesanddeterminetargetval-

    uesthathelptomeasurehowfartheseobjectiveshavebeenreached.

    Duetothebalancedconsiderationofthementionedperspectives,theBSC-approachtriestocopewiththe

    challengingtaskofcomprisingdifferingcontextsandinfluencingfactorsofsubject-mattersandofanalysing

    andinterpringoutcome-linkagesmoretransparentlyandclearly(Scheytt2007).

    AccordingtoKaplanandNorton(1996)theimplementationofaBalancedScorecardcanbebasedonfivekey

    steps(Scheytt2007):

    1 . Definitionofthedifferentperspectiveswhichareoffundamentalimportancetothehighereducation

    institution.ThesecandifferfromtheabovementionedeconomicalBSCmodel.

    2. Deductionofobjectives,whichareparticularlyimportanttofollowthestrategicplan(operationalisation

    ofobjectives).

    3. Definitionofindicators,whichinformaboutcontent,extentandtimeframetoreachtheobjectivesand

    thushelptomanagetheorganisationalprocessesofperformanceassessment.

    8Theseperspectivescanbeadaptedtotherespectiveneedsofaninstitution.

    ABSC translates thevision andthe strategyofa higher education institution intocoherent objectivesand indicators

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    38

    4. Definitionoftargetvaluesbasedoninfluencingparameterstobereachedinacertainperiod(e.g.one

    year).

    5. Definitionofinitiatives/activitiestoberealisedinordertoreachtheobjectivesduringadefinedperiod.

    Figure 5 System of a Balanced Scorecard (adapted from Scheytt 2007)

    Themanagedprocessing,asitisdescribedintheillustration,istheparticularcharacteristicoftheBSC,alsoto

    bedistinguishedfromotherconceptsofperformancemanagementsuchasindicatorsystems.Suchprocess

    orientationfacilitatesthediscussionabouttarget-performancecomparisons:Onetheonehand,thecurrent

    statusisdefinedbyanalysingthequestionswhodoeswhat,when,whereandhow?Ontheotherhand,tar-

    getvaluesandthequestionwhohastobeinvolvedandwhichinformationistobeneededfromwhomand

    tillwhen(Scheytt2007)

    Deducingindicatorsforthetotalobjectivehierarchyofahighereducationinstitutionaimsatguaranteeing

    congruencebetweenthedifferentobjectivesandatcoordinatingstrategicplanningwiththeorganisational

    processesofdailyperformanceassessment.Basedonthis,theBSCcansupportcommunicationprocesses

    betweenthedifferentdepartmentsandstaffbydevelopingaframeworkthatenablesacontinuousprocess

    ofself-evaluationandorganisationallearning(Rbken2003,4).Thisincludesaimingcontinuouslyatquality

    enhancementandwithitestablishingandsystemisinginternalqualityassurancestructures.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    39

    2.4 Challenges of Using Quantitative and Qualitative Indicators

    Whilecompaniesgenerallyhaveonebigstrategictargettobeachievedbyallemployees,athighereduca-

    tioninstitutionswecanfinddifferentlooselycoupledtargetsystems,whicharenotrelevantforallmembers

    oftheinstitutionbutonlyforpartialgroups.Thedifferentfaculties,theseniormanagement,aswellasthe

    administrationofahighereducationinstitutioncanhaveratherdiffering,sometimesevenconflictingtargets

    withdifferentpriorities.Forexample,aprofessorwhoisdoingresearchmightbeparticularlyinterestedin

    gainingsufficientthird-partyfundstobeabletodoresearch.Atthesametime,fortheseniormanagement

    third-partyfundsofferapossibilitytobalancebudgetdeficits.Furthermore,theystrengthenexternalinsti-

    tutionalprofiling.Meanwhile,alecturermightbeespeciallyinterestedinadequateresourcestobeableto

    facilitategoodteachingandlearningconditions.Thelatterisalsoakeyconcernofthestudentswhowantto

    completetheirstudiessuccessfully.

    Accordingtothis,anotherchallengetodealwithistheformulationofobjectives.Whatlevelofclarityand

    precisiondoobjectivesneedinordertobemeasurable?Andwhichamplescopecantheyhavetoenable

    a broadflexibilitywith regard to their design and implementationaccording to the academic freedom in

    researchandteaching.

    Basedonthis,anotherobstaclewhendefiningandusingindicatorsisthattheycannotbedefinedforseveral

    objectivesatthesametime,butonlyforoneconcreteobjective.Duetothissingle-sidedfocus,itmayoccur

    thatcausalitiesbetweendifferentobjectivesarenotconsideredandwithitentailcontradictoryorevenwrong

    interpretationsfortakingfurtheractions.UsingaBSC,requiresconsideringsuchcausalitieswhencombining

    differentindicatorsforanobjective.

    Theproblemofcontradictoryconclusionscanalsobeaconsequenceofdifferentunderstandingsaboutindi-

    catorsandtheirassumedprioritylevels.Thefollowingmetaphoricalcomparisoncouldhelptoillustratethis

    problem:whentalkingaboutapples,wecanassumethatonepersonconsidersanappletobebig,sourand

    Further Reading

    Kaplan,R.S.(2011).StrategicperformancemeasurementandmanagementinNonprofitOrganiza-

    tions.Nonprofit Management & Leadership, 11(3),353370.

    Kaplan,R.S.,&Norton,D.P.(1993).Puttingthebalancedscorecardtowork.Harvard Business Re-

    view,71(5),134-147.

    Kaplan,R.S.,&Norton,D.P.(1996).Usingthebalancedscorecardasastrategicmanagementsys-

    tem.Harvard Business Review, 74(1),75-85.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    40

    green,meanwhileanotherthinksaboutsmall,crispandredapples.Translatingthistothehighereducation

    contextmeans, forexample, thatgoodteachingat the facultyofmathematicscanbecharacteriseddif-

    ferentlythanatthefacultyofsocialsciences9.Also,internationalpublicationstobeusedasanindicatorof

    researchqualitycanberatherimportantinonefaculty,whileinanothertheyarenotasrelevant.

    Thesedifferingunderstandingshavetobeconsideredandclarifiedwhendefiningindicators.Onlythen,are

    weabletoachieveacommonbasisfortheiranalysisandinterpretationandcanthereforeavoidtheapple

    comparisonbecomingacomparisonofapplesandpears.

    Anotherprecedingchallengeisthathighereducationinstitutionsneedanoverarchingstrategyasabasisto

    defineanduseindicators.Whatwecanobserveisthatstrategiesonlyexistonpaper,buttheydonotplaya

    rolewithregardtooperationalisingprocessesandactivities.Ifhighereducationinstitutionswanttodealwith

    indicators,strategicplanningisanobligatoryrequirementitisthestrategythatistranslatedintoconcrete

    operationalisedtargets(e.g.basedonaBSC)thataremeasuredbasedonappropriateindicators.Thatmeans,

    theessentialprerequisiteforintroducingaBalancedScorecardisahighereducationinstitutiondetermining

    itsstrategicorientation,documentingitandmakingittransparentamongthewholeorganisation,forexam-

    plebydevelopingstrategicplansoninstitutionalorfacultylevel.

    Furthermore,whenusingindicatorsdifferentcomparisondimensionshavetobeconsidered:forexample,for

    internalpurposesindicatorsareoftenusedtocomparedatainahistoricaltimeframe.Thatmeans,theymon-

    itorcertaindevelopmentsduringagivenperiodoftimeandserveasabasisforfutureperformancelevelsto

    beachieved,andwhicharenegotiated,e.g.viatarget-performanceagreements(Rbken2003).Forexternal

    purposes,indicatorscansupportthecomparisonofhighereducationinstitutions(orafaculty,aunitetc.)in

    termsofrankingsorbenchmarking.

    Focussingonthevalidityofindicatorsanotherchallengeisthatveryoftentheycannotbecontrolledcom-

    parably,which lead to furtherdiffering interpretation frameworks. For example, higher education institu-

    tionscanhardlyinfluenceinput-parametersbecausetheycannotinfluencetheprovisionofresources.This

    changeswhenwelookatprocess-parameters:toensureandenhancethequalityofteachingandlearning,

    weshouldnotonlyconsidertheprovidedresources,butfocusonaspectssuchascurriculumdesign,didac-

    tics,programmeandassessmentmanagement,planningstudentinfrastructure,evaluationofchairsorother

    teachingunits.

    Thementionedchallengesindicatethatdealingwithindicatorsinvolvesahighworkloadandexpenditureof

    time.Themorecomplicatedthemethodsandtechniquesforthedatacompilation,themoreriskofanincom-

    pleteandnonpermanentdata-collection,andwithitindicatorsthatareneitherrelevantnorsignificant.Con-

    sideringthis,wealsohavetoquestiontheintendedbenefitscomparedtotheintroducedcosts.Tocountervail

    thisproblem,itisimportanttoreflectwhichdata-collectionmethodsandwhichdataisalreadyavailableto

    describehighereducationprocesses,whichadditionalinformationmightbeusefulandtowhatextentexten-

    sionsoradaptionsoftheexistingdata-systemmightbepossibleanduseful.

    9ForfurtherexplanationsonhowtooperationalisethequalityofgoodteachingpleaseconsiderModule2,Chapter5.2.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    41

    Furthermore,highworkloadwithregardtocollectingdataandthe followingdocumentationandcommu-

    nicationflowscanresult inopposingandnegativeattitudesamongstaff.Toreducesuchoppositions, it is

    veryimportanttoexplainandcommunicatetheadditionalbenefitandthepurposeoftheintroductionofan

    instrumentliketheBSCorotherindicatorsystemsforahighereducationinstitution.

    ThedescribedchallengesbringoutthenarrowlimitsofusingaBSCandindicatorsasameanstoimprovepro-

    cessesandactivitiesthatservetoachievecertainobjectives.Wehavetokeeptheselimitsinmindandshould

    notunderestimatethem,sinceitmightbecomeevenmoreproblematicandcomplicated,whencontradic-

    tionsarenotclarified,butcontinuouslyproceeded.Inthiscase,theexpectedbenefitofworkingwithindica-

    torsasaninstrumenttosystematiseandmanageprocesseswouldbenotberealised.

    Consideringthis,whenoperationalising indicatorswecontinuouslyhavetocheckwhichindicatorcanpro-

    videwhichcontributionandhowrelevantthiscontributioniswithregardtoachievingtheintendedstrategic

    objective.

    Comingbacktoqualityassuranceprocessesathighereducationinstitutions,qualitymanagersplayanimpor-

    tantroleindealingwiththeabove-mentionedchallenges.Theycanhelptodefineappropriateindicatorsfor

    thekeyprocesses,teachingandresearch.Furthermore,theyshouldrevealbothopportunitiesandalsolimits

    ofusingindicatorsandmakethemtransparenttotherespectivetargetgroups.Basedonthis,theycanfacili-

    tateacoordinatedandadequateinformationfundamentfordecision-makingprocesses.

    Challenges when dealing with (performance) indicators

    Example

    AtHEItherearedifferentstakeholderswithmul-tiple,sometimescontradictoryobjectives.

    Topmanagement:getthird-partyfundsforrea-sonsofcompetitionandcompensationofbudgetdeficits.

    Professor:getsthird-partyfundstodomoreresearch.

    Anindicatorcannotrepresentmultipleobjectives

    butonlyonedefinedobjective.

    Theindicatorthird-partyfundsofafacultyreferstotheallocationofthird-partyfundsatafaculty.Itdoesnotrefertoresearchquality.

    Adefinedstrategyisaprerequisitetouseabal-anced-scorecard.

    HEIstrategy:toincreasetheinternationalisationofteachingandlearning.

    Indicators: Numberofinternationalstudyprogrammes Numberofinternationalcollaborativeresearch

    projects

    Thecomparabilityofindicatorsmaydiffer(e.g.dependingontheirlongitudinalorinter-organi-sationaluse)

    Longitudinaluse:comparedatawithregardtothedevelopmentofstudyprogrammesoveracertainperiod.

    Inter-organisationaluse:comparetwofacultieswithregardtothenumberofgraduates.

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    42

    Challenges when dealing with (performance) indicators

    Example

    Theinfluenceonthevalidityofindicatorsmaydiffer(e.g.inputvs.processindicators)

    Indicatorsforthequalityofteachingandlearn- ing:

    Inputindicators:resourceallocationthatis determinedbyexternalstakeholders(e.g.min- istry).HEIcanhardlyinfluencetheamountof resourceallocation.

    Processindicators:curriculumdesign,didactics, managementofassessment/studentinfrastruc- ture,evaluationetc.HEIcaninfluencethequali- tyoftheseindicators.

    Therelationbetweenexpenditureoftimewhencollectingdatafortheindicatorsandtheeffectsshouldbebalanced.

    Whichadditionalinformationdoweexpectfrom studentsdrop-out-rates?Dowegetmoreinfor- mationthanwhatwealreadyknow?Isthis informationworthinvestingtimeonrespective datacollection?

    Whichinformation/dataalreadyexistandwhichadditionalinformation/datashould/couldbeaddedoradjusted?

    Both,administrationandfacultycollectdata aboutstudentswhogoabroadduringtheirstud- ies.Itshouldbecheckedinhowfarthesenum- bersarecoherenttoeachotherand/orcanbe matched.

    Whichnotionsofresistanceamongstaffhavetobeconsidered?

    Staffresistanceduetooverlappingresponsibil- ities

    Table 3 Challenges of (performance) indicators

  • Chapter 2: Translation of Higher Education Objectives Into Numbers

    43

  • Chapter 3: Reporting: Presentation and Communication of Data and Information

    44

    3 Reporting:PresentationandCommunicationof DataandInformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    3.1 DefinitionofReportingObjectivesforDifferentTargetGroups . . . . . . . . . . . . . . 45

    3.2 ContentofReporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    3.3 OrganisationalConditionsforReporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

    transfercollecteddataintoacoherentandtransparentreportingsystem,

    define reporting objectives for different target groups (e.g. (internal) accountability, strategic decision-

    making,qualityassurance),

    setupareportstep-by-step,consideringaspectssuchastargetgroups,afundamentalplan/actualdata

    analysis,andanappropriatecompositionofvalidandrelevantinformation,

    supportthedevelopmentofareportsystematyourinstitution.Youwillbeabletodetermineresponsibili-

    tiesandfunctions,defineworkflows,deadlinesandreportingfrequencies,aswellasanappropriateformat

    ofreporting.

    On successful completion of this chapter, you should be able to

    Chapter 3

    Reporting: Presentation and Communication of Data and Information

  • Chapter 3: Reporting: Presentation and Communication of Data and Information

    45

    3 Reporting: Presentation and Communication of Data and Information

    3.1 Definition of Reporting Objectives for Different Target Groups

    According toBlohm,a reportingsystem includesallunits, regulationsandactivitiesofahighereducation

    institutionwhichsupportcollecting,analysingandcommunicatinginformationforinternalandexternaluse

    (BlohminGrochla1980,316).Basedonthis,thedistributionandexchangeofinformationiscarriedoutby

    reportswhichincludesummarisedinformationthatrefertoanoverridingaimandaninformationpurpose

    (translatedfromBlohm1974,15).

    Therefore,reportsplayanimportantrolewithregardtoqualityassuranceandenhancementathigheredu-

    cationinstitutions.Theyhelptodocumentevaluatedstatusquosandtodescribeopportunitiesandthreats

    on theway to achieveexpectedperformance levels. Furthermore, they serve accountability purposeson

    achievedoutputs-statusinthecorefieldsofteachingandlearning,researchorservicesbyprovidingafunda-

    mentalbasisfordecision-makingprocesses.

    Qualitymanagerscanbeassignedwithdevelopingsuchreportsorsupportingotherstaffmembersduringa

    reportingprocess.Thisiswhytheyshouldhaveabasicunderstandingabouttheobjectivesofreportingto

    thedifferenttargetgroups.Basedonthis,theyshouldbeabletodesignanadequatereportstepbystep(e.g.

    coordinatingresponsibilities,workflows,deadlines,reportingfrequenciesorformats).

    Inthefollowing,youwillgettoknowdifferenttypesofreportingthatcanbeusedfordifferentpurposesand

    targetgroups.Basically,wecandifferentiatethreedifferenttypesofreporting:standardreports,reportson

    demandanddeviationreports(Hrvath2011,535;Horvth2008,21etseq.;Kpperetal.2013,231etseq.;

    Gpfert2007,3etseq).

    Standardreportsarepublishedinregularlyfixedperiods.Theyarestandardisedinformandcontent,based

    onadefinedsetofinformationneeds(e.g.standardisedteachingreports,reportforthetopmanagement/

    ministry,evaluationreport).Generally,inthiscasetheaddresseehastoidentifyandselecttheinformation

    thatisrelevanttoher/himfromthereportonher/hisown.Oneproblematicaspectofsuchstandardreports

    isthequestionoftheirsignificancewithregardtoanoverarchingpurpose.Duetothestandardisationitcan

    occur, thatcertain informationneedsofanaddresseearenotreportedcorrectly.Or,dependingonwhich

    informationaddresseesselectfromthereport,theycaninterpretwrongorunclearcorrelations.

    Consideringtheseproblems,reportsondemandaregainingrelevanceandcansubstitutestandardreports

    withregardtocertainpurposes.Reportsondemandarenotbasedonstandardiseddata,butaredesigned

    Distribution ofinformation basedon reporting

  • Chapter 3: Reporting: Presentation and Communication of Data and Information

    46

    forspecificinformationdemandsoftheaddressees.Theydonothaveapre-fixedrhythmofbeinggenerated.

    Basedonadatabasethatincludesallrelevantdataforhighereducationmanagement,theaddresseescan

    generatetheindividualinformationneededontheirownwithdirectaccess.Therefore,theaddresseestake

    onamoreactiverole,onlyselectingsuchinformationthatisrelevanttothem(e.g.informationtobeconsid-

    eredinaself-reportofaself-evaluationinteachingandlearning/research).Usingsuchreportsrequiresthe

    addresseestoknowhowtousethedatabaseinordertobeabletogeneratesuchinformationrequests.

    Thethirdtype,deviationreports,servestofocusonplan-actual-deviationsofmanagementissuesthatexceed

    orfallbelowcertaindefinedtolerancevalues.Suchreportsareonlyusedwhennormalprocessesareinter-

    ruptedbyconspicuousdeviationsordisturbancestoreachtheexpectedoutcomes(e.g.non-predictablefallin

    studentsenrolment).Thecontentandformatofthesereportsarenotstandardisednormally.Theaddressees

    can,forexample,bedeansoffaculties,acontrollerorthetopmanagement.

    3.2 Content of ReportingHowcanhighereducationinstitutionsdesignandusereportsadequatelywithregardtotheirpurposesand

    withjustifiableworkload?

    Figure 6 Criteria to design reports (translated illustration adapted from Tropp 2002, 70)

    In the following, it is suggested that some fundamental conditions shouldbe consideredwhendesigning

    reportsforinformationtransferpurposes.

    Designing reports

    Purpose ofReporting

  • Chapter 3: Reporting: Presentation and Communication of Data and Information

    47

    1. Why Reporting? (purpose)

    Reportsareusedtofulfilpre-definedpurposesandthereforearenotanendinthemselves.Areportspur-

    poseisdeducedfromtheinformationneedsoftherespectivetargetgroup.Hence,areportcanbeusedfor

    accountabilityanddocumentationreasons,bothfrominternalaswellasexternaladdressees(Kpperetal.

    2013,230).Examplesincludeprotocols,listsofapproved/not-approvedexaminationsorself-reportsofinter-

    nal/externalevaluation.Furthermore,reportsserveformanagementpurposesandwithitforpreparingand

    controllingdecision-makingprocesses.Forexample,basedonareportoffinancialliquidity,theseniorman-