|
Paper Written For Publication in International Journal of Operations & Production Management
Development of a Methodology for Assessing the Education Requirements for the Effective Implementation of ERP / MRP II Manufacturing Control Systems
Austen K Jones
A K Kochhar
M W M Hollwey
ABSTRACT
This paper outlines the need for a production planning and control (PPC) system user education requirements specification methodology. After the need for a methodology has been outlined, the various methods of education and training requirements specification are outlined and the developed methodoloy is described. The subsequent testing and validation of the requirements specification methodology are discussed.
1 Introduction
The planning and control of production activities represent important tasks that manufacturing companies need to address competently if they are to prosper in their respective markets [Muhlemann et al 1992]. There are a variety of approaches that manufacturers can utilise in the management of the operations of their production facilities and most of these approaches are computerised.
Implementation of a computerised Production Planning and Control (PPC) system is a very complex undertaking that requires much change to job roles and to existing procedures and systems within the company implementing the PPC system. These factors combine to make implementation difficult and despite the fact that computerised PPC systems have been being implemented for a good many years a significant proportion of implementations fail to meet customer expectations [Browne et al 1996, Brown 1994]
Research has been undertaken analysing the PPC system implementation process [Ang et al 1995, Burns et al 1990, Cox & Clark 1984, Duchessi et al 1988, Turnipseed et al 1992]. Some studies have determined the requisite factors for implementation success while others have set out to elicit the barriers to implementation success. These studies highlight a number of issues and factors that require addressing if systems implementation is to be effective.
A main group of factors is staff education and training [Duchessi et al 1988, Udo & Ehie 1996]. Education & Training, as a PPC implementation success factor, is itself composed of a number of different issues, including [Wallace 1985, White 1980]: · Timing of the education / training in the overall context of the implementation project · Disproportionate focus on training at the expense of education · Budgets allocated to staff education and training · Delivery of the education and training · Course having the appropriate and adequate content · Course being attended by the relevant personnel
The last two points are generally the most problematic since their solution is non-trivial, i.e. it requires information to be gathered and analysed. Performing an education needs analysis is one way to ensure that the relevant personnel attend courses of the appropriate content. Figure 1 indicates the vital stage of requirements specification in the context of the education / training process [Kenney & Reid 1988].
2 Need for an Education & Training Requirements Specification Framework
The fact that there has been a dearth of research that has attempted to address the PPC implementation problem of users receiving inadequate or inappropriate education and training is surprising given that user education and training are widely accepted as being major obstacles to successful PPC implementation [Udo & Ehie 1996, Duchessi et al 1988].
The development of an effective methodology for accurately specifying a PPC system user’s education and training requirements is justifiable and necessary for the following reasons:
1. The majority of PPC system implementations are unsuccessful, i.e. they fail to meet expectations [Browne et al 1996, Burns & Turnipseed (1991), Duchessi et al (1989)]. These failures are not without cost, i.e. lost opportunity, personnel time, money and so on. As such, research which broaches a PPC system implementation problem should be of significant benefit to the manufacturing industry in the UK.
2. Specifications of PPC system users’ education and training requirements that were accurate would reduce the risk of people receiving inappropriate / inadequate education and training and, therefore, increase the chances of implementation success. An approach that can make implementation easier and less problem prone would be very useful to manufacturing companies attempting a PPC system implementation, as well as companies/ system houses responsible for implementing the systems.
3. No research work has been undertaken to attempt to develop an effective approach that can accurately establish the education and training requirements of PPC system users such that they can receive adequate and appropriate education and training.
To summarise, research work concerned with developing an effective approach to the specification of education and training requirements would be useful since it would assist the process of PPC system implementation. As such, the tool would be of importance to companies attempting to implement such systems.
3 Methodology for Specifying PPC System User Education and Training Requirements3.1 Education and Training Needs Analysis Methods
The goal of education and training needs analysis is to determine the needs as they currently exist within the company or as they have the potential to develop, for instance, needs that may arise due to the implementation of a new production planning and control system.
The more precisely a person's education and training needs can be specified the more focused the education and training interventions can be. This ensures that the education and training will actually meet the original need and in a cost-effective way [Bee & Bee 1994]. McClelland (1994) considers there to be four major methods of gathering data for education and training needs analysis, these being: 1. Questionnaires 2. Individual interviews 3. Focus groups 4. On-site observations These methods and their respective pros and cons are discussed below.
3.1.1 Questionnaires
The questionnaire is the most prevalent method of gathering data for the purposes of needs assessment. The reasons for this are that they can provide a cost-effective and reliable means of gathering information that can be qualitative as well as quantitative. Survey questionnaires can provide accurate and relevant data through thoughtful design, testing and administration [McClelland 1994].
Other advantages of the questionnaire are that it can easily be administered to a large population and does not require people from different places to assemble together. Questionnaires are a non-intrusive method of data collection as individuals are free to complete them in their own time and in an intimidation-free setting. Further, biasing, which is often a problem in interviews due to the nature of the sender - receiver relationship (i.e. interviewer - interviewee) is not an issue with questionnaires. On the down side, the design and piloting of questionnaires is quite an involved process and the information they yield may not be as rich as can be achieved using other methods, for instance, interviews.
3.1.2 Individual Interviews
Interviews are an important method of data collection as they allow direct interaction between the person trying to determine the training need and the person for whom the education / training is intended [Bee & Bee 1994]. However, much care needs to be taken by the interviewer that the purpose of the interview is not obscured by the interviewee or the topics of discussion. For instance, it is not uncommon for interviews to drift into gripe sessions or for the interviewee to focus on wants rather than needs [McClelland 1994].
While the information yielded from interviews can be an excellent source of qualitative data and useful for the purposes of education and training needs analysis [Boydell & Leary 1996] they do have some important drawbacks. Face-to-face interviews are an expensive method of gathering data and are very time-consuming. Another disadvantage of interviews is the need for an experienced interviewer.
3.1.3 Focus Groups
Focus groups, despite becoming popular in the marketing profession, are infrequently used as the sole method of data collection for the purposes of education and training needs assessment. McClelland (1994) suggests that focus groups should comprise of between 8 to 12 people that represent a cross-section of the training population. Advance preparation of group participants is essential if the outcome of the data collection process is to be constructive and objective.
The problem of bias is often associated with the data gathered from focus groups. This can arise for a number of different reasons, for instance, the moderator asking leading questions; the group being dominated by one or two participants; the moderator failing to keep the discussion on the relevant subject areas, or by not having well-defined goals for the group to achieve [McClelland 1994].
Focus groups are a useful method of gathering qualitative data that are best used in conjunction with another, more quantitative, data collection method. However they are relatively costly, need to be moderated by an experienced professional, and provide feedback on education and training needs that is, at best, broadly defined.
3.1.4 On-Site Observation
The value of observing work tasks has found considerable acceptance and success in the industrial engineering profession [Boydell & Leary 1996, Bee & Bee 1994]. Data provided by such observations is generally qualitative although it may produce quantitative data as well. The main strength of on-site observation as a data collection method is that the data collected is empirical in nature. Where general observations are being used to determine potential training needs, however, the data provided are generally not significant enough to draw conclusions with any degree of certainty. As such, it is normally necessary to incorporate on-site observations with another type of data collection method which yields quantitative information.
The principle behind on-site observations is to perform a job analysis (gain an understanding of the parameters of the job itself) and then a task analysis and then to determine the skill / knowledge / experience levels required such that the individual can complete the task successfully.
The different methods of collecting data for the purposes of education and training needs analysis reported in the HR literature have been outlined in the previous sections. Despite the importance of education and training needs analysis the education and training needs of personnel in manufacturing companies at the time of PPC system implementation, if done at all, are assessed in the most primitive of ways. The standard method of determining who is taught what rely on consideration of the person's job title / department and the standard package courses that are offered by PPC system seller [Jones et al 1998, Roberts 1986, Iemmolo 1994]. For instance Roberts (1986) prescriptively instructs: "Determine the training needs required by a functional area of the organisation, as they relate to each system module"
This practice is nothing more than a rudimentary method of allocating bodies to courses and makes no attempt to determine the requirements of attendees or the suitability / relevance of course content. The following section outlines the development and validation of an innovative methodology that was developed to make the requirements specification process more credible, accurate and scientific.
3.2 Development of the Education Requirements Specification Methodology
In deciding which of the four above methods were best suited to determine the education and training requirements of staff using / implementing production planning and control systems a number of criteria were drawn up. These criteria were: · The output must be sufficiently detailed to be useful for the intended purposes · Data collection must be relatively quick (inexpensive) · Data collected must be as accurate as possible · Data must be easy to collect · Data collected from one person must be easily comparable to that collected from another person · Data collection method must be flexible enough to be effective in a number of different contexts (i.e. job roles and companies) · The resulting education and training requirements specifications must be accurate
Questionnaires were selected as the best method of data collection since even though they required much up front preparation they would provide a quick, detailed and potentially accurate method of data collection that could easily be applied to different people in different companies.
A task analysis-style questionnaire and a questionnaire to determine the requisite skills and knowledge were developed. As such the questionnaires consisted of a list of tasks or Activities and associated with each task there was a particular Knowledge profile. The developed questionnaires were tested by 32 experienced PPC system users from different manufacturers and resulted in a list of over 140 generic Manufacturing Resource Planning (MRP II) focused activities with associated knowledge profiles. Figure 2 shows how a person's job role is comprised of a number of discrete tasks and how each of these tasks has its own requisite skills / knowledge profile associated with it.
The needs assessment framework is based on the following logic [Jones et al 1998, McClelland 1994]: · Staff in manufacturing companies have specific Job roles · Each Job role is responsible for a number of different tasks or Activities · To undertake a particular Activity successfully an individual needs to possess certain skills and have knowledge of certain information or Concepts
The methodology has a further level of detail built into it so as to enhance the accuracy of education and training specifications that use of it yields. The methodology incorporates "Involvement levels" which have an impact on the "Knowledge levels" that are required of each Concept. There are four Involvement levels ranging from None to Detailed: None: No involvement in the Activity Limited: Being involved in an Activity indirectly or in a superficial sense. Some: Being involved in an Activity which does not constitute a major part of the person's job description. The involvement may be in some sort of assisting or overseeing role. Detailed: Being actively involved in an Activity, generally on a day-to-day basis. The Activity is likely to be a significant part of this individual's job description
Likewise there are four Knowledge levels also ranging from None to Detailed: None: No knowledge required of the Concept. Awareness: Conceptual background information whereby an individual has some knowledge of the topic under consideration. Overview: The level of understanding that a person requires to effectively manage the day to day activities. (This may be higher level conceptual information as opposed to a thorough understanding of the topic.) Detailed: The level of understanding required of a particular topic by an individual whose job activities necessitate a thorough understanding of the topic.
The Involvement levels allow detail regarding the importance and frequency of performing an Activity to be incorporated in the requirements specification process.
The Activities and Concepts that were compiled in the framework are relevant to production planning and control systems and focus on manufacturing resource planning. Determining an individual's Activity profile, with Involvement levels, implicitly defines their respective Required knowledge profile (i.e. the set of Concepts with associated required Knowledge levels that are relevant to the individual). Comparing the person's Required knowledge profile with their Actual knowledge profile yields education shortfalls which represent the education / training needs of that individual.
The Required Knowledge profiles, i.e. the benchmarks against which the existing knowledge levels of individuals are measured to determine their requirements, were developed using empirical data collected from a number of manufacturing companies. The process of compiling the Required Knowledge profiles and populating the framework with valid Activities and Concepts is discussed in the following section.
4 Population of the Framework
The framework population stage of the research consisted of two distinct phases, namely: 1. Initial population of the framework (from the available literature) 2. Empirical refinement of the framework These different stages are discussed in turn.
4.1 Initial population of the framework
Once the structure of the education requirements specification framework was in place it was then necessary to populate the framework with relevant and meaningful information, i.e. PPC Activities and Concepts and the associated links between them. This stage involved a thorough analysis and synthesis of the available literature.
4.1.1 Activities & Involvement levels
Building the preliminary catalogue of Activities involved searching a number of textbooks and papers that described manufacturing planning and control approaches including MRP II [e.g. Vollman et al 1984, Greene 1987, Muhlemann et al 1992]. Much attention was also paid to existing work in related fields that had included the elicitation and synthesis of manufacturing activities, in particular, the work of Kochhar et al (1994) which was concerned with performance indicators and benchmarking of PPC systems and also the work of Howard et al 1998 which was concerned with functional requirements specification of PPC systems. Using these sources helped to ensure that the Activities were generic, accurate and comprehensive and, as such, were in an appropriate state to allow them to be tested and refined by industrialists. Table 1 gives examples of the Activities in the framework.
Table 1: Example Activities
As described in the previous section a four-group classification system was developed to categorise individuals' Involvement levels with Activities. A four level scale was decided upon as this was deemed to be the optimal compromise between having sufficient granularity to make the requirements specification accurate and relevant and having too many levels such that the differences between adjacent levels were unacceptably vague.
4.1.2 Concepts & Knowledge levels
A list of over 340 Concepts relevant to the Activities was also compiled from the available literature. For instance, scanning the indices of a number of textbooks highlighted Concepts for inclusion in the framework [e.g. Technical Committee M/136 1975, Wallace 1985, Cox 1995, Greene 1987], these were then defined and added to the list.
During this stage of development it became apparent that there was a degree of terminological differences between the different information sources that were used. While this was by no means great it was significant enough to warrant an amount of redundant information to be incorporated in the framework, i.e. including commonly used synonyms in the framework, for example Safety stock - Buffer stock and ABC analysis - Pareto analysis.
As for the Involvement levels discussed in the previous sub-section, a four-group classification system was used to categorise individual's Knowledge levels of Concepts. The reasons for having four Involvement levels were equally applicable to the decision of the number of Knowledge levels of Concepts that would be most appropriate.
4.1.3 Activity - Concept Relationships
Prior to the empirical population of the framework tentative relationships were set-up linking Concepts with Activities. That is, a set of Concepts was associated with each Activity in the framework thus forming a mini requisite knowledge profile for each Activity.
Each Activity was analysed in turn and consideration was given to the: · Activity's inputs and factors which affected it · Activity's outputs and implications By doing this it helped to ensure that the tentative sets of Concepts associated with Activities were comprehensive.
By way of an example of the relationships that were compiled in the framework during this phase of the research consider the Activity: "Create a formal production plan" In the framework this Activity is associated with a number of different Concepts, these are listed in Table 2 along with their respective definitions and reasons for being associated with the Activity. Note: that the list in Table 2 was the tentative list and was added to during the empirical phase of framework construction.
Table 2: Example of Concepts Tentatively Linked to an Activity
During this stage no attempt was made to ascribe strengths, i.e. Knowledge levels, to individual Activity - Concept relationships as this was the purpose of the empirical framework construction phase, discussed in the next section.
4.2 Industrial refinement of the framework
The principal aim of this phase of the research project was to develop the framework such that its contents, i.e. Activities, Concepts and the relationships between them, were accurate and complete for the typical PPC system used by manufacturing companies.This was to be achieved using information gathered from people in a number of different types of manufacturing companies using PPC systems. To ensure that coverage of the contents of the framework was as thorough as possible it was important that participants were from a range of different departments, holding a variety of different job roles.
An education requirements specification framework that was constructed using information and knowledge gathered from good-practice PPC companies would be more accurate and useful than a similarly developed framework that used data collected from companies less successful in the usage of their PPC systems. As such, the principal criterion for selecting companies for participation was that they were "good-practice" PPC companies.
Four good practice manufacturers were recruited to help with the research (the characteristics of these companies are shown in Table 3). Senior contacts were established in each of the companies and these contacts acted as the project co-ordinators at the respective sites. The co-ordinators assisted with the different stages of the framework population process.
Table 3: Company Characteristics
After initial contact was made with the companies assisting in the developmental phase of the research, the collection of the necessary data was undertaken in two stages, namely: 1. Activity validation and determination of the Activity Involvement profiles of the participants 2. Concept validation and determination of appropriate Activity - Concept relationships and associated Required Knowledge levels 4.2.1 PPC Activities
The four companies that helped to develop and populate the education requirements framework were manufacturers of: · Off-road vehicle transmissions · Image processing hardware · Satellites · Electrical connectors The companies were good-practice PPC system users which was important as it meant that data collected from them was of a higher quality than would have been collected from less effective users of PPC systems. This also served to make the results of the development phase of the research more credible.
Having identified a project co-ordinator at each of the four participating sites, copies of the Activity Involvement Questionnaire were distributed to the companies. The questionnaires listed the tentative Activities and requested individuals to list other Activities that were relevant to PPC that were not included in the list.
A total of 32 people from the four companies contributed to the compilation of the education requirements specification framework. To ensure breadth of Activity coverage the participants were from a range of different departments in the company, for instance, there were representatives from Sales, Production, Finance, Engineering, Purchasing and Materials. As such, job titles of the individuals included: · Production Supervisor · Production Manager · Production Engineer · Shift Supervisor · Applications Engineer · Systems Engineer · Business Manager · Management Accountant · Engineering Manager · Materials Co-ordinator · Buyer · Operations Systems Manager · Head of Manufacturing · Business Systems Manager · Chief Accountant · Planning Scheduler (Manufactured) · Purchasing Manager At the time of taking part in the research project the people from the four companies had been in their job roles (as opposed to with the company) for an average of nearly 5 years, minimum 1 year, maximum 26 years.
The forms that were received contained many corrections to the tentative Activities and indicated that the 32 people were involved with a total of 1,820 Activities 605 of which were at a detailed level. This meant that over 98%, i.e. 141 of the 143, of the Activities in the tentative framework were valid and relevant and that 97% (139 from 143) of them (i.e. those people were involved with at a Detailed level) would have their provisional relationships with Concepts judged and added to.
Further to these results there were 59 suggested additions to the list of Activities in the framework. Before adding any Activities to the framework it was necessary to consider whether the suggested Activity was: · relevant to PPC systems - some of the suggested Activities were important to the operation of the company but were not necessarily relevant to PPC systems, for instance, "Preparation of sales promotion literature." · already in the framework - this duplication was due to the length of the questionnaire, i.e. by the time people got to the end of the questionnaire they may have forgotten reading a particular Activity (especially if the questionnaire was not completed in one sitting). If an Activity that was suggested for addition to the framework was found to be a duplicate then it was instead considered to be a suggestion for rewording and the appropriate action taken. The Activities that passed both of these checks were then incorporated in the framework and tentative relationships with Concepts were outlined as per the process described in Section 0.
The information gathered using the Activity involvement questionnaire was used to improve the accuracy and comprehensiveness of the list of Activities in the framework. The information regarding the individual's level of involvement with each Activity was used to tailor a second questionnaire which was concerned with determining what Concepts a person would need to know in order to carry out an Activity competently.
4.2.2 PPC Concepts & Relationships with Activities
The Concepts and Activity - Concept relationships in the framework were validated and compiled using a second questionnaire. This tailored questionnaire listed the Concepts (and definitions) that were tentatively associated with the Activities that the individual had indicated they were involved with at a detailed level. Only people involved with an Activity at a detailed level were requested to recommend the Concepts that a person performing that Activity would need Knowledge of.
Questionnaires took the form of a list of Activities. Each Activity was followed by a list of Concepts which the person completing the form was required to indicate what level of Knowledge of the Concept would be required by a person performing that Activity at different levels, i.e. none, detailed knowledge and so on. At the end of the list of the Concepts there was a gap where the participants were asked to include any additional Concepts that they considered relevant to the Activity. All of the Concepts that had featured in the questionnaire were listed and defined at the end of it.
These tailored questionnaires were posted to the relevant people in the four companies for them to complete and 26 completed forms were returned for analysis. This represented a total of 3,772 discrete Activity - Concept relationships with associated Knowledge levels.
The collected data were entered into a database to ease the task of analysis. When in the database the responses could be analysed comparatively, i.e. the required Knowledge level that one person ascribed to a particular Activity - Concept relationship could be directly compared with that suggested for the same relationship but by a different person.
In order to conduct a meaningful analysis of the data it was necessary to convert it to numeric data. Numbers were ascribed to the four Knowledge levels as indicated: · None 0 · Awareness 1 · Overview 2 |