Project training evaluation: Reshaping boundary objects and assumptions

Comparison of data coming from case studies with current theory and the ability to create novel ideas and new theory from the data itself.Next, open-ended questionnaires explored differences in perceptions pertaining to knowledge, learning, culture, and the nature of organisations, and the role of information for project communication and coordination. For each word, (knowledge, learning, data, information, culture, organization etc.) respondents were asked to give their definition and then use the word in a sentence demonstrating their interpretation.

The subject of the research and its epistemological underpinning align it with a qualitative approach as the focus is exploration and development of new insights.A comparative case study methodology was adopted in order to undertake an in-depth investigation into a specific set of circumstances in a particular context. These were instrumental and exploratory case studies, in that their purpose was to provide insights into an issue that will enable a deeper understanding of the role of boundary objects for identifying project training needs and subsequent evaluation of their outcome effects.

How did participants find the learning experience? Did the intervention provide the learning the participants expected?Did participants change their behaviour because of the training or intervention? Was the training aligned with the strategic development aims of the organisation and did it deliver the intended learning outcomes?•Appraisal•Levels of learning•Targeting and rewarding of knowledge development

Four organisations were selected as two matched pairs: one pair from the manufacturing sector and one from the services sector. All the cases are companies which are publicly listed and are of a similar capitalisation, workforce size and using projects as a means of achieving their contracts and in-house initiatives. Companies 1 and 4 are manufacturing organisations using projects for non-process activities and inhouse improvement initiatives. Companies 2 and 3 are projectbased service organisations. Company 2 and Company 4 are nationally acclaimed learning organisations (LO). Learning is strongly encouraged and employee learning is fed back into the organisation as increased innovative capabilities.

Three sets of data from documentary reviews, open ended questionnaires and semistructured interviews were drawn from the cases. Samples of organisational literature and documentation were analysed to understand the organisational views of their own learning and knowledge development practises. Written definitions of professional development, learning, knowledge and the differentforms of processes and their implementation were also examined.

How to drive learning up the quadrant to achieve behaviour and attitudinal changes that can truly contribute to the organisation's overall results.How personal learning can benefit other colleagues, the project, other projectsand the organisation.

Training interventions appear to be successful but references to gaining information via training and education did not include sharing or developing others either from a lack of means or motivation.Performance improvement was hard to assess.Greater attention should be paid to the front-end of training, not just computing the economic value or benefit of training, but ensuring true value-added by developing a shared understanding and identification with the organisational goals and working out a strategy to manage their achievement through appropriate boundary objects (e.g. benefits realisation maps and balanced scorecards).

All four participating organisations operated a formal performance appraisal system that led to the output of a training and development schedule for their employees.The findings suggest an organisational mind-set that pre-empted new learning by developing boundary objects that were too specific not suitable for complex issues.Close examination of the types of boundary objects in use revealed that the boundary repositories, forms, objects and models were similar across the four companies. Participants acknowledged the importance of keeping well informed in terms of the latest tools and techniques and the need to learn from project lessons but could not provide insight into the mechanisms for assessing learning and application in subsequent projects.There was no specified methodology to evaluate their impact on team cohesion, project delivery and outcomes. Instead, cost driver meant that ‘value-for-money’ related to how cheaply could the organisation procure and offer their employees project training.The levels of learning were analysed across all four companies and, despite the LOs' expressed focus on clear objectives and personal development pathways, the levels of learning being demonstrated were similar.Training interventions appear to be successful but references to gaining information via training and education did not include sharing or developing others either from a lack of means or motivation. Participants were aware of what went wrong in their projects and knew how to fix problems generally, but did not always consider the Know Why. Knowing what and how without addressing the underlying causes can lock project members into a crisis-management cycle.How knowledge can ‘transfer to the job’ remains unknown.

To propose using alternate boundary objects to incorporate reflective practises that can increase the utilisation of learning by embedding it in specific contexts. Helping project workers to think differently about the purpose and outcome of training attendance.Ensuring effective talent management for managers and developing managers with the added confidence and capacity to question the status quo and able to objectively evaluate the relevance of new frameworks and tools for better problem solving.