Week 13 - Lessons Learned

Introduction

Throughout a project's life cycle, we learn lessons and discover opportunities for improvement. Documenting lessons learned helps a project team discover both strengths and weaknesses. It provides an opportunity for team members and/or partners to discuss successes during the project, unintended outcomes, and recommendations for others involved in similar future projects.

Videos

Lecture Material

• Article from Conservation Gateway: A Guide to Capturing Lessons Learned

Assignment(s)

Assignment 4 - The Construction of a Log Cabin

Introduction, continued

Use of lessons learned is a principal component of an organizational culture committed to continuous improvement and adaptive management. Lessons learned mechanisms communicate acquired knowledge more effectively and ensure that beneficial information is factored into planning, work processes, and activities. The mechanisms or processes used to collect, share, and disseminate lessons learned may vary, but in general such a process is comprised of five main elements: defining the project, collecting information, verifying applicability, storage, and dissemination. The below figure is a generic representation of the lessons learned process.

Question: The design of microchips often rely upon a library of standard cells. These cells are typically fundamental combinational logic gates such as AND gates, OR gates, etc... and are typically fundamental sequential logic gates such as D flip-flops, the latch flip-flop, the JK flip-flop etc... The standard cell library might also contain circuits that are unique but used in many of the company's microchips, such as a digital signal filter. Do you think it is a good idea to have a standard cell library for software? What do you call such a library?
Toggle Answer Answer: It is a very good idea to have such a library for software. We use such libraries all the time from the C/C++ standard libraries. But we should also make our own libraries. The process of collecting libraries for reuse is called software reuse.

Basic Lessons Learned Process

1. Define the Project. This step is the initial step wherein the need for lessons learned is identified and the process and team through which the lessons will be collected is established. It is important to establish the specific need and purpose for lessons, the audience for the product, and which individuals should comprise the project team.
2. Collect. The collection process involves the capture of information through structured and unstructured processes such as project critiques, written forms, and meetings. The collection of lessons may come from as many sources as an organization is willing to solicit.
3. Verify and Synthesize. This process serves to verify the accuracy and applicability of lessons submitted. Domain or subject matter experts may be involved in coordinating and conducting reviews.
4. Store. The storage aspect of lessons learned usually involves incorporating lessons into an electronic database for future sharing and dissemination.
5. Disseminate. The final element, and the most important, is the dissemination of lessons learned, since lessons are of little benefit unless they are distributed and used by people who will benefit from them.

Question: Who benefits the most from the lessons learned knowledge base?
Toggle Answer Answer: Inexperienced project managers and project leads benefit the most from this. There are many project management tasks that require experience, such as risk analysis or scheduling. An inexperienced project manager/lead can benefit from the mistakes of others without having to make these same mistakes him/herself.

Deciding on a process for collecting lessons learned

There are primarily two different approaches to capturing lessons learned, and each project team must decide which approach, or perhaps a combination of approaches, works best for their project.

1. Integrated. The simplest approach is to incorporate lessons learned early, regularly, and consistently through regular project reporting, or within the context of the initial management plan. Capturing lessons learned would be part of the regular annual or semi-annual reporting cycle and may even be embedded in the initial project management plan.
2. Post-Facto. The more detailed, complex approach is one which requires a thorough examination of the project post-facto. This is sometimes done in projects reactively or as an afterthought when project managers realize things could have been done differently. However, many organizations who have invested heavily in a project over a long period of time, or who are interested in replicating similar projects are willing to spend the time and money necessary to improve future efficiency. While more resource-intensive, this approach offers the benefit of bringing project members and partners together for an extensive look into the operations, successes, and shortcomings of the project.
   
   
3. Combination. While it is preferred to begin with the integrated approach wherein lessons learned are part of the initial project plan and team members meet regularly to capture lessons learned, it is also helpful to bring together key partners and stakeholders with the project team and the end of or during a project. This allows for a broader analysis and may help to build a sense of collaboration and communication within the partnership or group responsible for project implementation. The below table outlines how this combined approach may unfold.
   
   

Speech Synthesis on a Mobile Device

You are running a project which integrates speech recognition into barcode scanning. The context is a warehouse. Normally, the user will scan items in a warehouse, entering text whenever necessary to navigate through the scanner menus. This process can be sped up if the user can free his/her hands by giving verbal commands to the scanner, rather than navigating by keyboard. Therefore a speech recognition unit is required.
The speech recognition unit requires shared memory to communicate with processes. Digitized audio data is passed to the speech recognition unit from a microphone through shared memory and letters/words are returned from the speech recognition through another part of shared memory.
Question: What are some things that could go wrong here?
Toggle Answer Answer: Since this is a mobile device, RAM might be limited. The speech recognition unit might not recognize speech correctly, especially if the user has an accent. There might not be enough CPU power on the mobile device for the speech synthesis unit.

In reality, neither memory nor CPU power was a problem. The shared memory allowed fast communication between the speech recognition device and the processes requiring speech recognition. The problem was in speech recognition itself.
Question: How to resolve this? What lessons were learned here?
Toggle Answer Answer: Resolution - Perhaps it is possible to train the speech recognition device to recognize a certain voice. Each warehouse worker owns a scanner, therefore the speech recognition device on each scanner could "learn" how to understand the worker using that scanner. Noise cancellation hardware may be added to cancel the noise from the warehouse. Also, perhaps update the software. Also, look at the microphone. Perhaps update the microphone.
Lessons Learned - USE PROTOTYPING before engaging in a project. Test the speech recognition device with a crude prototype to see if it is as effective as advertised. Also, perhaps purchase a much more CPU intensive, complex speech recognition software and put it on a central server for all mobile devices to use. Such a speech recognition unit can be tested before being purchased and implemented.

See also Risks and Mitigation for the speech recognition barcode scanner.

Process Details

In identifying the project team, it is important to build initial engagement from all key players who will be involved in advance of the project. Include the project manager, the project team, and the key stakeholders in the lessons learned exercise. Select staff with specific expertise or knowledge of the project and other needed skills, such as communication and writing.

Multiple resources exist for creating questionnaires and surveys, and designing interview questions. Whatever method you are using, concentrate on obtaining information in four general areas:

1. What went well?
2. What didn't go well or had unintended consequences?
3. If you had it all to do over again, what would you do differently?
4. What recommendations would you make to others doing similar projects?

• Were the project goals attained? If not, what changes need to be made to meet goals in the future?
• What surprises did the team have to deal with?
• What project circumstances were not anticipated?
• Did you develop any useful workarounds or solutions to problems that cropped up during the project? Document the details in a way that will make sense later.
• For any problems that went unresolved what preventative measures can you invent now that can help things go more smoothly next time?
• Are there any new “best practices” you can derive from this project? Note anything that went so well - and now seems to be so thoroughly “road tested” - that you would want to repeat the positive experience next time.
• Can you create an easily accessible repository for lessons learned and best practices you have documented? This could be a database, website, or even a simple document.

Once again, look at the speech recognition project and answer the above questions.
Toggle Answer What went well?The barcodes were scanned successfully. CPU power and RAM were not a problem.
What didn't go well? The speech recognition itself.
What would you do differently next time? Prototype. Spend more money on a better speech recognition unit that can be put on a server. This can be tested easily before purchase.
What recommendations would you make to others doing a similar project? Use server based software. Test the product well yourself before purchase.
Did you develop any useful workarounds or solutions to problems that cropped up during the project? Yes. We had each scanner learn how to recognize its user. Also, use noise cancellation.
Are there any new “best practices” you can derive from this project? It's better to spend more for quality. We have to thoroughly test a product for our use-case scenario before purchasing it. Also, better scheduling for our project.

Suggested Lessons Learned Case Study Format

The Concorde Project

Consider the Concorde project. The origins of the Concorde project date to the early 1950s, when Arnold Hall, director of the Royal Aircraft Establishment (RAE), asked Morien Morgan to form a committee to study the supersonic transport (SST) concept. The group met for the first time in February 1954 and delivered their first report in April 1955.

At the time it was known that the drag at supersonic speeds was strongly related to the span of the wing. This led to the use of short-span, thin trapezoidal wings such as those seen on the control surfaces of many missiles. The team outlined a baseline configuration that resembled an enlarged Avro 730 fighter jet.

This same short span produced very little lift at low speed, which resulted in extremely long take-off runs and frighteningly high landing speeds. This would have required enormous engine power to lift off from existing runways.

The original programme cost estimate was £70 million before 1962, (£1.39 billion in 2020). The programme experienced huge cost overruns and delays, with the programme eventually costing between £1.5 and £2.1 billion in 1976, (£9.44 billion-13.2 billion in 2020). This extreme cost was the main reason the production run was much smaller than expected. The per-unit cost was impossible to recoup, so the French and British governments absorbed the development costs.

On 10 April 2003, Air France and British Airways simultaneously announced they would retire Concorde later that year. They cited low passenger numbers following the 25 July 2000 crash, the slump in air travel following the September 11 attacks, and rising maintenance costs: Airbus, the company that acquired Aerospatiale in 2000, had made a decision in 2003 to no longer supply replacement parts for the aircraft. Although Concorde was technologically advanced when introduced in the 1970s, 30 years later, its analogue cockpit was outdated. There had been little commercial pressure to upgrade Concorde due to a lack of competing aircraft, unlike other airliners of the same era such as the Boeing 747. By its retirement, it was the last aircraft in the British Airways fleet that had a flight engineer; other aircraft, such as the modernised 747-400, had eliminated the role.

For more details, see Concorde.

Question: What are the lessons learned?
Toggle Answer What went well? From an engineering point of view, the plane performed as expected.
What didn't go well? The overall business case. Are people in such a rush? Will they pay so much to save a few hours of travelling time? From an engineering point of view, the plane was somewhat unstable when landing or taking off.
What would you do differently next time? Do marketing first. Don't do something just for the heck of doing something.
What recommendations would you make to others doing a similar project? Balance engineering with marketing. Balance engineering with the needs of society. Balance engineering with the environment. Also, make upgrades to your product even if there isn't any competition.
Did you develop any useful workarounds or solutions to problems that cropped up during the project? Mass production would bring the price of the Concorde down, reducing ticket price Fly at a higher altitude to reduce the effect of the sonic boom. Build a muffler to reduce the noise of a sonic boom.
Are there any new “best practices” you can derive from this project? Engineering needs to be a balanced profession. It is not about building the best thing. The business case, needs of society, the environment, etc... need to be taken into consideration.
For more on lessons learned, see The lesson of Concorde is that we can't go any faster and Lessons from the Concorde Jet: Innovate and Disrupt.