I’ve been sitting on this post for nearly two years. I originally developed the content for internal discussions within our Standards Department as the result of an occurrence on a check; then further developed it as I discussed the issues raised with my fellow Checkers; along with Checkers and Standards Managers from several other airlines. Interestingly while the issues were common across other airlines – there is a wild divergence in how far down this path various Airline Standards Organisations have gone. For myself I couldn’t publish this content while in a Standards Management role; subsequent to that I’ve been working and re-working the following until it’s at a point where I’m not happy with it – but it’s going out anyway. In truth I think the following is best suited to a discussion during a ground training day while upgrading new Check Captains. However before any discussion is entered into – the Standards Management team needs to consider the implications carefully and thoughtfully develop and document policy. The following content is also relatively complex (unfortunately) for which I apologise to anyone reading this not directly involved in Aircraft Training/Checking. That said – these issues could easily be mapped across to just about any industry seeking to train and maintain standards within an operational regime.

Update 06Jan16 : A friend of mine in a management role at a UK airline read this and asked a follow on question; question and answer at the bottom.

Training vs Checking

An airline standards organisation is typically responsible for both the training of pilots that takes place in the airline, as well as the checking of the standards of those pilots. As a generalisation, training is typically provided for the purposes of achieving a satisfactory standard in a subsequent check; rarely is training provided to qualified pilots as a means to an end (unfortunately). Such a following check can be at the end of a long course of training – such as a new aircraft type rating transition course; or following a short one – such as a single day of training in the simulator every six months followed by the Check simulator session. A Check event can also come to crew without any preparatory training – such as an annual line check in the aircraft.

Note : Some airlines have split Checking from Training into separate departments with separate lines of reporting for the Check and Training Captains. The Training Department is seen as a service industry, the service being the delivery of training and the product being the standards of the airline’s pilots. The Checking department is seen as an independent quality assurance mechanism, ensuring the standard of the product and providing improvement feedback to the Training Department based on assessment of the product – the standards of the pilots.

Having worked under a system like this (as well as the more traditional combined training/checking department); I like the split, not in the least of which because when you combine the two the Training often becomes subservient to the Checking; whereas I believe it should be the other way around, or in the very least equal. The biggest failing of the split system I believe is when you end up with Checkers who never train. Maintaining your training skills is crucial as a Checker, and it’s not something you can do effectively during a check.

Training

Training is typically characterised by published lesson plans so the student(s) are fully aware of what they will be expected to do, and an open environment in which questions are asked by all involved – and equally, answered by all involved, before during and after the training.

The training is conducted by a Training Captain (or exceptionally a Training First Officer); and input/feedback/instruction (verbal and otherwise) from the trainer to encourage the student towards higher proficiency, is manifest. A good Trainer pitches the level of input for each student not just to push them towards at least the minimum required standard; but to improve each student’s own personal standard as well. Training can be of the non-recurrent type, typically for the purpose of gaining a new qualification – or of a recurrent nature, the latter being essentially training provided against tasks for which the student is already qualified to do, usually be followed thereafter by a recurrent Check event.

Note : The issue of placing training before assessment is a hot topic of discussion. Many airlines have moved to a “First Look” concept where pilots are checked before they are trained. While this may seem overtly unfair, this is because you’re coming at it from the wrong end of the stick. The purpose of recurrent training is not to prepare you for the check – but to improve your proficiency as well as address any shortcomings that may have developed in your standards.

The purpose of the check is not to assess whether the training you were given was adequate – it’s to ensure you are meeting the standard required to fly the aircraft safely, and to identify areas in which you need, or would benefit from additional training. Hence the ideal paradigm is firstly a check in which your ability to perform straight off the line is assessed; then you are given training which should be driven at least in part by the assessments taken during the check.

The content of the training and checking being delivered to pilots is slowly moving towards a paradigm where that content is driven by the performance of the airline’s pilots during checking. As the data from checking identifies that a particular manoeuvre is done poorly by a statistically significant number of pilots – more of the focus of training and assessment is introduced in subsequent recurrent phases of training onto that poorly performed event. This works best when your Check is done before the training, when the vagaries of your line pilots is not masked by the training delivered before a check.

Occasionally events are repeated in training to achieve a better outcome or illustrate a technique; moreover there is typically no limits on the number of repeats available to the student in order to reach a good standard (within the limits of time, etc). Checks on the other hand usually offer a limited number of “Repeats” within the check to allow a candidate to demonstrate competency only after having already demonstrating a lack of competency in that event within the check. More, an event can only be repeated only once, and must achieve a higher standard than was required at the initial attempt.

While Trainers assess performance and grade accordingly – including “failing” grades (which should be termed as a “Failure to Progress” or “Not Ready for Check” rather than “Fail”) this assessment is made on the back of training delivered and does not consider whether the student would have been successful at the task without the input of the trainer.

From the point of view of recurrent training, this is an interesting conundrum – why should we be providing training to pilots in something could have happened to them in the aircraft the day before they were in the simulator? Such training is provided on the back of decades of clear evidence that it’s required; in part this is to maintain and improve proficiency in an entire suite of events in which we expect all pilots to be able to deal with – events you can’t or shouldn’t practice in the aircraft. Jet Upset/Unusual Attitude Recovery is something best not done when First Class passengers are trying to balance their champagne glasses 10 meters behind you …

I deliberately use the word “Proficiency” when training, and “Competency” when checking.

Years ago I came across a phrase which has stuck in my mind to this day – through various training, checking and management roles.

Our intent is to Train to Proficiency; and Check for Competency.

Perhaps more than any other this characterises the differences in the task and the role of the assessors of Training and Checking. The implication here is that we look to achieve a higher standard in our students in training than we require of our candidates in checking – is this how your organisation does it?

Checking in the Simulator

Checking on the other hand, is a different animal indeed. In some parlance, the perfect check event is where during the assessment the Check Captain may speak to the crew as ATC; as the Flight Manager or Purser; as the Engineer; as Dispatch; as the Fire Chief; as the Company – but never as the Check Captain.

The candidate(s) are given an initial set of criteria as they would normally expect for a flight – the from, the to, various performance and weather data; in short all that’s required to get a serviceable aircraft from A to B; but they aren’t told what non normal events are likely to occur, how the weather will change, or which instrument approaches they are going to do. The Check Captain will vary the conditions (weather, airport/runway status, etc) and inject normal and non-normal events into the flight based on a confidential script to provide the opportunity for the candidate to demonstrate desirable behaviours; whether the basic competencies such as procedural application or manipulative skill, through to the higher order aspects of situational awareness; task management or decision making. Broadly speaking this skillset is divided in the Technical Skills, and the Non Technical Skills (formerly CRM).

This assumes that the session runs from beginning to end without interruption. Unfortunately this is rarely the case and often “repositions” are required in the simulator to position the aircraft and crew to a time, place and condition where they can then go on to demonstrate competence in a specific sequence or single event that will occur once the simulated aircraft is released. With this breakdown in the natural ebb and flow of the flight, the Check Captain is expected to “set the scene” for the candidates so they have a clear mindset adequate to the task they are about to undertake.

One major difference between simulator and aircraft checking is that in the simulator, the Check Captain is “God”, able to control the weather, the serviceability of the aircraft, the progress of the flight – even freeze the entire simulation or reposition the aircraft around the airport or around the world. While checking in the aircraft, we don’t have that option, unfortunately.

It’s a little excessive, but not incorrect to say that any input from the Check Captain to the Candidate during any check has the potential to invalidate the independent assessment of that check. It’s also not unfair to say this is a tremendous waste of resources and talent. Here you have a highly qualified, trained, experienced and motivated individual, sitting in a simulator (or an aircraft) behind two pilots who are probably willing to learn. But the Check Captain is not there to train; the Check Captain is there to check. From the point of view of making better pilots – 50% of our resources are wasted on checking. That’s a Trainer’s purist view, but a somewhat valid one I think.

So it can be seen that the role of Trainer and Checker is quite different. It is expected that all Checkers have the ability to train – but this is not necessarily a requisite skill. It is required that all Trainers have the ability to assess and grade performance – but this kind of assessment is subtly different from actual Checking. It’s fair to say that some Trainers would make better Checkers. It’s also fair to say that some Checkers would make superb Trainers and are in fact “wasted” in the Checking role. Unfortunately this is a consequence of a system that places undue reward (both monetarily and from a”prestige” point of view) on Checking over Training. To my mind we have this ass about; in the very least the roles should be remunerated equally.

At various times I have been involved in the conduct of interviewing Training Captains for the role of Check Captain. One of my questions of the interviewees (all of whom were current Training Captains) has been –

“Once you become a Check Captain – which pilots do you think I (the Standards Manager) want to see you Fail?”

In some cases, the response was that they were confident I (Ken) wouldn’t want to see anyone fail. While that’s not necessarily untrue, the correct answer of course is I (the Standards Manager) want anyone – everyone in fact – who fails to reach the minimum standard (of Safety) to be given an Unsatisfactory result. That’s incumbent in both the Checker and the Checker Manager.

Another area in which Checking is different to Training is the personal responsibility of the Trainer/Checker in terms of the Result.

As a Trainer when I’ve “failed” a student, it’s always a shared responsibility. The Student has been unable to reach the required standard to progress on to the next stage of training (or the Check) – on the back of the training I provided (or failed to provide). By definition, I own part of that failure as the Trainer who wasn’t able to train the student to proficiency. This truism remains valid whether the student is patently unsuitable for the role he/she is in or is training for; or if that student is just having a bad day. As the trainer, you own at least part of that failure.

On the other hand – as a Checker, you’re not there to assist anyone to achieve competency – you’re there to see if they can do it themselves. That said, most organisations recognise that the simulator is not the aircraft; that people have bad days; and as mentioned the system allow for some form of Repeat for a failed event within a check simulator session. This is usually only provided if there’s time available to do so; and only offered against an event for which the Checker has already graded the candidate a Fail/Unsatisfactory grade.

Unlike the Trainer, the Checker needs to be circumspect (depending on airline/regulator policy) on the feedback provided to the Candidate when giving a repeat. The specific reasons for the Repeat (against the failed event) must be provided to the Candidate. More input than this, and you’re starting down the road of training within the check, thereby invalidating the check itself. Your airline may allow this to some degree; most don’t.

Of course this is within the hothouse environment of the Flight Simulator – when you get out on the Aircraft; it’s a different kettle of fish altogether. This is where it really starts to get interesting.

Checking in the Aircraft

By definition – since repeats are only offered against an event that is a clear failure (below minimum standard); and of an event that brings an overall fail of the check – you can’t repeat failed events in the aircraft.

“Ok Bloggs, that landing was unsatisfactory since you touched down about 50m off to the side of the runway in the grass and nearly hit the Control Tower. You control inputs were incorrect for the decreasing crosswind down final, and you didn’t apply rudder correctly in the flare to maintain the centreline. I’ll now position you and your 350 passengers out to 5 miles and we’ll have another go at that, shall we?“. If only.

In essence – any event in the aircraft that requires a repeat instead results in a failure of the line check. Any event which the candidate fails but does not result in a failure of the check, by definition is not a “fail” event in the check. For example, Bad Approach/Landing – fail your check. Crappy PA to the Passengers – not so much.

Traditionally the mantra of the line checker was that if you had to intervene in the progress of the flight, by definition this implies the failure of the candidate. What this means in practice however is in some airlines, a less than optimal situation may be allowed to progress to the point of a clear SOP/ATC violation before that intervention is forthcoming from the Assessor. Now we have a problem …

Annual Line Checking as a Crew Member

Annual Line Checks are conducted by a Check Captain who typically sits on the flight deck jumpseat as an extraneous member of the crew. While our aircraft only require two crew pilots; we have four pilots on board for reasons of crew rest. There isn’t room for the Check Captain as a fifth; hence on our flight deck the Check Captain is there also in an operational role – as a relief crew member. Logically (in terms of the check) when the Check Captain sits on the flight deck, he/she should not be involved in the flight; or at least involved to the least extent (safely) possible. However even when you’re not in a long haul environment, the point is moot. As a qualified Captain, as a professional (at least partly) responsible to the airline for the safe and efficient operation of the flight – can you really just there and let things degenerate to the point of a violation and not speak up – because it’s a check?

For each flight there is a Captain (CA), a First Officer (FO), and a Relief Crew Member (RCM). There is also the Pilot who is Flying (PF), and the Pilot who is Monitoring (PM). When the PF takes an action or fails to take an action that results in the failure of the check – what is the impact on the Pilot Monitoring who should have caught this? Or the Relief Crew Member who is sitting on the jump seat in the middle of the action with a (relatively) lower cognitive load than the other two pilots. Is the failure of one pilot the failure of the crew?

Consider also the situation where only one crew member (say just the FO) is under check. If this crew member similarly fails the check in such a way that could have been prevented by appropriate action or prompting by the PM or RCM who aren’t under check – is there any implication for their ability to operate their next rostered flight?

An important point to keep in mind is that the primary intent of this operation – and the focus of all crew (including the Check Captain) is not the assessment pass/fail of the crew members assigned to the flight that day – the mission is to get all the passengers and crew safely and efficiently from A to B.

This issue becomes more important as we discuss fail scenarios. With all this in mind – we’re finally at a point where I can discuss the issue at hand – assessment of candidates during aircraft line operations.

Scenarios

The following scenarios discuss some of the issues encountered by Check Captains during Line Operations, and highlight the need for Airline Check/Training Standards policy development on Aircraft Checking and the role of the Assessor. Let’s start with the most obvious, and probably most common.

Unstable Approach

After decades of incidents and accidents, it is universally recognised that best practice is for an airline standards department to establish an altitude at which the aircraft must be “Stable”. This includes appropriate lateral and vertical positioning from which a safe landing can be made; fully configured for landing (gear and flap); appropriate airspeed and thrust control; normal procedures such the Landing Checklist complete. Most airlines have this as 1000 ft; some airlines lower it to 500ft in visual conditions (some do not). If the aircraft is not stable by this “hard” altitude, the PM must call “Unstable – Go-Around” and the PF must comply. Similarly if the approach was stable but becomes unstable below stabilisation height – the crew must go-around.

Scenario : On approach, it becomes clear that the crew may not be able to comply with the company 1000 ft stabilisation requirement (on speed, configured for landing, checklists complete, etc).

• Should the Check Captain intervene?
• When should the Check Captain intervene?
• If the Check Captain prompts the crew to take action to bring the aircraft back towards a stable profile – what impact does this have on the check?

The crux of the issue is not necessarily the independence of the assessor, but the conflict between the need for the Checker to remain hands off, let a situation develop and see the crew react appropriately – against the requirement to see the aircraft landed safely and efficiently at the destination.

While it would be nice to operate under the assumption that the only time an aircraft will get unstable is just before the crew fail a line check, in the real world, aircraft become unstable for a variety of reasons – environmental conditions and ATC intervention being the most common. It’s crucial that a crew who are becoming unstable have the capacity to recognise this situation; know how to effect change to bring the aircraft back towards stable parameters; have the judgement to call for a go-around when it becomes obvious that the approach is not going to meet the stabilisation requirements. How does a Checker know that the crew in front of him have these skills if he/she intervenes before the situation develops fully?

Against this is the requirement of (a) safety; and (b) the needs of the operation (to land the aircraft at the destination). The Checker must not allow a situation to develop that puts either of these requirements in doubt. Assuming the Checker has no immediate concerns as to the safety of the aircraft at this stage – remember that the aircraft, crew and Checker aren’t here today to evaluate their knowledge/skills/attitude (KSA’s) on a line check – we’re all here to take 350 passengers to the destination. The check is a side issue to the requirement to achieve the mission.

As with most of these issues – this will come down to the experience and judgement of the Checker. Similarly, if the Checker decides to intervene (“We seem a little fast today, don’t you think …“) the implied failure of the crew is also within the judgement of the Checker. The Checker will use a variety of parameters to assess whether this is a fail – how out of tolerance the approach was; whether the crew demonstrated awareness of the situation; mitigating factors such as ATC, Weather, etc. Intevention may not be a fait accompli for failure.

If this (unstable) approach is called as “Stable” at 1000 ft – what then?

• What is the impact on the Check for the Pilot Flying (PF); Pilot Not Flying (PM); Captain; Relief Crew Member (RCM)?
• Should the Check Captain call for a go-around? If he/she does – what is then impact on the Check? For who?

Approach Stabilisation is a concept developed the the Flight Safety Foundation as part of the Approach and Landing Accident Reduction (ALAR) program, on the back of many, many incidents and accidents that were directly attributed to the inappropriate continuation of a fast/high approach to an unsuccessful landing. As such, continuing any unstable approach below the nominated height implies that safety has been compromised on the approach.

That said, there are degrees of stabilisation (aren’t there?); while technically an approach may be a little fast, or the last line of the checklist not quite done – it’s pretty close and it’s easy for the Crew and the Checker to allow the approach to continue confident that it will shortly meet the criteria. But while a line check is an assessment of a crew’s “normal operation” – if this crew is willing to take a “slightly” unstable approach past the stabilisation height requirement – what are they willing to do when no-one’s watching?

Contrary to this, the industry is starting to recognise the risks inherent in the get-out-of-jail manoeuvre that is the approach go-around. As more and more crew avail themselves of a go-around, and as we start to look more closely at it in the simulator, it’s being re-discovered that this manoeuvre itself presents some challenges. It involves a sudden and usually unexpected radical change to flight path, large amounts of thrust and pitch change – which is not usually a good thing. On the back of this is decades of crew being required by Regulatory Authorities to demonstrate proficiency in the single engine go-around from very low altitude in the simulator based on the assumption that surely if someone can do an engine out go-around near the ground, an all engine go-around from higher up is a no-brainer? As such regular exposure to the vagaries of all engine go-arounds, particularly from higher altitudes (such as off unstable approaches) has suffered. As it turns out – the two engine go-around from 1000 ft to a missed approach altitude that may be as low as 2000 ft can be a handful. Just ask an A320 pilot.

So therefore, sitting in your Checker’s jumpseat, watching a crew who have just operated a long haul flight find themselves a bit high/fast/late on the approach at 1000 ft – is it safer to let them get stabilised and land, or “force” them into sudden go-around by calling for it from the jumpseat? It must be acknowledged that such a call from a Checker may carry more “weight” than a similar call from the RCM. It’s probably better to reserve the call “UNSTABLE – GO-AROUND!” for particularly severe circumstances and instead perhaps highlight the relevant parameter(s) to the crew instead. Again – Judgement on the part of the Checker. This is why we pay them the big bucks.

So why didn’t this crew call “Unstable Speed – Go-Around.“? There are several possibilities including poor Situational Awareness (SA) or a lack of procedural knowledge. Another factor may be the PM not wanting to bring on the “fail” of the PF. While this may seem incredulous, you do get this in both the simulator and the aircraft – the PM is reticent to highlight deviations out of concern for bringing to the attention of the Check Captain the deviation. Like we didn’t see it anyway …

Ostensibly it’s the PF who is to blame for allowing the situation to develop whereby the aircraft is out of tolerance at stabilisation height. Therefore if this is a failure, that failure belongs to the PF, doesn’t it? However much more than “along for the ride” is the PM, who presumably sat there with a lower cognitive load than the PF, allowing the situation to develop. There was bound to be a point prior to 1000 ft when it was clear that (a) we might not be stable; and (b) we aren’t going to be stable. The failure to clearly call this to the attention of the PF is a clear failure of the crucial PM role – either in terms of SA (didn’t notice the situation developing); or violation (didn’t make the call). So it’s entirely possible that this failure will be shared with the PM as well – even if the PM isn’t actually on a line check. Now there’s a can or worms …

Finally we have the RCM who is under the lowest load of all, and can clearly see the entire flight deck from the center jumpseat. This crew member should have the experience and training to detect a developing unstable approach and call it in absence of anything coming from the PM. A check failure of the PF (and PM) may reasonably be stretched to the RCM as well.

Just to further illustrate the complexities – if approach was allowed to deteriorate without intervention of comment where a go-around at stabilisation height became a requirement, and the Checker didn’t make any calls to bring this to the attention of the operating crew – shouldn’t the Checker be failed as well? The answer is Yes – and this has actually happened.

Scenario : Very Unstable Approach

Another approach is clearly unstable, and results in a go-around at 1000 ft in accordance with SOPs. However it is clear to the Check Captain that the approach was never going to be stable, but the PF/Crew persisted with the approach all the way down to stabilisation height (1000 ft). The alternative – discontinuing the approach much earlier – was never considered by the crew.

• Is this a Fail for the PF/PM/RCM?
• Should the Checker have commanded a go-around (or otherwise highlighted the unstable nature of the approach) much earlier? And if the Checker did?

Note that this discussion assumes (a) that the Airline’s stabilisation policy includes a recommendation to discontinue an unstable approach when it becomes clear to the crew that the aircraft is not going to be stable by the stabilisation height; and (b) the scenario here is that the approach has been flown in such a way as to be grossly unstable, and that it is clear to the Checker well before 1000 ft that the approach is never going to be stable in time to comply with policy.

For an approach to be discontinued well before 1000 ft AAL, it is certainly within the purview of the Check Captain to intervene and subsequently fail Candidate/Crew who choose to continue the approach (or not choose to discontinue it). Judgement is exercised as to whether the action to continue is done wilfully, or through a lack of SA; or whether there was a genuine (mistaken) belief that the approach could have become stable in time. Even if the crew express such a belief – if in the judgement of the Check Captain stabilisation was never possible, a fail is definitely a likely outcome.

There is much discussion around this point. The issues of failing a pilot who executes a missed approach because of approach instability is rife with contradiction. Don’t we want crew who are unstable at 1000 ft to execute a go-around? Don’t we want to encourage this behaviour? While true, we also want crew who can position the aircraft appropriately for landing as well. Again the point of this exercise is not to pass/fail the candidate(s). The point of this exercise is not to see a crew member correctly assess approach stability at 1000m ft and commence a go-around. The point of the exercise is to deliver 350 passengers safely to the destination. The safety and intent of the operation itself should not be subverted by the supposed needs of a line check.

Scenario : Descent Altitude Compliance

During descent, the PF demonstrates a clear lack of SOP compliance in the setting of the MCP altitude selector to ensure Standard Terminal Arrival Procedure (STAR) altitude compliance. The PM/RCM make no comment on this. No actual STAR restrictions are breached.

• What is the impact on the check?
• Does it make a difference if the PF is the Captain vs the First Officer?

The FCTM makes it clear that unless altitude restrictions are closely spaced such that workload would be high as would be the risk of unintentionally “capturing” an altitude restriction – all STAR altitude restrictions should be set to protect the flight path from a violation. The most obvious reasons for this crew member not doing this are (a) uncertainty as to the correct procedure; (b) wilful deviation from the procedure; (c) bad technique through fatigue/error/etc. Without a clear violation, this occurrence in an of itself is unlikely to be a reason for a fail – but certainly it should be reflected in the grading of all three crew members. The occurrence may also be part of a larger picture of each crew member’s KSAs that might drive an overall unsatisfactory result for the check.

How does this change if the lack of SOP compliance results in a likely altitude breach (without Check Captain intervention)?

• When does the Check Captain intervene?
• What impact does this intervention have on the check?

Check Captains are absolutely required to intervene in the operation to prevent an altitude violation. It is likely that this intervention would result in a failure, but it is certainly within the judgement of the Check Captain to assess the likelihood of an altitude violation if no Checker intervention had taken place. By definition intervention of this sort must take place before it’s too late to correct it; but that then leaves a window in which the crew could have self corrected. Again, it’s big bucks time and the Checker is the one to make the call about whether the crew could have/would have self corrected in time.

How does this scenario change if the lack of SOP compliance results in an actual altitude violation?

• Is this a fail? What if there were no other aircraft, no airspace breach, no real risk to the aircraft?
• Who fails? The PF, the PM, the RCM?
• What role does any fatigue of the first rest crew member play in relaxing fail criteria (to all the above scenarios)?

An altitude violation is a clear breach and should result in a fail assessment. This begins with the PF, but may necessarily extend to the PM as well. One further aspect is that if the PF was the First Officer, the PM (Captain) carries an additional responsibility here as the Aircraft Commander. It would be more likely that the Check Captain will fail the PM for an altitude violation if the PM was the Captain than the reverse (logically). Also within the realm of the Checker’s judgement is whether the RCM should share the fail assessment. Again if the RCM could have seen the impending violation – should have seen and called the violation – it’s probably a point of failure for the RCM as well.

Additionally – where was the Checker when this situation was developing without intervention; and the breach occurred without the Checker speaking up. Once again to take this to it’s logical conclusion – this is a fail for the Checker as well. Just imagine the paperwork …

Finally, it should be noted that Fatigue is an (un)necessary evil in the operation of all long haul flying. The Fatigue factor can certainly be part of the Checker’s assessment of crew performance, right up to but excluding a violation.

Scenario 4 : Taxi

After landing, the crew exit the runway via the wrong taxi way (as was instructed by ATC).

• What impact does this have on the Check? For the PF, PM, Captain, RCM?

Once again, this could be considered clearance violation with potentially significant consequences. As always, there are judgement calls to be made – how clear was the exit instruction? Is there anything miss-leading about the guidance provided? Was the required exit realistic? Is this an intentional, unintentional or inadvertent violation? Big Bucks time. Clearly if the PF took the wrong taxiway despite clear instruction and clear markings, the outcome of the check could be in doubt. As always the complicity of the PM and RCM must be considered, and the Captain as PM wears additional responsibility for safe conduct of the flight. Finally – did the Checker really sit back and let it happen? Was the Checker asleep? Did the Checker missread the instructions/markings as well?

After Landing, the crew are instructed to hold short of the (active) second runway – but continue across it (or make the intention to do so clear).

• Impact on the Check? For the PF, PM, CA, RCM?

Similarly we now have a definite violation with possibly catastrophic consequences, likely to lead to a failure of the check, mitigating factors aside. Assuming the Checker intervened to prevent the violation – is it still a fail? It should be, once again within the judgement of the Checker as to role and culpability of the PM, Captain, RCM.

During taxi in, the Captain elects to do Single Engine Taxi In (SETI), when clearly the turns, taxiway slope and configuration of the gate makes this an unwise decision.

• Should the Checker intervene?
• If the Checker intervenes, what impact on the Check? For the Captain, for the First Officer?

Single Engine Taxi In (SETI) is a fuel saving initiative where after landing and engine cooldown (3 minutes) one engine is shut down by the PM. Ideally it’s the engine that will be on the inside of all the taxi turns as turning against the live engine can be challenging (but not impossible). If you anticipate sharp turns in both directions, if there’s a lack or maneouvring on the apron for the final turn to the parking stand – SETI may not be a good idea. Additionally there can be other restrictions for operations with slick ramp ways, inclement weather, etc. In this instance it’s assumed that SETI is not clearly precluded by the operating restrictions, but clearly not ideal. SETI can save up to 15 Kg of fuel per minute of taxi (or holding position on the ramp) and multiplied across a fleet adds up to a significant cost and environmental saving.

Whether the Checker should intervene depends on circumstances, but certainly the option remains. In either case choosing to do SETI is unlikely to affect the outcome of the check, and may not even justify a grading impact or comments on the form … unless …

Having elected to do SETI, aircraft stalls to a halt halfway through a turn towards the operating engine, and the geometry/surrounds clearly don’t permit straightening or excessive thrust to regain movement. The Captain/Crew elect to re-start the engine to continue taxi.

• Impact on the Check? Captain only?

Having (perhaps) poorly decided to conduct SETI, when confronted with the reality of the problem, the Captain elects to return to full operation, rather than risking a taxi excursion or damage to the surrounds through excessive thrust. Having made one questionable judgement, this Captain has made a good decision. For me personally this is a good sign and would be reflected as such on the check form. It’s unrealistic to expect that crew will make the best decisions at all times; how crew deal with poorly made decisions is at least as important as making good decisions in the first place. Not being a slave to your previous decisions is a good NTS attribute.

Having elected to do SETI, the PF is required to use excessive thrust to turn towards the operating engine and to taxi onto stand. As far as the Check Captain is aware, there is no damage or injuries to ground equipment or personnel, although the thrust applied was clearly excessive.

• Impact on the Check? PF vs PM vs Captain?
• What if the Tower advises that ground equipment was damaged/blown onto the taxiway?
• What if the Ground Agent advises that one of the ground marshalers was injured by jet blast?

Use of excessive thrust at anytime is a significant risk in the 777, and a known problem with SETI and certain taxiway configurations. Unless the PF is certain the area behind the equipment is clear, thrust above that normal for taxi should not be used. Using this thrust places people and equipment at risk. Irrespective of who decided to do SETI and who is the PF during taxi, both crew carry culpability for excessive thrust use, although if the PF is the First Officer the Captain as PM carries perhaps more responsibility than the reverse. Excessive thrust use during taxi is unlikely to bring about a check failure – but damage to equipment and/or personnel is certainly likely to do so.

Summary

I can’t tell you how many times I’ve been back through this extensive, at times inarticulate and confused writing. At times it’s sat in drafts for weeks without input. I’m not sure that a Summary is helpful, or even appropriate – but here we go.

• No doctrine or philosophy no matter how well thought out and documented takes the place of the expertise invested in the Check Captain on the spot, on the day. Within established guidelines – it’s the Checker who determines the outcome of the check based on observed performance and outcomes.
• Remember that on every flight there is a Crew. While an individual pilot may bear the direct responsibility for an action, omission or error – our basic operating paradigm is a crew working together to pickup the errors of others. In that there is safety for all – not just the crew on their check but the passengers on their flight. As much as a candidate may be under review on a check – so is the crew paradigm. Additionally the Captain of the aircraft (whether PF or PM) bears additional responsibility for the safe and efficient conduct of the flight that cannot be over estimated by the Check assessment.
• While only one candidate may be under check today – all crew on the flight deck can be considered liable under the assessment and potentially could find themselves stood down as a result of a bad check. While most obviously the Captain bears responsibility for the untrapped outcome of an action/omission/error on the part of the First Officer – the reverse should hold true as well. In a long haul crew where relief crew are on the flight deck – an uncorrected mistake that isn’t picked up by the relief crew would also justify criticism, if not an actual fail assessment.
• It is incumbent on the Training/Standards Department to train Checkers well; clearly document a policy in relation to Checking – and then back their Check Captain’s judgement when the inevitable Monday morning quarter backing comes. Despite the plethora of instrumentation and recording that takes place on a modern flight deck – none of this information is likely to be available to provide context for a check fail (unless it involved an actual incident). It therefore comes down the Candidate vs the Checkers word. If you can’t trust your Checker – either He/She shouldn’t be checking or you shouldn’t be in Management.
• Never forget why we are here on the flight deck today. We are here to (a) Safely; and (b) Efficiently land the passengers at the Destination (or possibly, Alternate). The completion of the check assigned to this flight comes a long way behind. An un-necessary go-around on the way to that landing that could have been avoided if someone (including the Check Captain) had spoken up is likely an unacceptable compromise of both Efficiency and to a lesser degree, Safety. If as the Checker you find you can’t speak up on the flight deck without failing your candidates – you need to go talk to your Standards Manager.
• A violation on a Check is just as unacceptable (if not more so) as a violation on a normal line flight, and should reflect extremely badly on all involved – including the Check Captain. Having to speak up as the Check Captain (or as a crew member) to avoid a violation may not be an automatic fail of the check – but is certainly a likely outcome and within the judgement of the Checker.
• The responsibility and authority of the Checker rests partly on the Regulator; but mainly on the support of Standards Management and documented policy. If you have the support, but not the documentation – it’s time to roll up your sleeves and do some admin. If you don’t have the support of your Standards Management – I can’t help you; no-one can.

Follow On

After reading this post, a friend of mine in a Management role in a UK airline asked me the following:

“When you rock up for work as the check captain how involved do you get with the crew paperwork preparations and what kind of brief do you give them?”

There are two questions here, starting with the easy one … on the back of my clipboard is the following:

Pre-Flight Checkers Brief

• Introduction (Normal Line Ops)
• Crew not Under Check (also PM/RCM Roles)
• Asking Questions
• Role of the Assessor (Safety, Efficiency, Assessment, NNM)

This translates into something like the following. As with all briefs, it’s pitched to my perception of the experience of my candidates, and the operation on the day, and my personal sense of whimsy.

“Just quickly, this is obviously an annual line check today for competency. I’m here to assess normal line operations and that’s all I expect to see.

Optional : [John, I realise today that this is not your check and you are along for the ride. However I feel it’s important to point at that you are responslible to the safety of the operation today just as much as XXX, and since I am a Check Captain, you must consider yourself under check as well. That doesn’t mean there will be any paperwork, questions, etc – but if something occurs that would result in a fail grade if this were your check, then being placed SOC is a likely consequence of that today.]

I won’t be asking you a series of difficult technical or procedural questions, but you can expect me to ask you questions during quiet time about what has come up in the normal course of events, and to talk about general or topical items of operational concern in our flight decks. I would also like discuss any questions or issues you want to talk about – so remember the more questions you ask of me, the less I ask of you.

My role today is independant assessment. As such I would ask you to allow me to sit back and watch you operate. I will not intervene unless I feel it’s appropriate, but don’t take my intervention as an indicator that your check is going badly. Remember I’m here not just for safety – but I’m a Captain in this airline and bear part responsibility for the safety and efficiency of our operation, so I might choose to speak up for something other than a fail point if I believe it’s appropriate and won’t compromise your check. You can definitely expect me to speak up for reasons of safety or to avoid a violation, but don’t rely on that – you guys are responsible for the operation today as you always are. In any event – I want to be a fly on the wall but I’m here if I’m needed.”

Logically the Checker needs to be involved in all aspects of the operation to the point where he can assess competence in the various activities, as well as provide the safety net from violation as is required of a Crew Member on the flight deck (if not the actual aircraft commander).

Practically that means becoming involved in the review during pre-flight, otherwise how can you (a) assess that the crew are performing as required and (b) detect that something has slipped by such as an illegal dispatch?

Now we’re firmly back in the groove of the participation of the Checker contaminating the check, but we were always there really. As a Checker there’s a constant reminder in the back of my mind that this is a check and I’m (a) keeping an eye on what’s going to so I can assess; (b) keeping an eye on what’s going on to ensure safety/legality/efficiency; and (c) be constantly aware of my level of involvement and the associated risk of contaminating the check. As long as I remember to keep that reminder going (no, I don’t actually have that voice in my head – just to note) – I’m confidant that I’m keeping all three requirementes in balance.

Like any other role, you develop your personal techniques over time, and learn from your mistakes if you’re honest about your own performance. In the classic two-crew-with-a-Checker-on-the-jumpseat operation I would suggest the Checker has to be a sponge in the preflight, aware of everything going on enough be able to assess and secure safety; but hands off as much as possible to ensure integrity.

The contrast with a LOSA audit is interesting. LOSA is an industry standard of assessment of flight deck operations developed by the University of Texas and the FAA in the late 90’s. Here you have an assessor on the flight deck always over an above the actual crew requirement with specialist training; someone who is aware of the SOPs and how things are supposed to be done – often from a “book” point of view rather than what actually happens in the aircraft on the day – often without local knowledge of the crew, the route, the aircraft type or even the airline. Truly an indendant assessor and free of the requirement to pass/fail the crew; from any real responsibility towards the efficiency or the safety of the flight (other than the obvious – “That’s the ground rushing up, isn’t it?”); free to observe and report. Add a Checkers rating to the observer without an aircraft/route/airline qualification and you have the perfect checker, from some points of view.

We are reviewing Decision Making Models at the moment. On the 777, we’ve used FORDEC, which is very close to the European model, except we’ve replaced “Check” with “Communicate”, which may or may not have been a good thing. Other fleets in our airline are using GRADE or NMATE.

There are several reasons why Decision Making Models are in use. The popular notion is that there are some pilots who can’t make decisions, and need a model; just as there is a popular notion that some pilots are natural “decision makers” and no matter how complex the decision, they never need a model; the truth is perhaps somewhere in the middle.

Modern aircraft are both very complex and highly simplified. Information presentation systems, coupled with alerting and electronic checklists take what could be a very complex system and reduce it to essences that pilots can pretty easily deal with. There are a couple of problems with this though.

One of these is the Non Technical Skills requirement. Even after the Captain (or First/Second/Relief Officer) has reached a good decision based on complex information after risk analysis and implementation review – this has to be communicated to (and agreed to by) everyone else – this is better achieved when everyone is along for the ride, rather than told when you’ve reached the destination. Or at least, ideally it should. Some of the natural decision makers out there who aren’t necessarily thinking these things through methodically (consciously) but still coming up time and time again with the right decision – may not be the greatest of communicators.

Finally – there’s the QF32 factor. Where the problem is really bad and the information is so complex, so changing and so overwhelming, that a reasoned decision taking all factors into account allow the situation to develop fully to avoid impulsively rushing in – may not be possible. It’s get the aircraft on the ground time.

In any event – with a few models on the table (in the Group) – we’ve been trying to reach a consensus …

F O R – D E C

Facts

• What is the full extent of the problem?
• Gather all relevant Facts.
• A problem which has been well defined at best usually suggests its own solution and at worst prevents the crew from going down the wrong path.
• It is important to stay focused on defining and understanding the problem rather than rush to the solution.
• There will often be more than just the one problem requiring a solution and they will all need to be carefully considered and then dealt with in order of priority.

Options

• What options are available?
• Define the different options you have, considering that there may be several possible options to facilitate a safe outcome.
• Time can be considered as; critical, available and required. There are very few problems that require immediate action. In the vast majority of cases a considered and well developed plan is going to lead to a safe optimised resolution.
• The use of open questions can assist in staying problem centred. “What do you think …?”

Risk/Benefit

• What are the risks and benefits associated with each option?
• With the given situation, what are the assessed risks in pursuing a course of action weighted against the perceived benefit?
• With the given situation, do we return for an immediate landing overweight or do we take up the hold and jettison fuel?
• With the given problem, do we land on the longer runway with a crosswind or the shorter runway with a headwind?

Decision

• Which option have you decided on?
• After spending an appropriate amount of time on the first three steps, the commander must eventually make a decision.
• This is the step that many people instinctively leap to, however correct application of a management model will lead to a process driven solution that will have initially focused on accurately defining the problem, analysing the options before finally deciding on the solution.

Execute

• Execute the selected option. Once the decision has been made, the plan must be put into action

Communicate

• Communicate your intentions.
• Once the plan has been executed, the commander must ensure that his intentions are communicated to all interested parties.
• This will include the cabin crew and passengers within the aircraft, along with relevant agencies on the ground.

Irrelevantly, one special moment in all this has been finding out the various models that are around and in use. It’s been fascinating – here’s a sample. All models have their good and bad elements. Many share common ideals and drivers – since the problems all the models are trying to address are substantially similar.

DODAR (British Airways)
D – Diagnose
O – Options
D – Decide
A – Assign
R – Review

DECIDE (US FAA)
D – Detect
E – Estimate
C – Choose
I – Identify
D – Do
E – Evaluate

NMATE (Boeing)N – Navigate
M – Manage
A – Alternatives
T – Take Action
E – Evaluate

SAFE
S – State the Problem
A -Analyse the Problem
F – Fix the Problem
E – Evaluate the Result

GRADE
G – Gather Information
R – Review the Information
A – Analyse the (you guessed it) Information
D – Decide
E – Evaluate the Course of Action

FATE
F – Fly the Aircraft
A – Analyse the Alternatives
T – Take Action
E – Evaluate

RAISE
R – Review the problem
A – Analyse
I – Identify solutions
S – Select an Option
E – Evaluate

ADFP
A – Aircraft (Consider the Problem)
D – Destination (Appropriate)
F – Fuel (Sufficient)
P – People, Pax, ATC, Company etc.

3P’s
P – Perceive
P – Process
P – Perform

OODA
O – Observation
O – Orientation
D – Decision
A – Action

CLEAR
C – Clarify the problem
L – Look for data and share information
E – Evaluate different solutions
A – Act on your decisions
R – Review performance

PILOT
(this has to be the best one surely?)
P – Pool the facts
I – Identify the problem
L – Look for Solutions
O – Operate
T – Take Stock
(perhaps not)

SADIE (Emirates in the 90’s)
S – Share Information
A – Analyse Information
D – Develop the Best Solution
I – Implement your decision
E – Evaluate the Outcome

RCCSDAD
R – Recognise
C – Control the aircraft
C – Contain the emergency
S – Safe Flight
D – Decide
A – Act
D – Divert?

SOCS
S – Situation, define
O – Options
C – Consequences of actions
S – Select (an Action)

DESIDE
D – Detect
E – Estimate
S – Set Safety Objectives
I – Identify
D – Do
E – Evaluate

During the discussion the following models were advanced by the “Managers” I work with. I must admit I got at least halfway down them until I realised they were pulling my chain. Some of these require inside knowledge – next time you have Paul or Stu on the other end of a Beer, ask them.

A ssess the problem
P erform the correct memory item
P erform the correct checklist
L et ATC know what you are up to
E xecute the diversion

I nterrogate
P robe
A ssess
D ecide

B elt sign ON
E xamine the problem
S ound the alarm
T ell the cabin
P riorities
R isk assessment
A ction plan
C hecklist complete
T hreat and error management
I ndicate intent
C onsider the options
E xecute the plan

K now the problem
E xecute the diversion
N otify ATC
S ecure the aircraft
C hecklist complete
H old if required
A irport for diversion
I nform passengers
R eview the risk

What model does your airline use?