{"text_format":"1","workbook":"xlsx","n_sheets":3,"sheets":["Guidance and Tips","AI Bias Assessment Template","Risk Rank Guidelines"],"not_rendered":{"charts":0,"images":3,"macros":0,"external_links":2,"defined_names":21,"comments":1,"pivot_tables":0},"unopened":[],"bounds":{"document_bytes":16777216,"sheet_cells":100000,"rows_first":2000,"rows_last":500,"rows_between":2000,"halvings":0,"cell_chars":16384,"cell_head_chars":4096},"cells_cut":0,"cells_raw":0,"read":"whole","date_system":"1900"}

{"sheet":"Guidance and Tips","index":0,"n_rows":2,"n_cols":1,"rows_kept":2,"cut":false,"hidden":false,"merged":[]}
[]
["AI Ethics – Bias Considerations\n\nTypes of Bias\nBias can have many forms. Some can be useful and intended. Some can be unintended, harmful and unethical. As a term, it can mean different things in different contexts, and different things to different people. It also needs governing for more than ethical reasons – commercial and safety impact are also extremely important reasons why bias needs to be understood and governed.\n\nIn a statistical context, bias is the tendency of a statistic to overestimate or underestimate a parameter. Bias can seep into results for many reasons including sampling or measurement errors, or unrepresentative samples. The word bias in the regular English language implies that there is a personal reason to misrepresent a piece of information. However, in statistics, it just means that the estimator being used doesn’t produce a good estimate.\n\nIn a philosophical context, a bias is a tendency, inclination, or prejudice toward or against something or someone. Some biases are positive and helpful - like choosing to only eat foods that are considered healthy or staying away from someone who has knowingly caused harm. But biases are often based on stereotypes, rather than actual knowledge of an individual or circumstance. Whether positive or negative, such cognitive shortcuts can result in prejudgments that lead to rash decisions or discriminatory practices.\n\nIn a sociological context, bias is a tendency (either known or unknown) to prefer one thing over another that prevents objectivity, which influences understanding or outcomes in some way. It is very similar to bias in the philosophical context.\n\nScope of this Governance\nWithin the context of governing bias in AI, all the above need to be considered. This therefore needs to include human conscious and unconscious bias, machine/sensor inaccuracies, SATAA (sense, acquire, transfer, analyse, act) errors, sampling errors, measurement errors, estimator errors, selection bias as well as many more. Furthermore, dependent on the domain/context, likelihood of these biases will be different. Note, it is unlikely that all bias can be eliminated, it is therefore important that risks of impact and likelihood are fully understood thereby enabling bias to be minimised as necessary.\n\nNotes\nFirstly, it is important to use this Framework as a prompt towards areas to consider. It is not intended to be completely comprehensive as all applications of AI can be different.\nThere will be occasions where delibarate 'skewing' of sampling needs to take place in order to get the necessary coverage of rare samples/populations. Care is needed to ensure that the testing etc is adapted to be able to cater for this. Additionally, bias can be beneficially intended - if the analytic/output is safety critical, safety factors may be included in the model/algorithm to thereby assure safety.\nAlso, bias may actually be an accurate reflection of reality and not an indication of anything wrong - quick, 100m race times will be biased towards fit and healthy parts of the population for example.\nThis PFMEA is intended to provide guidance/prompts which may require certain areas to be explored in more detail.\nUse the 'hover over notes' on the cells for specific guidance Also, a valuable resource is the AI Bias Playbook in the Digital Product Integrity Teamsite - here's the link:- https://rr.fp7k.exostar.com/customers/rollsroyceplc_forumpass/digital/ServiceIntegrity/_layouts/15/DocIdRedir.aspx?ID=AANEUQA5XN7T-167-1161 (paste into your browser)"]

{"sheet":"AI Bias Assessment Template","index":1,"n_rows":40,"n_cols":27,"rows_kept":40,"cut":false,"hidden":false,"merged":[[0,0,1,26],[2,10,2,15],[4,10,4,11],[4,12,4,14],[8,10,8,11],[3,10,3,11],[3,12,3,14],[9,10,9,11],[6,12,6,14],[2,0,10,9],[2,16,10,26],[8,12,8,14],[5,10,5,11],[5,12,5,14],[9,12,9,14],[6,10,6,11],[7,12,7,14],[7,10,7,11],[11,21,11,26],[10,10,10,11],[10,12,10,14],[11,0,11,10],[11,11,11,14],[11,17,11,20],[11,15,11,16]]}
[]
[]
["PFMEA Template to Govern AI Bias - FM MS 1.6.1.3",null,null,null,null,null,null,null,null,null,"Date originated: 15/07/2021",null,null,null,null,null,"Other relevant / supporting information:"]
[null,null,null,null,null,null,null,null,null,null,"Dates Revised:",null,"Revised by:",null,null,"Issue #"]
[null,null,null,null,null,null,null,null,null,null,"2021-07-16",null,"Lee Glazier",null,null,"Draft 1.0"]
[null,null,null,null,null,null,null,null,null,null,"2021-07-19",null,"Lee Glazier",null,null,"Draft 1.1"]
[null,null,null,null,null,null,null,null,null,null,"2021-08-16",null,"Lee Glazier",null,null,1]
[null,null,null,null,null,null,null,null,null,null,"2021-09-03",null,"Lee Glazier",null,null,2]
[]
[]
[]
[null,null,null,null,null,null,null,null,null,null,null,"Current Specified Process Controls:",null,null,null,null,null,"Recommendations",null,null,null,"Action results"]
["Step","Process step Value status: VA / NVA / NNVA","Function","Requirement","Potential Failure Mode","RRPS PFMEA:\nPotential Effect(s) of Failure -\nExternal Customer (product / system level failure)","RRPS PFMEA:\nPotential Effect(s) of Failure -\nInternal Process (typically impacting SQCDP KPIs)","Severity/Impact","Potential Cause(s) of Failure","7Ms / Fishbone cause category","Likelihood","Current Process Controls (Prevention)","Detection of potential Failure Mode Cause","Current Process Controls (Detection)","Detectability","Risk Priority Number","Potential risk / issue rank A, B or C","Recommended Action/Mitigation","Responsibility","Planned Completion Date","Responsible Business / Department","Actions Taken","Actual Completion Date","Severity/Impact","Likelihood","Detectability","Risk Priority Number"]
[null,null,"AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Requirements are biased"]
[null,"VA","AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Training data is biased","Man",null,null,null,null,null,{"$f":"H15*K15*O15","v":0},null,null,null,null,null,null,null,{"$f":"H15","v":0},null,null,{"$f":"X15*Y15*Z15","v":0}]
[null,"NNVA","AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Data preparation is biased","Method",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,"NVA","AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Model is biased","Machine",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,"AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Tuning/optimisation is biased","Materials",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,"AI operates as required",null,"AI is developed with unwanted bias",null,null,null,"Test data is biased","Maintenance",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,"AI operates as required",null,"AI in use produces unwanted biased output",null,null,null,"Production/use data acquisition method is biased","Measurement",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,"AI operates as required",null,"AI in use produces unwanted biased output",null,null,null,"Data transfer mechanism introduces bias","EnvironMent",null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,"AI operates as required",null,"AI in use produces unwanted biased output",null,null,null,"Data operations/manipulation introduce bias",null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]
[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,null,null,null,null,null,null,null,0,null,null,0]

{"sheet":"Risk Rank Guidelines","index":2,"n_rows":1000,"n_cols":12,"rows_kept":1000,"cut":false,"hidden":false,"merged":[[10,4,11,4],[0,0,0,11],[1,8,1,11],[1,4,1,6],[1,0,1,2],[4,4,6,4],[7,4,9,4]]}
[]
["SEVERITY",null,null,null,"OCCURRENCE",null,null,null,"DETECTION"]
["Rank","Effect","Criteria : Severity of Effect on Application\n(End Customer Effect)",null,"Likelihood of Failure","Criteria: Occurrence of Cause ","Rank",null,"Opportunity for Detection","Criteria:\nLikelihood of Detection by Existing Mechanisms/ Control","Rank","Likelihood of detection"]
[10,"Critical with no warning","Bias causes application to completely malfunction/mislead without warning",null,"Very high","Almost Guaranteed to occur",10,null,"No Detection Opportunity","No known controls available. Failure will escape.",10,"Almost impossible"]
[9,"Critical with warning","Bias causes application to completely malfunction/mislead with some warning",null,"High","Very highly probable",9,null,"Not Likely to Detect at any Stage","Random checks made at low frequency, no prevention in place. Failure highly likely to escape.",9,"Very remote"]
[8,"Very high","Major degree of dissatisfaction due to impact of Bias on downstream process or customer. Very high impact on ethics or business metrics",null,null,"Highly Probable",8,null,"Problem detection post-processing","Random checks in place, prevention based on operator noticing problem. Failure likely to escape.",8,"Remote"]
[7,"High","High degree of dissatisfaction due to Bias on downstream process or customer. High impact on ethics or business metrics",null,null,"Quite probable",7,null,"Problem detection at Source","Regular checks in place (not 100%). Prevention based on operator training. Failure likely to escape ",7,"Very low"]
[6,"Very significant","Bias causes very significant dissatisfaction, e.g. very significant impact on ethics or business metrics",null,"Moderate","Probable",6,null,"Problem detection post-processing","Prevention based on operator diligance and regular training, regular checks in place, significant chance of escape",6,"Low"]
[5,"Significant","Bias causes significant dissatisfaction, e.g. significant impact on ethics or business metrics",null,null,"Might occur",5,null,"Problem detection at Source","Failure prevention considered, 100 % checks on output, medium chance of escape",5,"Moderate"]
[4,"Moderate","Bias causes problems which have a noticeable impact on business performance metrics.  Medium  dissatisfaction, e.g. extra effort or rework needed or moderate impact on ethics or business metrics",null,null,"Might not occur",4,null,"Problem detection post-processing","Failure prevention designed into system, 100% recorded audit of process. Low chance of escape.",4,"Moderately high"]
[3,"Low","Bias causes minor problems which take a small amount of time to overcome. Some dissatisfaction and/or low impact on ethics or business metrics",null,"Low","Unlikely",3,null,"Problem detection at Source","Mistake proofing used to try and prevent failure. 100% recorded audit of process. Small chance of escape",3,"High"]
[2,"Very low","Nature of Bias only causes slight delay or minor rework. Slight  annoyance/very low  impact on ethics or business metrics",null,null,"Very unlikely",2,null,"Error detection and / or problem prevention","Controls nearly always prevent or detect failure. Remote chance of escape",2,"Very high"]
[1,"Minor","The customer(s) of the process will not notice the effect of the Bias. No impact on ethics or business metrics",null,"Very low","Almost impossible for it to occur",1,null,"Detection not applicable; Error Prevention (before any failure)","Controls will always prevent failure from occuring or, if it does happen, will always detect the problem before it escapes",1,"Almost certain"]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
["Password"]