Seminar des Fachausschusses Ultraschallprüfung - Vortrag 9Ditii ttiti l (hit/miss, maxdistribution...
12
O i fh Si l i Overview of the Simulation‐ Supported POD Approach in the CIVA Software Frédéric Jenson, Philippe Benoist CEA, LIST, 91191 Gif‐sur‐Yvette, France Background • Inspection reliability: one of the key issues in ensuring safety of Evaluation of POD Curves Based on Simulation Results Inspection reliability: one of the key issues in ensuring safety of critical structural components • Increasing use of probabilistic approaches based on statistical criteria such as POD curves and PFAs • Are currently obtained thru expensive and time consuming experimental campaigns Obj ti • To propose tools to replace some of the experimental data with simulation results such as those obtained with the software CIVA Objective Seminar des Fachausschusses Ultraschallprüfung - Vortrag 9 Lizenz: http://creativecommons.org/licenses/by-nd/3.0/de 1
Transcript of Seminar des Fachausschusses Ultraschallprüfung - Vortrag 9Ditii ttiti l (hit/miss, maxdistribution...
O i f h Si l iOverview of the Simulation‐Supported POD Approach in the
CIVA Software
Frédéric Jenson, Philippe BenoistCEA, LIST, 91191 Gif‐sur‐Yvette, France
Background
• Inspection reliability: one of the key issues in ensuring safety of
Evaluation of POD Curves Based on Simulation Results
Inspection reliability: one of the key issues in ensuring safety of critical structural components
• Increasing use of probabilistic approaches based on statistical criteria such as POD curves and PFAs
• Are currently obtained thru expensive and time consumingexperimental campaigns
Obj ti
• To propose tools to replace some of the experimental data with simulation results such as those obtained with the software CIVA
Objective
Seminar des Fachausschusses Ultraschallprüfung - Vortrag 9
Lizenz: http://creativecommons.org/licenses/by-nd/3.0/de
1
Outline
POD curves evaluation based on a sampling approach
Definition of the uncertainty sourcesUncertainty propagation thru the physical modelUncertainty propagation thru the physical modelExtraction of the output (the variable of interest)Computation of the POD curve and the lower confidence bound
Validation case
Validity of the simulated POD curve : error sources
HFET of fatigue cracks in Titanium
Validity of the simulated POD curve : error sources
Future work and objectives for CIVA
Aim of the approach: first, simulate a set of “realistic values” for the NDI system response
• « realistic values»: values that take into account uncertainty and noise sources and thus reflect fluctuations around the theoretical responses
Signal response
Signal response
p
Fluctuations
Theoretical response
Flaw dimension Flaw dimension
Deterministic approach Statistical approach
2
Deterministic case
… then, perform an estimation of the POD curve using a functional form
Statistical case
Flaw dimension
Signal response
Threshold
POD
Flaw dimension
Signal response
Threshold
POD
Estimation of μ, σ and confidence bound
Flaw dimension
0
1
Flaw dimension
0
1Lower confidence
bound
Functional form (depends on the estimates of μ, σ )
POD’s in CIVA Software: a sampling approach based on existing physical models
RandomRandom input
parameters CIVA Modeling
(UT, ET)
Random Response
+ Noise
Probabilistic characterization of
inspection capability: POD, PFA, ROC, …
),( dXhY
1. Definition of the inspection setup 2. Description of uncertainties on a set of input parameters3. Propagation of uncertainty thru existing models (Monte Carlo)4. Extraction of the output (the variable of interest)5. Evaluation of probabilistic criteria such as the POD, PFA
3
Evaluation of POD Curves Based on Simulation Results
Factors affecting the probe signal response due to a flaw
Factors that can be adressed using physical models
response due to a flaw
• NDI system: transducer, scan plan, electronic device, probe orientation
• Part: geometry, material properties, structural properties, surface roughness
NDI systemPart
Flaw
• Flaw: size, shape, orientation, position, material properties
Factors that are difficult/impossible to address using physical models
Some “human factors” : psychological factors, physiological factors
These factors are either deterministic or random during the inspection
Evaluation of POD Curves Based on Simulation Results
Factors that are thought to be deterministicg
Factors that are thought to be random or uncertain
A signal‐determining factor is taken to be deterministic if either the factor can be controlled in the inspection operation or if it is desired to estimate POD as a function of the level factor (flaw size for instance).
The factor is thought to be random during the production/field inspection if there is insufficient knowledge related to it, if it is not well controlled or if it implies physical phenomena with inherent randomness.
4
Examples of random (uncertain) factors:
Evaluation of POD Curves Based on Simulation Results
• Detail of metallurgical macro and microstructure: structural noise, beam distortions, beam deviations
• Flaw morphology: for a fixed size, various parameters such as shape, orientation, position, elastic properties, and conductivity can vary in a random manner
• Probe positioning: Lift‐off variations and/or probe orientation during a manual p g / p ginspection using an eddy current pencil probe
• …
Modeling uncertainty sources using statistical distributions
ProbeProbe
θ S
θnominal Snominal
Simulated responses
SkewNotchNotch
0.16
0.18
θi Proposed parametric distributions:
UniformNormal
θ1 S1
θ2 S2
θn Sn
-15 -10 -5 0 5 10 150
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Angle [°]
PD
F
Statistical distribution for the skew angle
Dedicated GUI
NormalLog‐normalRayleighExponential…
Defined using engineering judgment
5
ânom
Performing uncertainty propagation using a sampling approach (Monte Carlo)
θ1
θ2
θn
θnominal
Extraction
â1
â2
ânSkew
D i ti i t ti ti l Extraction(hit/miss, max amplitude…)
Simulated responsesModel
computation
Random sampling
Description using a statistical distribution
Skew
Probability
Mean value
Kirchhoff
Simulations are based on existing physical models
Physical models :
• UT beam modeling using ray tracing (pencil method)
• UT beam-defect modeling using Kirchhoff approximation, Geometrical Diffraction Theory, Born approximation
• ECT field modeling using Finite Integration Technique, Analytical expressions
ECT d f t d li i V l I t l M th d• ECT defect response modeling using Volume Integral Method
6
Estimation of the POD curve using a functional form
• POD curves relate the detectability of a flaw to its size (or to another geometrical characteristic)
• Usual approach (MIL HDBK 1823) consist in: pp ( ) assuming a functional form for the POD curve estimating the parameters of the function from the inspection results estimating the associated confidence bound
• Hit/miss data format:
1ln
3exp1)(
aaPOD
Log‐odds functionBerens
• Signal response data format: /ln)( aaPOD
Cumulative log‐normal distribution function
• Estimated from a finite size data sample sampling uncertainty
confidence bound
Berens
Cheng
Part NDT
Material: Titanium (TA6V) Configuration: High Frequency Eddy Currents Testing (HFET)
Validation case: HFET of fatigue cracks in Titanium
Geometry: Flat areas
Currents Testing (HFET)
Probe: Pencil probe (2MHz)
Defects: Fatigue cracks
Conditions: In‐service (manual)
7
• Characteristic variable: crack length (mm)
• Uncertain parameters:
Design of Numerical Experiments HFET on Titanium
Start scan position Crack height (mm) Angle of the probe (°)Start scan position Crack height (mm) Angle of the probe ( )Corresponds to the position of the probe for picking the maximum amplitude signal
Fatigue cracking is subject to manyuncertainties
Translated into an additional lift‐off using geometrical rule
0.2
0.4
0.6
0.8
1
Pro
ba
bili
ty d
en
sity
1
1.5
2
2.5
3
3.5
4
Pro
ba
bili
ty d
en
sity
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Pro
ba
bili
ty d
en
sity
Uniform in [‐0.5;05](scan increment=1mm)
Gaussian with dependency to the crack length (fatigue crack)
0.5*length +N(0,1)*0.12*lengthGaussian(0°;1°)
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
Y start position (mm)0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
0.5
Crack height (ratio of length)-4 -3 -2 -1 0 1 2 3 4
0
0.05
Probe angle (°)
Sampling strategy: Monte Carlo
– Experimental database
• 69 cracks from 0.33 mm to 6.66 mm
• 5 operators
Simulated vs. Experimental data
20
40
60
80
100
Sig
na
l re
spo
nse
(%
FS
H)
Experimental data
– Simulations
• 100 crack lengths from 0.25 mm to 5.0 mm
• 6 samples per crack length
0 1 2 3 4 5 6 70
Crack length (mm)
8
100
Signal response data analysis
Exp=3.4mmSimul=3.6mm
First saturated data
80
90
100
110
FS
H)
Linear‐linear plot
0 1 2 3 4 5 6 70
20
40
60
80
Crack length (mm)
Sig
na
l re
spo
nse
(%
FS
H)
Measured data
Simulated datawith uncertaintiesDeterministic
4
4.5
5
â (
log
)
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.510
20
30
40
50
60
70
80
Crack length, a (mm)
Sig
na
l re
spo
nse
, (%
Data
MLE regression
MLE+delta
MLE-delta
All cracks are detected
Log‐log plot
Exp=1.4mm
Simul=1.5mm
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
2.5
3
3.5
4
Crack length, a (log), mmS
ign
al r
esp
on
se,
Data
MLE regression
MLE+delta
MLE-delta
Good agreement on data until 3 mm cracks.
High variability for larger cracks not well represented
Signal response data analysis
High variability for larger cracks may be due to the complex crack shapes withpossible electrical contacts between the two faces of the crack aperture. Thesecontacts are responsible for signal amplitude dropscontacts are responsible for signal amplitude drops.
Dataset is augmented with an additional uncertain input parameter: the rateof electrical contacts along the crack length.
30
35
40
45
50
con
tact
s (%
)
0 1 2 3 4 50
5
10
15
20
25
30
Crack length (mm)
Ra
tio o
f ele
ctric
al c
9
Signal response data analysis
100
Exp=3.4 mmSimul=2.9 mm
First saturated data
80
90
100
110
FS
H)
Linear‐linear plot
0 1 2 3 4 5 6 70
20
40
60
80
Crack length (mm)
Sig
na
l re
spo
nse
(%
FS
H)
Measured data
Simulated data
Simulated datawith electrical bridges
Exp 1 4mm
All cracks are detected
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.510
20
30
40
50
60
70
Crack length, a (mm)
Sig
na
l re
spo
nse
, (%
Data
MLE regression
MLE+delta
MLE-delta
4
4.5
5
â (
log
)
Log‐log plot
Exp=1.4mm
Simul=1.55 mm
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
2.5
3
3.5
Crack length, a (log), mm
Sig
na
l re
spo
nse
,
Data
MLE regression
MLE+delta
MLE-delta
Adding electrical contacts as an uncertain inputparameter allows for reproducing the highvariability in the whole crack length range.
Hit/Miss POD analysis: 4 input uncertain parameters
• 600 simulated data• 4 uncertain parameters (start position, crack height, probe angle, elec. contacts)
10
20
30
40
50
60
70
80
90
100
Pro
ba
bili
ty O
f De
tect
ion
(%
)
Observed ratio (experimental)Estimated POD (experimental)
POD 95 % conf. (experimental)
Observed ratio (simulation)Estimated POD (simulation)POD 95 % conf. (simulation)
10
20
30
40
50
60
70
80
90
100
Pro
ba
bili
ty O
f De
tect
ion
(%
)
Observed ratio (experimental)
Estimated POD (experimental)
POD 95 % conf. (experimental)Estimated POD (simulation)
POD 95 % conf. (simulation)
POD curves very similar•Slope corrected but still the effect of the number of data
•Confidence band smallerfor simulation dataset becausemore data than in the experimental dataset
mma
mma
8.1
5.1exp
95/90
exp90
mma
mmaECsimu
ECsimu
7.1
5.1,
95/90
,90
0.5 1 1.5 2 2.5 3 3.50
10
Crack length (mm)0.5 1 1.5 2 2.5 3 3.50
10
Crack length (mm)
O 95 % co (s u at o )
Empirical POD (simulation)
Values of interest verysimilar
10
Validity of the simulated POD curve : error sources
When determining a POD curve from simulation results, various error sources may impact the results:
Sampling uncertainty ‐> lower confidence boundp g y
Errors on the estimation of the lower confidence bound ‐> if sample size is to small. Need more simulations.
Hypothesis of the POD model: linear relationship, constant standard deviation…
Modeling errors ‐> systematic error (bias on the estimates of the parameters of the curve). Need validation of the models/ definition of the validity domain.
Errors on the definition of the inputs ‐>choice of the uncertain parameters of the inspection, of the statistical distributions describing them and of the parameters of the distributions. Need more engineering judgment.
Perspectives
•Additional tools are needed for the modeling of uncertainty sources: take into account correlations between input parameters
Parameter 1
Param
eter 2
Param
eter 2
Parameter 1
Example: Correlations between the length and the height of a crack
• Perform experimental validation for EC and UT application cases
• New approaches for the modeling of the POD curve
PICASSO Project : European project (FP7), started in 2009
11
Thank you for your attention
Questions?
12
