IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 65, NO. 8, AUGUST 2018
1879
SEE Error-Rate Evaluation of an Application
Implemented in COTS Multicore/
Many-Core Processors
Pablo Ramos , Vanessa Vargas , Maud Baylac, Nacer-Eddine Zergainoh, and Raoul Velazco
Abstract— This paper evaluates the error rate of a
memory-bound application implemented in different commercialoff-the-shelf multicore and many-core processors. To achieve
this goal, two quantitative experiments are performed: faultinjection campaigns and radiation ground testing. In addition,
this paper proposes an approach for predicting the application
error rate by combining the results issued from both types of
experiments. The usefulness of the approach is illustrated by
three case studies implemented in processors having different
manufacturing technologies and architectures: 45-nm siliconon-insulator (SOI) free-scale P2041 quad-core processor, 65-nm
CMOS Adapteva Epiphany multicore processor, and 28-nm
CMOS Kalray multipurpose processing array-256 many-core
processor. The reliability of the processors for avionics is obtained
from their experimental error rates extrapolated to avionic
altitudes. Reliability curves are plotted for observing the prediction accuracy. A comparison of the failure in time of the
selected processors shows that the greater single-event effect
vulnerability of CMOS technology compared with the SOI one
can be compensated with the implementation of effective error
detection and correction. These protection mechanisms allow
the use of CMOS devices having huge memory capacity in
applications operating in severe radiation environments.
Index Terms— Accelerated testing, error rate, fault injection,
many core, multicore, reliability, single-event effect (SEE), singleevent upset (SEU), soft error.
I. I NTRODUCTION
T
HE actual trend in computing systems is the use of
multicore and many-core processors for achieving high
performance, reliability, and low power consumption. The
high processing capacity and intrinsic redundancy capability of
these devices make them ideal for implementing applications
ranging from high performance to embedded systems domain.
Hence, aerospace and avionics industries are interested in
validating their use for critical applications [1], [2].
Manuscript received April 7, 2018; revised May 3, 2018; accepted May 13,
2018. Date of publication May 18, 2018; date of current version August 15,
2018. This work was supported in part by the Universidad de las Fuerzas
Armadas ESPE, in part by the Secretaría de Educación Superior, Ciencia,
Tecnología e Innovación del Ecuador under Grant PIC-2017-EXT-005, and in
part by the STIC-AmSud-EnergySFE Project.
P. Ramos and V. Vargas are with the Departamento de Eléctrica
y Electrónica, Universidad de las Fuerzas Armadas–ESPE, Sangolqui
171-5-231B, Ecuador (e-mail: pframos@espe.edu.ec; vcvargas@espe.edu.ec).
M. Baylac is with the LPSC Laboratory, CNRS, Université Grenoble Alpes,
4070038058 Grenoble, France (e-mail: baylac@lpsc.in2p3.fr).
N.-E. Zergainoh and R. Velazco are with the TIMA Laboratory, CNRS,
Université Grenoble Alpes, Grenoble, France.
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TNS.2018.2838526
In avionic system architecture, the new tendency is to rely
on integrated modular avionic (IMA) instead of the classical
federated architecture. The main difference between both
architectures is that in the federated one, each system has
private resources while in IMA, the resources can be shared.
One of the most important challenges for IMA architectures
is the integration of commercial-off-the-shelf (COTS) multicore processors. The use of COTS multicore processors is
convenient due to budget and availability issues. Nevertheless, the selection of these components requires certification
against errors. One of the main concerns of certification for
critical-embedded systems is the sensitivity to radiation of
electronic components. The interaction of energetic particles
with the semiconductor material may result in a wide range
of transient and permanent failures, called single-event effects
(SEEs). Among them, the single-event upset (SEU) is the most
representative since it may change the content of a memory
cell affecting the device reliability.
The high degree of miniaturization, huge number of memory
cells, and complexity of multicore/many-core architectures
increase their vulnerability to SEEs. Fortunately, the inherent
redundancy capability of these processors makes them suitable
for implementing fault-tolerant techniques which significantly
mitigate radiation effects [3]. In addition, protection mechanisms such as error-correcting codes (ECCs) and parity are
commonly implemented in cache and shared memories in
order to improve their reliability.
For multicore processors intended to be used in avionic
applications, the reliability is a mandatory issue to
consider [4]. It can be evaluated through the device’s failure
rate which can be obtained by extrapolating the cross section
to the operating radiation environment. Since the failure rate
of a processor is application dependent, any change in the
application will require new tests. Unfortunately, the costs of
radiation tests and the availability of radiation facilities make
them difficult. Consequently, it is necessary to apply a lowcost strategy to estimate the application error rate.
This paper proposes an approach to evaluate the SEE error
rate based on the principles of the code emulating upset (CEU)
strategy, which combines fault-injection with radiation ground
testing [5]. The proposal is evaluated by means of three case
studies using multi cores/many cores with different process
technologies and architectures. The comparison of the estimated error rates to the results issued from radiation tests
illustrates the pertinence of the method [6]. Obtained results
are extrapolated to avionic altitudes in order to compare the
0018-9499 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
1880
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 65, NO. 8, AUGUST 2018
predicted and measured reliability curves. This paper is based
on the first author’s Ph.D. dissertation [7].
The remainder of this paper is organized as follows.
Section II presents the related work. In Section III, the adopted
approach for estimating the application error rate is described.
Section IV details the case studies. The experimental results
are presented and analyzed in Section V. Finally, Section VI
concludes this paper and provides directions for future work.
simulators: MaFIN for MARSS simulator, and GeFIN for
Gem5 simulator for x86 and ARM architectures. The results
present a vulnerability analysis of different hardware structures
(registers, queues, and cache memories) and the vulnerability
differences of both architectures.
Rosa et al. [14] present the OVPsim-FIM, which is a
simulation-based fault-injection framework to evaluate multicore systems’ soft-error reliability. The framework uses
processor models and supports parallel simulation considering
real-time operating system. This framework allows targeting
several cores in parallel reducing fault-injection time. Experiments were performed on ARM A9 and A15 processor models
targeting general purpose registers (GPRs). The results show
an overhead from 2.3% to 12.6% depending on the target
application.
Application-level fault-injection tools are commonly applied
to COTS processors. Oliveira et al. [15] present CAROL-FI
fault injector which identifies the most sensitive zones of
the application code running on high-performance computing
systems. It works in the debug mode introducing an overhead
about 4–8 times the standard execution time. The target device
was the Intel Xeon Phi coprocessor 3120A. This work is
similar to our previous work presented in [16] that evaluates
fault injection on parallel applications running on multicore
processors.
Hari et al. [17] present SASSIFI (source and assembler
instrumenter-based fault injector), which is an architecturelevel fault-injector tool for NVIDIA graphic processing
unit (GPU) application resilience evaluation. This tool can
inject faults in GPRs, GPU memory, condition code, and
predicate registers. The strategy is based on the instruction
instrumentation in the low-level GPU assembly language.
Summarizing, MaFIN, GeFIN, and OVPsim-FIM are faultinjection tools based on simulation and require processor
models. Then, their main limitation is the availability of
the models and the necessity of a deep knowledge of the
architecture. In addition, working with simulation models does
not represent the real behavior of physical devices. Regarding
application-level fault injectors, they cannot be used for SEE
error-rate estimation purposes since they are limited to target
the application itself and not the whole system. In turn,
SASSIFI is a dedicated fault-injection tool for NVIDIA GPUs
which uses proprietary instrumentation handlers.
In contrast, the approach presented in this paper is a generic
strategy that can be implemented in any multicore device and
does not require a processor model. Also, the overhead of this
approach is less than the one presented by OVPsim-FIN and
CAROL-FI, as shown in Section V-B. However, the source
code of the application is required. Fortunately, several source
codes of benchmarks for testing high-reliability systems, such
as those proposed in [18], are available.
II. R ELATED W ORK
In this section, some relevant works dealing with error-rate
estimation in multicore processors are summarized. Considering that our proposal combines fault injection with radiation
experiments, the most significant fault-injection works on
multi cores are briefly discussed.
A. Error-Rate Estimation Studies
Cabanas-Holmen et al. [8] propose an approach for predicting the SEE error rate of a radiation hardened by
design (RHBD) processor based on the sensitivity of the
constituent circuits. Heavy-ion radiation campaigns were performed to validate the approach on an RHBD version of
the Advanced Reduced instruction set computing (RISC)
Machines (ARM) CortexTM R4 core and single-core Tilera
processor. In [9], it is described the structure’s architectural
vulnerability factor (AVF), which is the probability that a
fault in a particular structure will result in an error. This
method generates accurate estimates of processor error rate by
combining the raw error rate of the structure with the AVF.
Both methods are suitable for system designers but not easily
applicable for evaluating COTS devices since they require a
detailed knowledge of the system architecture and building
blocks, which are not available for end users.
Stolt and Normand [10] establish an SEE cross section
model for a multicore server based on a quad-core processor
built in 45-nm-bulk CMOS technology. The target device was
an HP c7000 BladeSystem designed for aircraft altitudes. The
validation of the approach was done by radiation experiments
with 14-MeV neutrons. This model presents a good error-rate
estimation but is dedicated to the target system.
Velazco et al. [5] presented the CEU, which is an errorrate estimation approach based on software-implemented fault
injection and the worst-case sensitivity of the device. This
approach designed for monocore processors has been demonstrated to be very effective to provide error-rate results close
to those obtained in radiation tests [11], [12].
In [6], CEU approach principles were applied to the Kalray
multipurpose processing array (MPPA) many-core processor to
estimate the application error rate. The closeness between the
estimated and obtained values motivates the use of this strategy
to other processors. Hence, this paper improves the approach
by adding derating factors and extends its applicability to other
devices.
B. Fault Injection on Multicore Processors
Kaliorakis et al. [13] implement, study, and compare the
effectiveness of two fault injectors for microarchitectural
III. A DOPTED A PPROACH
The proposed approach for predicting the error rate of
applications implemented in multicore and many-core processors is based on the principles of the CEU strategy [5].
In order to estimate the error rate (τSEU ) of an application
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
RAMOS et al.: SEE ERROR-RATE EVALUATION
1881
implemented in a processor, the CEU approach combines the
error rate issued from fault-injection campaigns (τInj ) with
the intrinsic sensitivity of the device (σStatic) obtained from
radiation experiments as described in the following equation:
τSEU = τInj × σStatic .
(1)
The aim of CEU fault-injection approach is to reproduce
the effects of SEU faults in accessible memory cells. This
can be done by asserting asynchronous interrupt signals by
an external device. The execution of the interrupt handler in
the target processor produces a bit flip in a randomly chosen
memory cell. For monocore processors, this implementation
does not need a deep architectural knowledge of the target.
However, for multicore/many-core processors, there are
several constraints that have to be overcome in order to
implement the fault injector due to the device complexity,
mainly related to memory management, synchronism, and
intercore communications. To minimize the difficulty of fault
injection, the proposed strategy benefits of the multiplicity of
cores by using one of them as a fault injector while the others
run the chosen application.
The purpose of this paper is to extend the CEU approach to
multicore/many-core processors and evaluate the effectiveness
of the prediction by targeting accessible memory cells by software means. Considering that multicore/many-core processors
implement different types of memory cells, the total error rate
should be expressed as the sum of the individual contribution
of each component.
This approach has two main limitations: 1) upsets occurring
during the instruction execution cannot be simulated and 2) it
is not possible to target all possible sensitive areas such as
internal flip-flops, control unit, and latches inside processor’s
architecture. However, considering that advanced processors
implement different levels of cache memories as well as a
growing internal Shared MEMory (SMEM), the targetable area
comprises most of the total sensitive area of the device, which
provides a significant validity to this approach.
A. Fault Injection Strategy
The extended CEU strategy considers a master-slave scheme
where the master core performs as fault injector, whereas the
slave cores execute the selected application. The master core
initializes data that is going to be used by the other cores, and
sends a message through an interprocessor interrupt to indicate
the slave cores to start the execution of the application. While
the application is running on the slave cores, the master core
performs the fault injection. It randomly selects the target core,
the injection instant (in terms of clock cycles), the address,
and the bit to be altered. When slave cores finish the execution
of the application, the master core compares the resulting data
with a set of golden results. To guarantee the independence
between the fault injector and the other cores, the fault injector
must be isolated. When the target device is a coprocessor,
the fault generation is performed by the host processor and
the fault-injection parameters are passed to master core to
guarantee randomicity in data. To inject a bit flip on the
selected target, the following tasks are done:
•
reading the content of the target memory cell;
performing an XOR operation with an appropriate mask
value that contains a “1” for the bits that are going to be
flipped and “0” elsewhere;
• writing the corrupted value to its original location.
The targeted zones are GPRs and special function registers
belonging to each processor, as well as the SMEM of the
multicore/many-core device. In addition, faults on cache memories are emulated by injecting errors in the main memory,
so that corrupted data are retrieved by caches. In order to
perform fault injection in memory area, the variables to be
used by the application are placed in the internal SMEM of
the device. In this manner, the variables can be modified at
any time by any core processor.
Concerning fault injection in processor’s registers, only
accessible registers can be modified. Due to the fact that master
core has no access to other cores’ registers, it can execute
an indirect fault injection via an intercore interruption to the
selected core, in which the interruption handler launches a
code that emulates bit flips as previously described.
The results of fault injection can be classified as follows.
• Masked Fault: It occurs when the injected fault does not
cause any consequence in the result of the program.
• Silent Data Corruption: Produces a corrupted result in
the application output.
• Exception: The output of the program halts. It is primarily
caused by faults injected on critical registers.
• Time-Out: When the program process does not respond
after a duration equal to 1.2 times the standard execution
time.
The SEU error rate of an application (τInj ) is derived from
fault injection. This quantity is defined as the average number
of injected faults needed to produce an error (exception, timeout, or silent data corruption) in the application
Number of Errors
.
(2)
τInj =
Number of Injected Faults
1) Intrusiveness: As this strategy uses one core as a
fault injector, the processing capacity of the chip is reduced
inversely proportional to the number of cores of the device.
For multi cores having few cores, this reduction is significant,
while for many cores having hundreds of cores, it is negligible.
2) Overhead: When injecting faults in the SMEM, there
is no overhead in the application execution since the fault
injector produces directly the bit flip. On the other hand,
when targetting processor’s registers, the overhead is related
to two aspects: 1) the time needed to produce the intercore
interruption and 2) the execution of the CEU code. The
first one depends on the complexity of the communication
infrastructure. For devices with a few number of cores, it is
very small while for a device with hundreds of cores, the time
increases accordingly to the use of buses and network on chip
(NoC). The second one is related to the time needed to execute
the interruption handler.
•
B. Radiation Experiments
Two types of accelerated radiation experiments are considered for evaluating the sensitivity of the device: a static test
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
1882
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 65, NO. 8, AUGUST 2018
to obtain the intrinsic sensitivity of the device and a dynamic
test for evaluating the dynamic response of the application to
compare it with the predicted value. As a result, the static
(σStatic ) and dynamic cross sections (σDyn ) of the device are
obtained. The cross section is the average number of particles
needed to produce a bit flip in a memory cell, and is defined
as follows:
Number of Upsets
.
(3)
σ =
Fluence
In this paper, both radiation tests have been conducted
with 14-MeV neutron beam to simulate the effects of highenergy neutrons present at avionic altitudes, since neutrons
are the most representative particles in the earth atmosphere.
Sections III and VI of the JESD89A document of the JEDEC
standard were used as a base protocol for the tests [19].
1) Static Test: The device under test (DUT) is placed facing
the center of the target perpendicularly to the beam axis at a
distance depending on the required radiation flux. Typically,
the method consists in writing a predefined pattern in memory
locations and accessible registers of the processor via the
instruction set (load and store). Once data initialization is
finished, the DUT is irradiated and the program checks periodically the registers and memory locations along the radiation
test to detect upset events. If an upset is detected, the program
writes the correct pattern in the associated memory location
and logs the results to an external host via serial or ethernet
ports. During the static test, all the sensitive zones are exposed
to radiation at the same time which do not represents the
normal behavior of the circuit, since not all the memory
resources are used simultaneously when an application is
executed. For this reason, the static cross section provides the
worst-case estimation of the device sensitivity. If the target
device implements protection mechanisms, it is necessary to
use the machine-check error report for logging corrupted data.
2) Dynamic Test: This test aims at evaluating the sensitivity
of the application executed in the processor. The strategy
consists in the periodic execution of an application while the
device is being irradiated. Errors are detected at the end of the
execution by comparing the results of the application with a
set of golden results.
while this communication is performed, since SEUs affecting
the internal memory of the device are not observed because of
data reinitialization at the beginning of each execution. This
factor is given by (5)
St H −C
(5)
Etf = 1 −
Texec
where St H −C is the synchronization time between the host
processor and the master core, and Texec is the execution
time of the application. Finally, the complete equation of the
proposed predicted error rate is illustrated in (6)
C. Derating Factors
For improving the accuracy of the error-rate prediction,
the proposed strategy considers the addition of derating factors
to the error rate contributed by shared and cache memories.
These factors depend on the memory used by the application
and the exposure time to the radiation of shared and cache
memories. The memory utilization factor (Mf) is the amount
of memory used by the application with respect to the total
memory of the device
Used memory
.
(4)
Mf =
Available memory
In some particular cases, where the multicore/many-core
processor performs as a coprocessor of a development board,
it is possible to need synchronization between the coprocessor
and the host processor for logging results. This fact leads to
a loss of exposure time to radiation of the internal memories
τSEU = τInj × σStatic × Mf × Etf.
(6)
IV. E VALUATION OF THE P ROPOSED A PPROACH
BY M ULTIPLE C ASE S TUDIES
For evaluating the proposed approach, this paper considers
the use of multiple case studies configured in bare metal
programming model. The selected devices are being the focus
of interest of different international projects in the avionic
domain. The effectiveness of the approach is evaluated by
comparing the predicted error rate with the measured one
obtained from dynamic radiation experiments.
A. System Configuration
The target processors were configured in asymmetric multiprocessing mode to guarantee independence between the fault
injector and the other cores. The selected programming model
was bare metal to maximize the fault-injection targeted zone.
The absence of operating system (OS) allows targeting memory cells that are not accessible by the user and injecting bit
flips directly in the SMEM. It also permits the use of intercore
interrupts minimizing the injection time and the intrusiveness.
However, the implementation of the fault injector in bare
metal requires low-level programming. In addition, the fact
that the fault injector is based on intercore communication
increases the difficulty in programming because there are many
interconnections that must be handled in a multicore/manycore device.
B. Targeted Devices
Relevant characteristics of the selected target processors are
summarized in Table I. In addition, complementary information of each device architecture is detailed below.
1) Free-Scale QorIQ P2041: It is a quad-core processor
manufactured in 45-nm silicon-on-insulator (SOI) technology.
It operates up to 1.5 GHz with a maximum power consumption of 12 W. It is based on four e500-mc cores built
on power architecture technology with high-performance data
path acceleration architecture. CoreNet fabric infrastructure,
as well as network and peripheral bus interface, is required
for networking. The memory architecture includes a threelevel cache hierarchy. Each e500-mc core implements independent on-chip 32-kB L1 caches for instruction and data,
and a unified 128-kB backside L2 cache. The four cores
share a 1-MB CoreNet platform cache fronting the memory
controller. The implemented error detection and correction
mechanisms are specified in Table I. Its SOI implementation
makes this device immune to latch-up events.
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
RAMOS et al.: SEE ERROR-RATE EVALUATION
1883
TABLE I
TARGET D EVICE C HARACTERISTICS
2) Adapteva Epiphany EG16301: It is a scalable multicore architecture sharing a common 32-bit memory space.
It defines a parallel computing fabric comprised a 2-D array
of processors nodes connected by a low latency mesh NoC.
The E16G301, which is based on the third generation of
the Epiphany multicore IP, is a 16 core system on chip
implemented in a 65-nm CMOS technology. It operates at
maximum 1-GHz frequency with a chip power consumption
less than 2 W. Each processor core is a 32-bit superscalar
floating point RISC CPU. The device has a peak performance
of 32 Gflop. The local memory is comprised four independent
banks, each one of 8 kB for a total of 32 kB for each CPU
core, resulting in 512 kB of on-chip memory.
3) Kalray MPPA-256: It is a many-core processor manufactured in Taiwan Semiconductor manufacturing Company
CMOS 28HP technology. The processor operates between
100 and 600 MHz for a typical power ranging between 15 and
25 W. Its peak floating-point performances at 600 MHz are
634 and 316 Gflop for single and double precision, respectively. The processor architecture is clustered with 16 compute clusters (CCs) and two input-output (I/O) clusters per
device. Each CC is built around a multibanked local static
memory (SMEM) of 2 MB shared by 16 processing engine
cores and one resource manager (RM) core. Each I/O cluster
has two quad-cores and two main memory banks of 2 MB.
The MPPA-256 cores are based on the same very long
instruction word (VLIW) 32-bit/64-bit architecture. The VLIW
core implements separate data and instruction cache. The main
components of the device are covered by error-protection
mechanisms specified in Table I.
V. E XPERIMENTAL R ESULTS
Target processors execute a standard matrix multiplication (MM) as application since matrix algorithms are useful for
evaluating the reliability of parallel processors [18]. The size
of the matrix was selected to maximize the use of resources
depending on the available memory space of each device. For
the P2041 and the Epiphany processors, a sequential version
of the MM was independently implemented on each processor
core. On the other hand, the MPPA-256 implements a parallel
version of the MM on each one of the 16 CCs. Since there is
no OS, the cores execute the application exclusively, thus maximizing the workload. Table II summarizes the characteristics
of the application.
TABLE II
C HARACTERISTICS OF THE A PPLICATION
TABLE III
S ENSITIVE Z ONES OF TARGETED D EVICES
A. Radiation Experiments
Radiation experiments were conducted with 14-MeV neutrons in GEnerator of NEutrons Pulsed and Intense particle
accelerator located in Grenoble, France. The neutron flux
applied to each device differs because of the sensitivity to
latch-up events of the manufacturing technology, and the circuitry density of the development board containing the DUT,
which favors the corruption of other elements than the multicore/many core. For instance, in the Parallella board, the host
processor ARM A9 was affected by 2 single-event functional
interruption (SEFI) during the static test.
The results of radiation experiments on the P2041 multicore
and the MPPA-256 many core were published in [20] and [6],
respectively. The experimental results on the Epiphany
EG16301 are new results and are presented in the following
subsection. Table III summarizes the sensitive zones of the
targeted processors.
1) Radiation Experiments on Epiphany EG16301: The
EG16301 is a 16 core microprocessor which performs as a
coprocessor on the Parallella Adapteva board.
a) Intrinsic sensitivity: The intrinsic sensitivity of the
multicore processor is evaluated by performing a static
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
1884
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 65, NO. 8, AUGUST 2018
TABLE IV
TABLE VII
S TATIC T EST R ESULTS
FAULT I NJECTION C AMPAIGNS
TABLE V
DYNAMIC T EST R ESULTS
TABLE VI
95% C ONFIDENCE I NTERVALS OF R ADIATION E XPERIMENTAL R ESULTS
radiation test targeting the internal memory and registers of
each core. The host processor of the Parallella board was
in charge of filling the internal memory of the Epiphany
multicore with a predefined pattern. For this reason, the whole
internal memory of the multicore was targeted.
During the static tests, 23 SEUs and one multiple cell upsets
that produce bit flips in the internal memory were detected. For
protecting the circuitry other than the DUT, the development
board was surrounded by a 5-cm polypropylene block. The
multicore was irradiated through a small window in the block
with a fluence around 2.59 × 108 n · cm−2 . In addition,
an antilatch-up circuit, controlling the level of input current,
was used to avoid destructive events. The results from the
static test allow estimating the static cross section σSTATIC of
the 65-nm CMOS Epiphany device as follows:
σSTATIC =
2
24
−8 cm
=
9.27
×
10
.
2.59 × 108
device
b) Dynamic response: The dynamic cross section
σDYNAMIC of the multicore processor is obtained by performing radiation experiments when the application is running on
the device. The fluence was around 2.59×108 n·cm−2 . During
the test, 11 SEUs that produced silent data corruptions and
1 SEFI that produces a hang were observed. The calculated
dynamic cross section is
σDYNAMIC =
2
12
−8 cm
.
=
4.63
×
10
2.59 × 108
device
2) Radiation Experimental Summary: Retrieving the experimental results of the P2041 multicore and MPPA-256 many
core from [20] and [6], Tables IV and V summarize the
radiation experiments of the three targeted devices.
Due to the scarcity of the collected experimental data during
the radiation tests, 95% confidence intervals were applied to
the obtained results, as shown in Table VI.
TABLE VIII
E RROR -R ATE P REDICTION
B. Fault-Injection Campaigns
In order to select the targeted zones for the faultinjection campaigns, the results of radiation experiments were
considered. For the P2041 and the Epiphany processors, only
cache and shared memories were taken into account since
there were no errors observed in registers during the static
test. On the contrary, for the MPPA-256, only registers were
considered since no errors were detected in the internal shared
memories. For emulating fault injection in cache memories
in the P2041, faults were injected on the external memory,
and then, corrupted data were retrieved by caches. Regarding
the fault injector, both multicore processors use one of the
processing elements, whereas the MPPA many core uses the
RM core of each cluster. Table VII summarizes the faultinjection campaigns and their results.
During experimental campaigns, the overhead caused by
fault-injection was measured. In the MPPA-256 case, it was
less than 0.70 Mcycle (1.75 ms at 400 MHz) representing
0.004% of the application. For the Epiphany, the overhead was
0.13 Mcycle (129 μs at 600 MHz) representing 0.7%. Finally,
for the P2041, the overhead was about 3.77 Mcycle (2.51 ms
at 1.5 GHz) representing 0.03%. The overhead expressed in
cycles is basically independent of the running application.
Since most applications last several seconds, this overhead can
be considered negligible.
C. Error-Rate Estimation
The application error rates are estimated by applying (6)
and presented in Table VIII. Derating factors Mf and Etf are
used if applicable.
For assessing the predicted error rate (τSEU), it is necessary
to compare this value with the measured one (σDYN ). From the
predicted and measured values, it is possible to calculate the
failure rate (λ) by extrapolating them to the operating radiation
environment as follows: λ = σ × ϕ, where σ is the cross
section and ϕ is the flux at the operating environment.
The closeness of the prediction can be appreciated
in Figs. 1–3 that illustrate the reliability curves R(t) = e(−λt )
plotted from the 95% confidence intervals of the predicted
and measured cross sections of the devices extrapolated at
avionic altitude (35 000 ft) where the neutron flux is about
2.99×103 n/(cm2 ·h). The considered period of time is 50 000 h
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
RAMOS et al.: SEE ERROR-RATE EVALUATION
1885
Fig. 1. Predicted and measured confidence intervals of the P2041 reliability
extrapolated at avionic altitude.
Fig. 4.
Fig. 2.
Predicted and measured confidence intervals of the Epiphany
reliability extrapolated at avionic altitude.
Fig. 3. Predicted and measured confidence intervals of the MPPA-256 reliability extrapolated at avionic altitude.
which is the estimated average lifetime of a commercial
aircraft.
From Figs. 1–3, it can be seen that an underestimation of the
predicted error rate for P2041 produced by the fact that not all
sensitive zones could be targeted during the static radiation test
and fault-injection campaigns. This fact is explained because
not all memory cells can be accessed by software means. In the
case of Epiphany and MPPA-256, the estimation is within
the range of measurements which means that the approach
provides good results for the MM application. The most
accurate estimation corresponds to the Epiphany multicore
because it does not implement protection mechanisms that
may affect prediction. In addition, a cross-comparison between
the reliability curves shows that the most reliable device is
the P2041. It confirms that the process technology plays a
preponderant role in the device reliability.
Failure rate comparison of the targeted devices.
On the other side, the reliability of multicore/many-core
processors strongly depends on the implemented application
(software). Hence, the failure rate of the studied devices
can be classified within the DO-178B (Software Considerations in Airborne Systems and Equipment Certification).
The DO-178B is a guideline used as de-facto standard for
developing avionic software systems [21].
From the results presented in Section V-C, it is possible
to observe that the error rates of the P2041 multicore and
MPPA-256 many core executing an MM, reach level C of
the DO-178B. Considering that the MM is a memory-bound
program, results suggest that these devices can be well suited
for executing memory-bound applications requiring major
failure conditions (level C). On the other hand, the Epiphany
multicore processor reaches level D, being convenient for
minor failure conditions.
D. Overall Comparison
This section provides an overall comparison of the three
studied processors. Even though the devices have different
manufacturing technologies and architectures, this comparison
can be useful as an example for choosing the appropriate
device depending on the criticality of the application, required
resources, power budget, and device cost. The failure rate of
each device is derived from its cross section and referenced
to the neutron flux in NYC which is 13 n/(cm2 · h) [19].
A comparison of the failure in time (FIT) for both worst
case and dynamic scenarios is presented in Fig. 4. On the one
hand, it can be seen that P2041 multicore is the most reliable
processor taking into account the failure rate per device.
However, the difference with the MPPA-256 is not so large
as one could expect due to the difference in manufacturing
technology (45-nm SOI versus a 28-nm CMOS). On the
other hand, if the FIT/Mb is considered, the reliability of
the MPPA-256 largely surpasses the one of P2041. This huge
difference in the reliability per megabit is mainly due to an
effective implementation of ECC and interleaving in its shared
memories as well as parity in its cache memories, which have
detected and corrected most of the observed errors during the
radiation tests.
In turn, the failure rate of the Epiphany E16G301 is 10 times
greater than the one of the P2041. This is explained by the
fact that Epiphany has no protection mechanisms. On the
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.
1886
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 65, NO. 8, AUGUST 2018
contrary, comparing other features such as the price of devices,
the P2041RDB development board is about 10 times more
expensive than the Parallella board containing the Epiphany
E16G301. Regarding maximum chip power consumption,
the QorIQ P2041 consumes about 6 times the E16G301. It is
thus necessary to carry out a deep cost-benefit analysis for
choosing the appropriate device depending on the application
and the operating environment.
[2] C. Villalpando, D. Rennels, R. Some, and M. Cabanas-Holmen,
“Reliable multicore processors for NASA space missions,” in Proc.
Aerosp. Conf., Mar. 2011, pp. 1–12, doi: 10.1109/AERO.2011.5747447.
[3] S. Mukherjee, Architecture Design for Soft Errors. San Francisco,
CA, USA: Morgan Kaufmann Publishers, Feb. 2008.
[4] G.-H. Asadi, V. S. Mehdi, B. Tahoori, and D. Kaeli, “Balancing
performance and reliability in the memory hierarchy,” in Proc. IEEE
Int. Symp. Perform. Anal. Syst. Softw., Mar. 2005, pp. 269–279.
[5] R. Velazco, S. Rezgui, and R. Ecoffet, “Predicting error rate for
microprocessor-based digital architectures through C.E.U. (Code Emulating Upsets) injection,” IEEE Trans. Nucl. Sci., vol. 47, no. 6,
pp. 2405–2411, Dec. 2000.
[6] V. Vargas et al., “Radiation experiments on a 28 nm single-chip manycore processor and SEU error-rate prediction,” IEEE Trans. Nucl. Sci.,
vol. 64, no. 1, pp. 483–490, Jan. 2017.
[7] P. Ramos. (Apr. 2017). Evaluation of the SEE Sensitivity and
Methodology for Error Rate Prediction of Applications Implemented
in Multi-Core and Many-Core Processors. [Online]. Available: http://
tima.univ-grenoble-alpes.fr/tima/en/mediatheque/PhDthesisresult_id452.
html
[8] M. Cabanas-Holmen et al., “Predicting the single-event error rate of a
radiation hardened by design microprocessor,” IEEE Trans. Nucl. Sci.,
vol. 58, no. 6, pp. 2726–2733, Dec. 2011.
[9] S. S. Mukherjee, C. Weaver, J. Emer, S. K. Reinhardt, and T. Austin,
“A systematic methodology to compute the architectural vulnerability factors for a high-performance microprocessor,” in Proc. 36th
Annu. IEEE/ACM Int. Symp. Microarchitecture (MICRO), Dec. 2003,
pp. 29–40.
[10] S. S. Stolt and E. Normand, “A multicore server SEE cross section
model,” IEEE Trans. Nucl. Sci., vol. 59, no. 6, pp. 2803–2810,
Dec. 2012.
[11] S. Rezgui, R. Velazco, R. Ecoffet, S. Rodriguez, and J. R. Mingo,
“Estimating error rates in processor-based architectures,” IEEE Trans.
Nucl. Sci., vol. 48, no. 5, pp. 1680–1687, Oct. 2001.
[12] R. Velazco, G. Foucard, and P. Peronnard, “Combining results of
accelerated radiation tests and fault injections to predict the error rate
of an application implemented in SRAM-based FPGAs,” IEEE Trans.
Nucl. Sci., vol. 57, no. 6, pp. 3500–3505, Dec. 2010.
[13] M. Kaliorakis, S. Tselonis, A. Chatzidimitriou, N. Foutris, and
D. Gizopoulos, “Differential fault injection on microarchitectural simulators,” in Proc. IEEE Int. Symp. Workload Characterization, Oct. 2015,
pp. 172–182.
[14] F. Rosa, F. Kastensmidt, R. Reis, and L. Ost, “A fast and scalable fault
injection framework to evaluate multi/many-core soft error reliability,”
in Proc. IEEE Int. Symp. Defect Fault Tolerance VLSI Nanotechnol.
Syst. (DFTS), Oct. 2015, pp. 211–214.
[15] D. Oliveira, V. Fratin, P. Navaux, I. Koren, and P. Rech, “CAROL-FI:
An efficient fault-injection tool for vulnerability evaluation of modern
HPC parallel accelerators,” in Proc. Comput. Frontiers Conf., New York,
NY, USA: ACM, 2017, pp. 295–298.
[16] V. Vargas, P. Ramos, R. Velazco, J.-F. Mehaut, and N.-E. Zergainoh,
“Evaluating SEU fault-injection on parallel applications implemented
on multicore processors,” in Proc. 6th Latin Amer. Symp. Circuits
Syst. (LASCAS), Feb. 2015, pp. 181–184.
[17] S. K. S. Hari, T. Tsai, M. Stephenson, S. W. Keckler, and J. Emer,
“SASSIFI: An architecture-level fault injection tool for GPU application
resilience evaluation,” in Proc. IEEE Int. Symp. Perform. Anal. Syst.
Softw. (ISPASS), Apr. 2017, pp. 249–258.
[18] H. Quinn et al., “Using benchmarks for radiation testing of microprocessors and FPGAs,” IEEE Trans. Nucl. Sci., vol. 62, no. 6, pp. 2547–2554,
Dec. 2015.
[19] Measurement and Reporting of Alpha Particle and Terrestrial
Cosmic Ray-Induced Soft Errors in Semiconductor Devices, document
JESD89A, JEDEC Solid State Technology Association, Dec. 2006.
[Online].
Available:
http://www.jedec.org/sites/default/files/docs/
jesd89a.pdf
[20] P. Ramos et al., “Evaluating the SEE sensitivity of a 45 nm SOI multicore processor due to 14 MeV neutrons,” IEEE Trans. Nucl. Sci., vol. 63,
no. 4, pp. 2193–2200, Aug. 2016.
[21] C. Spitzer, U. Ferrell, and T. Ferrell, “RTCA DO-178B/EUROCAE
ED-12B,” in Digital Avionics Handbook, 3rd ed. New York, NY, USA:
Taylor & Francis, Sep. 2014, pp. 195–206.
VI. C ONCLUSION
This paper presents the evaluation of the SEE error rate
of an application implemented in three COTS multicore/
many-core processors which are of interest to avionic industries. The experimental results support the widespread use
of multicore/many-core COTS processors in the embedded system domain. The fact that a many-core processor
built on 28-nm CMOS has a similar FIT than a multicore
45-nm SOI is very promising. Mitigation techniques effectively implemented in the MPPA-256 decrease the reliability
gap between both technologies regarding the manufacturing
process and miniaturization.
The fault injector developed in this paper can be adapted
to any multicore/many-core processor. It does not need any
processor model, and it has a reduced fault-injection time compared with other tools. However, for having a fair comparison,
all fault injection tools should target the same application.
This paper is an evaluation of the proposed error-rate prediction approach. For validating the approach, it is necessary
to increase the experimental data and test other applications such as the benchmark suite for high reliability systems proposed in [18], in order to confirm its effectiveness.
In the future work, the error-rate prediction approach will be
evaluated through heavy-ion experiments for testing applications intended to be used in space environment. Moreover,
the approach will be assessed using other system configuration with OS and programming models such as Posix and
OpenMP.
ACKNOWLEDGMENT
Part of this work was performed in the framework of
the CAPACITES, a LEOC Project. Work on GEnerator of
NEutrons Pulsed and Intense at LPSC, Université Grenoble
Alpes, Grenoble, France, was performed by the “Characterization Program” of the IRT nanoelec, co-funded by the French
Governement “Program d’Investissements d’Avenir” ANR-10AIRT-05. The authors would like to thank F. Villa and S. Rey
from LPSC for their valuable support during the radiation
experiments.
R EFERENCES
[1] Certification Authorities Software Team (CAST). (Nov. 2016).
Position Paper. CAST-32A. Multi-core Processors. [Online]. Available:
https://www.faa.gov/aircraft/air_cert/design_approvals/air_software/
cast/cast_papers/media/cast-32A.pdf
Authorized licensed use limited to: INSTITUTE OF SEMICONDUCTORS CAS. Downloaded on July 31,2023 at 02:56:56 UTC from IEEE Xplore. Restrictions apply.