Received: 3 March 2023
Revised: 7 May 2023
Accepted: 8 May 2023
DOI: 10.1002/dta.3517
RESEARCH ARTICLE
The presence of doping agents in dietary supplements:
A glimpse into the Brazilian situation
Clarisse L. Torres |
Felipe A. G. de Oliveira |
Monica C. Padilha |
Henrique M. G. Pereira
Chemistry Institute, Brazilian Doping Control
Laboratory – LBCD – LADETEC, Federal
University of Rio de Janeiro – UFRJ, Rio de
Janeiro, Brazil
Luis Felipe Jooris
|
Abstract
Dietary supplements (DS) are intended for healthy people to maintain or improve
their overall health. Its consumption is widespread in large part of the general popula-
Correspondence
Clarisse L. Torres, Chemistry Institute, Brazilian
Doping Control Laboratory – LBCD –
LADETEC, Federal University of Rio de Janeiro
– UFRJ, Av. Horácio Macedo, 1281, Polo de
Química, Ilha do Fundão, Rio de Janeiro
21941-598, Brazil.
Email: clarissetorres@iq.ufrj.br
tion and at all levels of athletes. Nevertheless, DS use can also pose health risks to
individuals and, in the case of athletes, may lead to adverse analytical findings (AAFs)
due to the possibility of DS contamination or adulteration with doping agents banned
by the World Anti-Doping Agency. Although educational initiatives are being performed in Brazil to warn the sports community about inadvertent doping cases, AAFs
connected to the DS administration have been increasingly growing. The findings of
Funding information
Brazilian Antidoping System
DS analyzed by the Brazilian Doping Control Laboratory (LBCD), between 2017 and
2022, after Testing Authorities (TAs) analysis requests, showed an alarming number
of tainted samples. Diuretics were the most common adulterants found in all supplement types. However, the profile of prohibited substances in manufactured and compounded dietary supplements (MDS and CDS, respectively) were distinct, with
stimulants being most prevalent in MDS and anabolic agents in CDS products. Additionally, MDS samples generally presented higher estimated concentrations of
banned substances (mg/g) than CDS samples (μg/g). The common practice of DS
intake by athletes continues to be of great concern for a doping-free sport, given the
high prevalence of prohibited substances detected in the analyzed samples by the
LBCD. The current Brazilian scenario reinforces the importance of raising awareness
in the sports community of the possible consequences of an unintentional doping
case linked to DS use.
KEYWORDS
adulteration, adverse analytical finding, contamination, dietary supplements, doping control
1
|
I N T RO DU CT I O N
the general population and at all levels of athletes aiming to optimize
physical performance,2 becoming a billionaire business.3–5 Although
According to the Brazilian Health Regulatory Agency (ANVISA), die-
all nutrients can be found in real food, different arguments have been
tary supplements (DS) are products that are orally taken to provide
used to justify athletes' consumption of DS. These include perfor-
nutrients, substances, enzymes, or probiotics missing in diets. DS are
mance enhancement, recovery from intense physical activity, meeting
not drugs and therefore do not serve to treat, prevent, or cure dis-
daily requirements of essential nutrients, and maintaining or improv-
eases.1 They are intended for healthy people, with potential benefits
ing health.6,7 Nevertheless, the use of such products remains contro-
regarding overall health. Its consumption is widespread in large part of
versial due to the need for more scientific evidence of their actual
Drug Test Anal. 2023;1–11.
wileyonlinelibrary.com/journal/dta
© 2023 John Wiley & Sons Ltd.
1
TORRES ET AL.
benefits.8 Additionally, knowledge gaps about the potential risks of
Management Authority (RMA), WADA, or hearing body authorities.
their use can lead to side effects and even failure in anti-doping tests,
According to the WADA rules, WADA-accredited laboratories, such
9
especially when considering high-performance sports athletes.
Recent studies have shown an increasing number of doping cases
associated with, a priori, the ingestion of contaminated DS with pro-
as LBCD, are not allowed to provide results, documentation, or advice
that, in any way, could be used as an endorsement of products or services. This regulation was strictly followed in this work.
hibited substances.10–12 This phenomenon has become increasingly
recurrent in Brazil, with several cases registered in the national sport
court, an integral part of the Brazilian anti-doping system. Mislabeling,
2.2
DS samples context
|
cross-contamination, or adulteration in DS products are frequently
claimed to justify the presence of doping agents or their metabolites
All DS samples (n = 140) received by LBCD between 2017 and 2022
in athletes' samples. The detection of such substances in athletes'
(up to May) were included in the study. The analytical procedures
samples is considered a violation of the World Anti-Doping Code.13
obeyed the International Standard for Laboratories.20 According to
Furthermore, the resulting adverse analytical findings (AAFs), reported
the TAs, athletes defense enlisted in the AAFs claimed that the results
by World Anti-Doping Agency (WADA)-accredited laboratories, entail
would be directly linked to the consumption of DS. Following internal
8
severe consequences for the athletes.
protocols, the DS recipients were inspected by trained staff, and the
The regulation of DS is another complicated worldwide issue that
seal's condition was recorded as part of the chain of custody. The
is worth mentioning. The regulatory systems vary globally, and there
products were considered “closed” if the original seals were intact
is no consensus regarding which category (nutritional supplement or
and “opened” if they were in use or transferred to another container.
14
The presence of doping agents
In the absence of the original seal, the lack of forensic traceability was
not declared on the labels in manufactured dietary supplements
highlighted in the chain of custody and shared with the relevant
(MDS), readily available online, has been well documented.12,15,16
TA. In such cases, an adulteration of the unsealed product by the ath-
therapeutic good) DS are classified.
In Brazil, issues regarding the potential cross-contamination of
lete or another party cannot be excluded. Information about the
supplements produced by compounding pharmacies have already
substance(s) detected in biological samples was welcomed but not
become a subject of intense debate. The compounding dietary supple-
mandatory.
ments (CDS) bring the perspective of a tailor-made product, which in
theory, allows one to get the maximum benefit from the product with
minor drawbacks. Despite their use being discouraged by the Brazilian
2.3
|
Analytical approach
Olympic Committee17 and other sports stakeholders18 through educational campaigns, such a strategy seems unrestrained. Anecdotal
Most of the DS received were solid contents (capsules, tablets, and
reports, testimonials in court, and media declarations highlight that
powder). A minimum of 1 g was weighed in all cases. The materials
some sports physicians, nutritionists, and nutrologists have motivated
were extracted with 10 mL of methanol. For liquid samples, 1 mL
the consumption of supplementation, with a preference for CDS, to
was transferred and diluted in 9 mL of the same solvent. An aliquot
reach high-performance levels.
of 1 mL of the methanolic solution was used for the analyses. The
In this context, to raise awareness regarding the risks associated
sample extracts were screened by gas chromatography–mass
with DS consumption by athletes, this work aims to present the
spectrometry (GC–MS) and liquid chromatography-high resolution
descriptive data concerning the presence of prohibited substances19
mass spectrometry (LC-HRMS), using similar method conditions
in DS products reported by the Brazilian Doping Control Laboratory
applied to the anti-doping samples routine analysis.21 The MS
(LBCD) after Testing Authorities' (TAs) analysis requests. All analyzed
acquisition data strategy is focused on the parent compounds as
DS were related to AAF cases, and product contaminations were
targets instead of their metabolites. In the case of a presumptive
claimed during the results management or in the sports tribunal to jus-
finding, confirmatory analyses were performed using a different
tify the adverse findings. The statistics and patterns of banned sub-
aliquot. In this confirmation step, the complete identification criteria
stances detected in each group of supplements (CDS or MDS) are
described in the WADA TD IDCR were adopted in this confirma-
presented, and some situations observed in the Brazilian system are
tion step.19,22
discussed.
3
2
2.1
|
RESULTS AND DISCUSSION
EXPERIMENTAL
|
|
Ethics in DS analysis
3.1 | DS market in Brazil: Regulations and its
implications in the sports context
The analysis of DS by WADA-accredited laboratories is grounded by
The MDS category in Brazil was only established in 2018 with the res-
the Code of Ethics in the “Other Analytical Activity” section.20 The
olution RDC 243/201823 to ensure the population's access to safe
analysis can only be performed when requested by the TAs, Results
and quality products. However, little is known about the impact on
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2
the quality of MDS products that reaches the end consumer after the
new Brazilian regulatory framework has been in place. For example,
3.2 | DS analysis in Brazil, forensic traceability, and
the world anti-doping system framework
Abe-Matsumoto et al.24 detected a significant variability in the levels
of vitamins A, E, and C declared on the labels of commercial supple-
On a global level, WADA is responsible for conducting investigations
ments sold in an urban area in Brazil compared with those presented
aiming to elucidate any anti-doping violations. Apart from WADA, the
in the analyzed samples. In another study, Molin et al.25 reported
so-called Anti-Doping Organizations “shall have the capability to con-
numerous inconsistencies regarding the labeling, ingredients used,
duct, and shall conduct, investigations and gather intelligence as
and effects claims in 44 dietary supplements purchased from Brazilian
required by the International Standard for Testing and Investiga-
online stores. Among them, the presence of dehydroepiandrosterone
tions.”13 According to the International Standards for Laboratories,
(DHEA), a controlled medicine in Brazil and a well-known banned sub-
WADA-accredited laboratories shall support the authorities by ensur-
19
ing TA and/or RMA responsiveness. In this context, and in alignment
To add further complexity, the new Brazilian regulatory frame-
with the WADA Code of Ethics, LBCD adopted the policy to assist
work for MDS does not apply to dietary supplements produced in
the TAs in investigations falling under their competencies, including
compounding pharmacies (CDS). From the sanitary point of view, the
possible DS analysis. Besides, under the adopted internal policy, LBCD
specific regulation concerning CDS products remains the RDC
does not make any value judgment about the merits of the DS analy-
67/2007. This regulation focuses on the “good manufacturing prac-
sis request or forensic traceability of the material. When relevant,
tice” system, ensuring that magistral preparations are consistently
LBCD includes the necessary disclaimers in the final report as, for
produced and controlled according to quality standards.26 Neverthe-
instance, if the sample received was sealed or not. From the analytical
less, the risk of cross-contamination of a CDS during the preparation
point of view, all qualitative criteria adopted in identifying doping
process in compounding pharmacies should not be discarded, consid-
agents in a biological matrix are followed, aggregating a high confi-
ering the possible presence of residues after standard cleaning proce-
dence level in the result. Nevertheless, all TAs in collaboration with
dures.27 In 2018, for instance, the United States Anti-Doping Agency
LBCD are aware that the methods used for DS analyses are not under
(USADA) concluded that Brazilian compounding pharmacies were the
the ISO17025 scope of accreditation.
stance by WADA, was also reported.
source behind the AAFs of three athletes from the Ultimate Fighting
Since 2017, the WADA-accredited laboratory LBCD, located in
Championship (UFC). The substances detected were hydrochlorothia-
Rio de Janeiro, has received requests from at least six different TAs to
zide (HCTZ) and anastrozole, both prohibited at all times under the
perform testing analyses of supposedly contaminated DS. Figure 1a
28
UFC Anti-Doping Policy
and WADA.
shows the number of DS samples delivered to analysis per year from
The bodies representing pharmacies, such as the National Associ-
2017 up to May 2022 at the LBCD facilities. An increasing tendency
ation of Master Pharmacists (Anfarmag), systematically deny potential
regarding the number of supplement analyses can be seen, except for
contamination of a CDS due to poor quality control procedures. As a
the period between 2020–2021, clearly reduced due to the global
rebuttal, they have suggested intentional adulteration of the product
pandemic crisis. From a total of 140 analyzed DS samples, LBCD
to improve athletes' performance through prohibited substances.29,30
reported 93 of them (66.43%) as “negative,” meaning “no prohibited
This debate has led to massive repercussions in the Brazilian Anti-
substances were detected.” The other 47 samples (33.57%) led to
Doping System, transcending the anti-doping analysis and the
positive identification, meaning “one or more prohibited substances
WADA-accredited laboratory's responsibility. Each argument's merits
were detected” (Figure 1b). The presented data corroborate the cur-
are beyond this paper's goals and represent one of the most signifi-
rent picture of the increasingly growing consumption of DS by the
cant challenges in sports tribunals.
sports community and its presumable association with doping
Data reported to the LBCD, considering the collaboration of six
cases.32,33
TAs with the laboratory in the last years, reveal how complex it is
The production sources of received DS samples (manufactured or
to deal with court cases when the athlete declares using DS. For
compounded) were unclear to the LBCD. Nevertheless, using all infor-
example, the number of AAFs related to ostarine, a selective andro-
mation available, the samples were classified into two groups accord-
gen receptor modulator (SARM), has increased in LBCD's reports
ing to their supposed origin of production: CDS, for DS prepared in
from 2017 to 2022. As expected, the argument of consuming
compounding pharmacies, and MDS, for DS obtained from conven-
tainted DS with ostarine has frequently been alleged as the cause
tional markets. Figure 1c,d shows that the cases involving MDS and
of AAFs in court. Results from CDS samples analyzed by LBCD
CDS were balanced: MDS were 71 out of 140 and CDS 69 out of
have demonstrated the presence of the ostarine doping agent,
140. However, the percentage of confirmed cases detected in CDS
often in extremely low concentrations (see Section 3.4 for more
samples was higher (40.58%) than in MDS samples (26.76%). These
information). The pharmaceutical community rebutted the contami-
results were unexpected, considering the quality standards described
nation thesis, considering its prohibited use in Brazil by ANVISA.
in the RDC 67/2007.
Afterward,
pharmacies
it
was
had
contrary position.31
demonstrated
access
to
that
ostarine,
Brazilian
despite
compounding
the
ANVISA's
Aiming to highlight the differences observed in prohibited substance profiles between CDS and MDS production sources, the data
from each group were separately evaluated.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
3
TORRES ET AL.
TORRES ET AL.
FIGURE 1
and (d) CDS.
(a) DS samples analyzed per year (2017 up to May 2022); percentage of AAFs and negatives detected in (b) general DS; (c) MDS;
3.3 | Profile of prohibited substances detected in
MDS samples
impressive. Therefore, possible adverse health effects on users' health
Buying MDS in Brazil is very simple and convenient. There is no need
to capsules for weight loss (slimming and fat-burning supplements).
to have a physician or nutritionist prescription. Thus, anyone can pur-
The literature has already advised that this type of product is consid-
chase MDS without restrictions. Furthermore, purchasing MDS has
ered high-risk for adulteration due to the promise of getting fast
become even easier with available online products. The lack of stricter
results.35 The substances detected were stimulants (sibutramine, class
regulation dedicated to MDS and the simplicity of commercializing
S6) and diuretics and masking agents, with great emphasis on HCTZ,
them have also promoted the expansion of products with doubtful
chlorothiazide (CTZ), and furosemide (class S5). Four of these 10 posi-
quality on the market.25
tive MDS products (#3, 4, 11, and 14) were obtained as originally
should be carefully considered.
Out of 19 samples, 10 (52.63%) had positive test results related
Our study revealed that 19 of 71 suspicious MDS samples contained banned substances by WADA. Detailed information is shown
sealed bottles, which indicated the content of undeclared substances
on the labels.
in Table 1. The concentration of doping agents in the material could
Six of the 19 (31.58%) contaminated MDS (#5–8, 12, and 15)
only be estimated as the methods employed for DS analysis were not
were considered ergogenic supplements intended to enhance
developed for a quantitative approach. The estimated concentration
sports performance.36 The doping classes reported were S1
confidence levels could be the subject of debate, considering the
(oxandrolone, ostarine, and testosterone), S6 (phenylethylamine
diversity of the matrices, the lack of certificated reference materials
derivatives and sibutramine), and S5 (furosemide). Doping agents
for method development, or interlaboratory proficiency tests to track
were estimated at high concentrations in samples #6 and #15
the laboratories' analytical capabilities. Such analytical aspects are fre-
(in the order of mg/g). Such high concentrations are frequently
quently explained to the TAs or in court. Despite its relevance, the
understood in court as an intentional adulteration process, although
analytical limitations associated with DS analysis in an anti-doping
no scientific evidence is available to support this hypothesis
context are beyond this paper's scope.
unequivocally.
The distribution of doping agents detected in MDS samples is
The remaining products (#9, 10, and 13) accounted for 15.79% of
illustrated in Figure 2. About 60% of the substances accounted for
the contaminated MDS samples. These included herbs and postbiotic
class S5 (diuretics and masking agents), followed by 30.77% for class
supplements. Product #10 was delivered as an originally sealed bottle,
S6 (stimulants), with minor contributions (7.69%) for class S1 (anabolic
suggesting the presence of mislabeled prohibited substances. Product
agents, S1.1 + S1.2) and 2.56% for class S9 (glucocorticoids). The high
#13 showed high concentrations of furosemide, indicating that an
prevalence of stimulants among Brazilian athletes has already been
adulteration process to increase its effectiveness could not be
studied.34 The number of cases involving diuretics in this study was
discarded.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
4
TABLE 1
MDS contaminated or adulterated with banned substances (2017 to May 2022).
Case/
year
Product
#
Product type
Presentation
Originally
sealed?
5440/22
1
Slimming product
Capsules
5421/21a
2
Slimming product
Capsules
3
5
Detected substance(s)
No
HCTZ, CTZ,
cloraminophenamideb
HCTZ 21.9 mg/g, CTZ 0.81 μg/mg,
furosemide 19.5 mg/g, sibutramine
49.1 mg/g
No
Sibutramine, HCTZ
Sibutramine 37.9 μg/mg, HCTZ
26.9 μg/mg, CTZ 16.3 μg/mg
Sibutramine 17.1 μg/mg
Yes
4
5426/21
AAF (urine)
Sibutramine 14.7 μg/mg
Yes
Pre-workout
Powder
No
Mephentermine
Phenylethylamine derivatives
5430/21
6
Pre-workout
Powder
No
Oxandrolone
Oxandrolone 2.81 mg/g
5434/21
7
Beta-alanine
Powder
No
Ostarine
Ostarine 142.55 μg/mg
5436/21
8
Creatine
Powder
No
Ostarine
Furosemide 10.2 μg/g
9
Epicor
Capsules
No
5438/21
10
Harpagophytum
Procumbens
Capsules
Yes
HCTZ, CTZ,
cloraminophenamideb
HCTZ 17.3 mg/g, CTZ 1.63 μg/mg,
cloraminophenamide 23.4 μg/mg,
Dexamethasone 4.0 mg/g,
5417/20
11
Slimming product
Capsules
Yes
HCTZ, CTZ, furosemide
HCTZ
5408/19
12
Beta-alanine
Powder
Yes
Sibutramine
Sibutramine 469 ng/g
5413/19
13
Maca supplement
Tablets
No
Furosemide
Furosemide 1.59 mg/g
5392/18
14
Slimming product
Capsules
Yes
HCTZ, CTZ,
sibutramine
HCTZ 15.94 mg/g, CTZ 1.30 μg/mg,
furosemide 18.2 μg/g, sibutramine
5.54 mg/g,
5394/18
15
Creatine
Powder
No
IRMS - testosterone
Testosterone 2.3 mg/g
5397/18
16
Slimming product
Capsules
No
Sibutramine
Furosemide = 214 mg/g;
sibutramine = 27.5 mg/g
5401/18
17
Fat burning
Capsules
No
Furosemide,
sibutramine
Furosemide, sibutramine
5402/18a
18
Slimming product
Capsules
No
HCTZ 7.56 mg/g, furosemide 4.84 μg/
mg, sibutramine 21.13 mg/g, CTZ,
cloraminophenamide
No
HCTZ 6.10 mg/g, furosemide 4.20 μg/
mg, sibutramine 15.60 mg/g, CTZ,
cloraminophenamide
19
Phenylethylamine 1.11 μg/g
Abbreviations: CTZ, chlorothiazide; HCTZ, hydrochlorothiazide.
a
Same product, different bottles.
b
Out-of-competition testing sample.
FIGURE 2
Distribution of identified doping drugs in MDS samples. CTZ, chlorothiazide; HCTZ, hydrochlorothiazide.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
5
TORRES ET AL.
TORRES ET AL.
3.4 | Profile of prohibited substances detected in
CDS samples
prescription, reinforcing the sense of security. Considering the
increasing number of doping cases potentially associated with DS
consumption, athletes should have been advised by the healthcare
Compounding pharmacies are very traditional in Brazil. These estab-
professional that CDS intake, besides its potential benefits, could also
lishments can manipulate medications and nutritional supplements in
pose a risk to their health and career.27
their facilities.26 Unlike MDS, which can be bought without a recipe,
The results of contaminated CDS samples linked to doping find-
CDS can only be purchased if a qualified professional has emitted a
ings are shown in Table 2. The samples encompassed a variety of
TABLE 2
CDS contaminated or adulterated with banned substances (2017 to May 2022).
Case/
year
Product
#
Product type
Presentation
Originally
sealed?
AAF (urine)
Detected substance (CDS)
5439/22a
20
Probiotics
Capsules
No
Oxandrolone
Clomifen = 1.1 μg/g
21
NADH
Capsules
No
Oxandrolone = 190 μg/g
22
Vitamins and minerals
Capsules
No
HCTZ = 0.45 μg/g; furosemide 0.16 μg/g
23
Immune system
supplement
Capsules
No
Spironolactone = 52.1 μg/g
24
NaHCO3
Capsules
No
Clomifen = 0.02 μg/g; furosemide = 0.09 μg/
g
25
Cognitive booster
Capsules
No
26
Vitamins and minerals
Capsules
27
Slimming product
Capsules
5427/21
28
Slimming product
Capsules
No
5428/21
29
Mass gainer
supplement
Capsules
No
Spironolactone = 1.8 μg/g;
Canrenone = 0.2 μg/g; Indapamide 0.4 μg/
g; Ostarine (trace levels)
5432/21a
30
Appetite modulator
supplement
Capsules
No
HCTZ = 86 μg/g
5441/22
5442/22
a
Ostarine
Ostarine = 0.027 μg/g
Stanozolol
Stanozolol = 3.77 ng/g
Stanozolol = 4.43 ng/mg
Ostarine
Ostarine (trace levels)
31
Slimming product
Capsules
No
HCTZ = 13 μg/g; Spironolactone 13 μg/g
5419/20
32
Mass gainer
supplement
Capsules
No
Ostarine = 0.36 μg/g, LGD-4033 (trace levels)
5421/20
33
Compounded
supplementb
Capsules
No
5422/20
34
Amino acids
Capsules
No
Ostarine
Ostarine = 2.30 μg/g;
spironolactone = 0.36 μg/g;
canrenone = 0.20 μg/g
Ostarine = 5.2 ng/g; anastrozole (trace levels)
35
Vitamins
Capsules
No
5423/20
36
Caffeine
Capsules
No
Anastrozole
Anastrozole = 3.5 μg/g
5406/19a
37
Vitamins and enzymes
Capsules
No
Ostarine
Ostarine = 860 ng/g
38
Vitamins and enzymes
Capsules
Yes
Ostarine = 760 ng/g
39
Ostarine 25 mg
40
Vitamins
Capsules
No
HCTZ
41
Antioxidant
supplement
Capsules
No
LGD-4033
5412/19
42
Antioxidant
supplement
Capsules
No
LGD-4033
5414/19
43
Energy booster
Capsules
No
Ostarine
5393/18
44
Compounded
supplementb
Capsules
No
5410/19a
5389/17
5390/17
Ostarine (trace levels)
Ostarine = 113 mg/g
Oxandrolone
Oxandrolone = 578 μg/g
45
Slimming supplement
Capsules
No
HCTZ; cloraminophenamide; fenproporex
46
Fiber mix
Capsules
No
4-Androstenedione
47
Pre-workout
Sachet
Yes
Abbreviation: LGD-4033, ligandrol.
a
Supplements prepared and sold by the same compounding pharmacy.
b
Compounded formulas containing dozens of ingredients.
Furosemide
Furosemide
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
6
product types, such as vitamins, probiotics, slimming products, and
the findings, respectively. Class S4 showed two occurrences for clomi-
mass-gaining supplements, among others. Observing the disclaimers
phene, a selective estrogen receptor modulator (SERM), in products
about the limitations in quantitative approaches of the methods
#20 and #24 and two for anastrozole, an aromatase inhibitor, in prod-
employed, when comparing CDS to correlated findings in MDS sam-
ucts #34 and #36. Fenproporex, a stimulant drug used to treat obe-
ples, there was a clear difference between the range of doping agents'
sity, was detected only once in product #45, a CDS intended for
estimated concentrations. In CDS samples, the prohibited substances
weight loss.
had lower estimated concentrations (μg/g) than in MDS samples
(mg/g). Therefore, they were described as “low” and “high” concentrations, respectively. According to Geyer et al,15 very low and varying
concentrations of doping agents could be inferred as cross-contami-
3.5 | DS consumption and evolution of the
regulation practices in the Brazilian anti-doping system
nations, whereas high concentrations (above 1 mg/g) were probably
due to intentional admixture. However, as no corresponding CDS
samples (originally sealed and from the same batch) were provided,
3.5.1 | The case of “100% natural products” to
induce weight loss
the lack of forensic traceability made the actual source of contamination questionable. In addition, if intentional contamination was an
The demand for natural products that support weight loss is con-
acceptable hypothesis, there was no reason to doubt the capability of
stantly growing not only due to health benefits but also because of
an interested party in a doping case to add very small amounts of dop-
the widespread culture of having a slim body.39 However, behind the
ing agents in a clean DS to strengthen their points of argumentation.
promises of having a fast and easy slimming process, there is usually
The distribution of doping substances in CDS samples (Figure 3)
was also distinct from that observed for MDS samples. The most fre-
an admixture of unsafe synthetic drugs hidden to increase the product's efficacy.40
quently reported substances were anabolic agents (oxandrolone, sta-
This study showed that over half of the analyzed MDS samples
nozolol, 4-androstenedione, ligandrol, and ostarine), accounting for
were products intended for losing weight (Table 1). All of them were
47.81% of the total findings. Among them, SARMs, especially ostarine,
freely marketed on the internet. They were advertised as capsules
were the most prevalent drugs. In 2017, USADA warned that athletes
containing 100% natural ingredients to induce weight loss. Also,
were at risk of unintentional anti-doping rule violations with different
catchphrases such as “it burns the accumulated fat in your body,”
SARMs.11 Since 2021, ANVISA has banned the production, commer-
“reduces your appetite by 80%,” and “has a diuretic action, promoting
cialization, handling, importing, advertising, and use of SARMs, as their
the elimination of toxins” were often claimed on the websites.
safety and effectiveness have yet to be established.26,37
The most commonly detected drugs were sibutramine and differ-
Class S5 accounted for 39.47% of the findings in CDS samples.
ent diuretics. Sibutramine is an appetite suppressant usually found in
The detected substances were diverse: HCTZ, furosemide, spironolac-
adulterated DS slimming products. Diuretics, drugs developed for the
tone, canrenone, and indapamide. In the MDS group, the same class
treatment of hypertension, are also often detected.8 They are illegally
represented 58.97% of the total findings, although only HCTZ, CTZ,
added to DS to increase the production and excretion of urine, induc-
and furosemide have been reported. Our results are in accordance
ing rapid weight loss.38 Unfortunately, the intake of these “natural
with the work published by Lee et al,38 which describes diuretics as
slimming capsules,” besides the positive doping tests, may pose a
one of the most common adulterants found in contaminated
potential health hazard for all consumers.41
DS. These substances are used to mask other doping agents or to
induce weight loss, being prohibited at all times by WADA.19
“Herbal slimming blends” faked with pharmaceuticals are well
known and have been widely reported.42–45 Unfortunately, this prac-
Minor contributions were due to the presence of compounds
tice is still common in Brazil, as demonstrated by this work. As a
belonging to classes S4 and S6, accounting for 10.53% and 2.63% of
result, consumers blindly take these products without the knowledge
FIGURE 3
Distribution of identified doping drugs in CDS samples. HCTZ, hydrochlorothiazide; LGD-4033, ligandrol.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
7
TORRES ET AL.
TORRES ET AL.
of potential risks. Therefore, raising awareness of these findings is
important question must be pursued: is it worth taking multiple CDS
important for the sports community and the general population.
products assuming the risk of potential contamination? Previous studies have shown that the intake of nutritional supplements crosscontaminated with small amounts of prohibited substances could
3.5.2 | Use of multiple CDS products: Risks and
education initiatives
result in inadvertent positive doping cases.15,46,47 Therefore, with the
support of health practitioners, athletes must carefully evaluate the
need for supplementation, taking into consideration the burden of a
Among the cases shown in Table 2, two drawn special attention:
possible anti-doping rule violation in an athlete's career.
5410/19 and 5439/22. In both cases, the athletes reported the con-
Unfortunately, inadvertent doping caused by adulterated supple-
comitant use of several CDS products handled in a compounding
ment consumption is frequently used as an excuse by athletes to
pharmacy (data not shown). In case 5410/19, the athlete declared
explain AAFs worldwide.48 Currently, the argument of a lack of anti-
using 17 different products simultaneously. Results showed the pres-
doping education programs in Brazil is hardly accepted to justify unin-
ence of HCTZ and ligandrol (LGD-4033, a SARM) in two of these
tentional doping related to DS use. Several educational initiatives
products. Both substances are banned at all times of competition by
have been performed in the last 5 years to improve awareness of DS
WADA. In case 5439/22, the athlete declared being taking 12 differ-
use by athletes, led by the National Anti-Doping Authority, the
ent CDS prepared in a compounding pharmacy at the time of his/her
Brazilian Olympic Committee, and the National Sports Federations.
notification. Out of these 12 (Figure 4a), 5 (41.66%) were contaminated with prohibited substances (Figure 4b). The testing analyses
confirmed the presence of clomiphene, HCTZ, and oxandrolone, estimated to be at concentrations in the order of μg/g.
3.5.3 | Change of paradigm for DS analysis in Brazil
in sports context
These numbers revealed an alarming percentage of contamination
in capsules of compounded supplements. As already stated, the con-
Until very recently, the analysis of an opened DS and its corre-
tamination source is beyond the scope of this work. Nevertheless, an
sponding original sealed container (same batch and expiration date)
F I G U R E 4 (a) A total of 12 supplements prepared in a compounding pharmacy declared being in use by an athlete; (b) products contaminated
with banned substances at concentrations in the order of μg/g.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
8
had been treated as an educative initiative by the Brazilian National
AC KNOW LEDG EME NT S
Anti-Doping Organization (ABCD). However, to add a more strict
The authors thank the stakeholder of the Brazilian Antidoping System
regulation, considering the evident lack of traceability in DS
for the support.
unsealed bottles, the Brazilian National Anti-Doping Organization
(ABCD) edited Resolution No. 2 of August 6, 2020,49 declaring that
RE FE RE NCE S
the provision of only opened/in-use supplements would no longer
1. Agência Nacional de Vigilância Sanitária - Anvisa. Perguntas e Respostas - Alimentos e Preparações em Farmácias de Manipulação.
Accessed June 11, 2022. https://www.gov.br/anvisa/pt-br/
centraisdeconteudo/publicacoes/alimentos/perguntas-e-respostasarquivos/alimentos-e-preparacoes-em-farmacias-de-manipulacao.
pdf/view
2. Maughan RJ, Burke LM, Dvorak J, et al. IOC consensus statement:
dietary supplements and the high-performance athlete. Br J Sports
Med. 2018;52(7):439-455. doi:10.1136/bjsports-2018-099027
3. Future Market Insights. Dietary Supplements Market Outlook
(2022-2032). 2022. Accessed January 18, 2023. https://www.
futuremarketinsights.com/reports/dietary-supplements-market
4. Precedence Research. Nutritional Supplements Market Size, 2020 to
2030. 2023. Accessed January 18, 2023. https://www.
precedenceresearch.com/nutritional-supplements-market
5. Or F, Kim Y, Simms J, Austin SB. Taking stock of dietary Supplements'
harmful effects on children, adolescents, and young adults.
J Adolesc Health. 2019;65(4):455-461. doi:10.1016/j.jadohealth.2019.
03.005
6. Rawson ES, Miles MP, Larson-Meyer DE. Dietary supplements for
health, adaptation, and recovery in athletes. Int J Sport Nutr Exerc
Metab. 2018;28(2):188-199. doi:10.1123/ijsnem.2017-0340
7. Garthe I, Ramsbottom R. Elite athletes, a rationale for the use of dietary supplements: a practical approach. PharmaNutrition. 2020;14:
100234. doi:10.1016/j.phanu.2020.100234
8. Walpurgis K, Thomas A, Geyer H, Mareck U, Thevis M. Dietary supplement and food contaminations and their implications for doping
controls. Foods. 2020;9(8):1012. doi:10.3390/foods9081012
9. Knapik JJ, Steelman RA, Hoedebecke SS, Austin KG, Farina EK,
Lieberman HR. Prevalence of dietary supplement use by athletes: systematic review and meta-analysis. Sports Med. 2016;46(1):103-123.
doi:10.1007/s40279-015-0387-7
10. Kozhuharov VR, Ivanov K, Ivanova S. Dietary supplements as source
of unintentional doping. Biomed Res Int. 2022;2022:8387271. doi:10.
1155/2022/8387271
11. Mareck U, Geyer H, Schertel T, Petring S, Krug O, Thevis M. Detection of undeclared doping substances in nutritional supplements in
the context of follow-up investigations concerning adverse analytical
findings. Drug Test Anal. 2021;13(11-12):1911-1914. doi:10.1002/
dta.3158
12. Mathews NM. Prohibited contaminants in dietary supplements.
Sports Health. 2018;10(1):19-30. doi:10.1177/1941738117727736
13. World Anti-Doping Agency (WADA). World Anti-Doping Code.
Accessed November 6, 2022. https://www.wada-ama.org/sites/
default/files/resources/files/2021_wada_code.pdf
14. Dwyer JT, Coates PM, Smith MJ. Dietary supplements: regulatory
challenges and research resources. Nutrients. 2018;10(1):41. doi:10.
3390/nu10010041
15. Geyer H, Parr MK, Koehler K, Mareck U, Schänzer W, Thevis M.
Nutritional supplements cross-contaminated and faked with doping
substances. J Mass Spectrom. 2008;43(7):892-902. doi:10.1002/jms.
1452
16. Kohler M, Thomas A, Geyer H, Petrou M, Schänzer W, Thevis M.
Confiscated black market products and nutritional supplements with
non-approved ingredients analyzed in the cologne doping control laboratory 2009. Drug Test Anal. 2010;2(11-12):533-537. doi:10.1002/
dta.186
be accepted. The new regulation in place aimed to discard possible
manipulations with the target doping substance(s).50 Therefore,
after August 25, 2022, if an athlete wishes to have a DS analyzed,
it became mandatory to deliver the corresponding sealed DS, with
the same batch number and accompanied by the purchase invoice.
It will be interesting to see the impact of this new protocol on the
DS statistics in the near future and the long-term reverberation in
court. Nevertheless, it is important to mention that no DS
was delivered for analyses in the first 6 months after adopting the
new rules.
4
|
C O N CL U S I O N S
The use of DS by the sports community is a worldwide phenomenon and well documented in Brazil. This work described the profiles
of prohibited substances detected in DS products by the Brazilian
Anti-Doping Laboratory from 2017 to 2022. Samples provided by
six different TAs (n = 140) were analyzed. Different patterns of
doping agents were reported in dietary supplements classified as
MDS or CDS. Contaminated MDS samples generally presented
higher estimated concentrations of banned substances (mg/g),
whereas CDS samples revealed lower estimated concentrations of
prohibited substances (μg/g). The major doping classes found in the
MDS category were diuretics / masking agents and stimulants,
mainly due to the slimming product profiles. On the other hand, the
predominant classes in CDS were anabolic agents and diuretics/
masking agents.
Although Brazil's new DS regulatory framework has improved
some aspects related to the quality of supplements, such as allowed
ingredients and label description rules, their safety has not been fully
guaranteed yet, as revealed by this work. We have shown that the
use of DS by athletes continues to be of great concern for a dopingfree sport, given the high prevalence of prohibited substances
detected in DS samples.
Despite the concept of “Strict Liability” being evident in the
Code, the athletes' defense usually makes statements in court claiming DS contamination without forensics traceability support or scientific evidence. Athletes shall be aware that regardless of whether an
AAF reported in a biological sample was derived from an unintentional
or intentional act, the TAs/RMAs can conduct a thorough investigation. The current outlook regarding the detection of doping agents in
supplement samples reinforces the importance of raising awareness
for the sports community of the possible consequences of an inadvertent doping case linked to DS use.
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
9
TORRES ET AL.
17. Comitê Olímpico do Brasil. COB fará campanha educativa sobre uso
de suplementos no esporte. Accessed January 18, 2023. https://
www.cob.org.br/pt/galerias/noticias/cob-fara-campanha-educativasobre-uso-de-suplementos-no-esporte
18. Autoridade Brasileira de Controle de Dopagem. Plano Nacional de
Educação 2021 – ABCD. Accessed January 18, 2023. https://www.
gov.br/abcd/pt-br/acesso-a-informacao/noticias/abcd-lanca-novoplano-de-educacao-no-dia-nacional-do-jogo-limpo/plano_nacional_
educacao_abcd_2021.pdf
19. World Anti-Doping Agency. The 2022 prohibited list. International
Standard. 2022. Accessed February 8, 2022. https://www.wada-ama.
org/en/news/wada-publishes-2022-prohibited-list
20. World Anti-Doping Agency - WADA. International Standard Laboratories (ISL) 2021. International Standard Laboratories (ISL) 2021.
Accessed October 22, 2022. https://www.wada-ama.org/en/
resources/world-anti-doping-program/international-standardlaboratories-isl
21. Pereira HMG, Sardela VF, Padilha MC, et al. Doping control analysis
at the Rio 2016 Olympic and Paralympic games. Drug Test Anal. 2017;
9(11–12):1658-1672. doi:10.1002/dta.2329
22. World Anti-Doping Agency - WADA. WADA Technical Document –
TD2021IDCR. Accessed October 22, 2022. https://www.wada-ama.
org/en/resources/lab-documents/td2021idcr
23. Da Costa BRB, De La Cruz MNDS, Roiffe RR, Padilha MC. Food supplements: an unintended doping source? Brazilian J Food Technol.
2021;24:1-16. doi:10.1590/1981-6723.25619
24. Abe-Matsumoto LT, Sampaio GR, Bastos DHM. Do the labels of vitamin a, C, and E supplements reflect actual vitamin content in commercial supplements? J Food Compos Anal. 2018;72:141-149. doi:10.
1016/j.jfca.2018.07.001
25. Molin TRD, Leal GC, Müller LS, et al. Regulatory framework for dietary supplements and the public health challenge. Rev Saude Publica.
2019;53:90. doi:10.11606/s1518-8787.2019053001263
26. Agência Nacional de Vigilância Sanitária – ANVISA. Resolução - RDC
no 67, de 8 de outubro de 2007. Dispõe sobre boas práticas de manipulação de preparações magistrais e oficinais para uso humano em
farmácias 2007. Accessed October 22, 2022. https://bvsms.saude.
gov.br/bvs/saudelegis/anvisa/2007/rdc0067_08_10_2007.html
27. Prioste T, Ferreira T, Fagundes V, Montanha MC. Recovery of drug
residues in equipment and utensils used by compounding pharmacies
after standard cleaning procedure. Braz J Pharm Sci. 2015;51(2):317322. doi:10.1590/S1984-82502015000200008
28. United States Anti-Doping Agency - USADA. Investigation Identifies
Compounding Pharmacies as Source of Tainted Supplements Behind
Three Positive Tests Under the UFC Anti-Doping Policy. Sanctions,
Announcement. Accessed February 8, 2022. https://ufc.usada.org/
compounding-pharmacies-supply-tainted-supplements-behind-threepositive-tests/
29. Anfarmag. Posicionamento Anfarmag sobre casos de doping.
Accessed January 18, 2023. http://www.anfarmag.com.br/lercomunicado/posicionamento-anfarmag-sobre-casos-de-doping/
possível tantos casos de doping por contaminação
30. Yahoo Sports. E
cruzada? Número de casos no esporte brasileiro preocupa. 2019.
Accessed January 18, 2023. https://esportes.yahoo.com/noticias/epossivel-tantos-casos-de-doping-por-contaminacao-cruzada-numerode-casos-no-esporte-brasileiro-preocupa-233600261.html?
guccounter=1
31. SARMS – Justiça de Curitiba – PR concede liminar em 10/05/2021 e
autoriza a manipulação. Accessed 6 November 6, 2022. https://besan.
com.br/sarms-justica-de-curitiba-pr-concede-liminar-em-10-05-2021e-autoriza-manipulacao/#::text=Porfim%2Comagistradoconcedeu,%
2Ctestoloneerad-140.&text=Curitiba%2FPR%2C10%2F05%2F2021
TORRES ET AL.
32. Savino G, Valenti L, D'Alisera R, et al. Dietary supplements, drugs and
doping in the sport society. Ann di Ig Med Prev e di Comunita. 2019;
31(6):548-555.
33. Lauritzen F. Dietary supplements as a major cause of anti-doping rule
violations. Front Sport Act Living. 2022;4:868228. doi:10.3389/fspor.
2022.868228
34. Pereira HMG, Sardela VF. Stimulant doping agents used in Brazil:
prevalence, detectability, analytical implications, and challenges. Subst
Use Misuse. 2014;49(9):1098-1114. doi:10.3109/10826084.2014.
907653
35. Garthe I, Maughan RJ. Athletes and supplements: prevalence and perspectives. Int J Sport Nutr Exerc Metab. 2018;28(2):126-138. doi:10.
1123/ijsnem.2017-0429
36. Peeling P, Binnie MJ, Goods PSR, Sim M, Burke LM. Evidence-based
supplements for the enhancement of athletic performance. Int J Sport
Nutr Exerc Metab. 2018;28(2):178-187. doi:10.1123/ijsnem.20170343
37. Agência Nacional de Vigilância Sanitária – ANVISA. Resolução - RE
No 791 - de 22 de janeiro de 2021. Accessed October 9, 2022.
https://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?data=23/
02/2021&jornal=515&pagina=48&totalArquivos=63
38. Lee JH, Yang YJ, Min AY, et al. Screening and elucidation of fragmentations of 23 diuretics in dietary supplements using UHPLC-Q-Orbitrap. Sci Justice. 2021;61(5):451-458. doi:10.1016/j.scijus.2021.
05.004
39. Skalicka-Woźniak K, Georgiev MI, Orhan IE. Adulteration of herbal
sexual enhancers and slimmers: the wish for better sexual well-being
and perfect body can be risky. Food Chem Toxicol. 2017;108(Pt B):
355-364. doi:10.1016/j.fct.2016.06.018
th B, Roza O, Csupor D. A systematic review of the
40. Koncz D, To
European rapid alert system for food and feed: tendencies in illegal
food supplements for weight loss. Front Pharmacol. 2021;11:611361.
doi:10.3389/fphar.2020.611361
41. Van der Bijl P. Dietary supplements containing prohibited substances:
a review (part 1). South African J Sport Med. 2014;26(2):59-61. doi:10.
7196/SAJSM.552
42. Wu N, Balayssac S, Danoun S, Malet-Martino M, Gilard V.
Chemometric analysis of low-field 1H NMR spectra for unveiling
adulteration of slimming dietary supplements by pharmaceutical compounds. Molecules. 2020;25(5):1193. doi:10.3390/molecules25
051193
43. Petkova-Gueorguieva E, Ivanov K, Gueorguiev S, Mihaylova A,
Madzharov V, Ivanova S. Detection of sibutramine in herbal food supplements by UHPLC/HRMS and UHPLC/MS-MS. Biomed Res. 2018;
29(14):3006-3009.
44. Geyer H, Gotzmann A. Sibutramine Found in Chinese Herbal Slimming Tea and Capsules. In: Schänzer W, Geyer H, Gotzmann A,
Mareck U, eds. Recent advances in doping analysis (15). Sport und
Buch Strauß; 2007:367-370.
45. Falcão TM. Aplicação de metodologia analítica para a determinação
de adultarentes sintéticos em formulações fitoterápicas emagrecedoras no Brasil. 2011. https://repositorio.ufsm.br/handle/1/5942
46. van der Merwe PJ, Grobbelaar E. Unintentional doping through the
use of contaminated nutritional supplements. South African Med J.
2005;95(7):510-511.
47. Geyer H, Mareck U, Parr MK, Thevis M. Cross-contaminations of
vitamin- and mineral-tablets with metandienone and stanozolol. In:
Schänzer W, Geyer H, Gotzmann A, Mareck U, eds. In recent advances
in doping analysis (14). Sport und Buch Strauß; 2006:11-16.
48. Chan DKC, Tang TCW, Yung PSH, Gucciardi DF, Hagger MS. Is unintentional doping real, or just an excuse? Br J Sports Med. 2019;53(15):
978-979. doi:10.1136/bjsports-2017-097614
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
10
49. Ministério da Cidadania/Secretaria Especial do Esporte/Autoridade
Brasileira de Controle de Dopagem. Resolução No 2, de 6 de agosto
de 2020. Accessed December 10, 2022. https://www.in.gov.br/en/
web/dou/-/resolucao-n-2-de-6-de-agosto-de-2020-271461805
50. Ministério da Cidadania/Secretaria Especial do Esporte/Autoridade
Brasileira de Controle de Dopagem/Coordenação-Geral de Gestão de
Resultados. Orientação sobre a Resolução ABCD no 2, de 6 de agosto
de 2020. Accessed December 10, 2022. https://www.cbclubes.org.
br/sites/default/files/inline-files/SEI_MC%20-%2012881176%20-%
20Of%C3%ADcio%20Circular.pdf
How to cite this article: Torres CL, de Oliveira FAG, Jooris LF,
Padilha MC, Pereira HMG. The presence of doping agents in
dietary supplements: A glimpse into the Brazilian situation.
Drug Test Anal. 2023;1‐11. doi:10.1002/dta.3517
19427611, 0, Downloaded from https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/10.1002/dta.3517 by Manchester Metropolitan University, Wiley Online Library on [23/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
11
TORRES ET AL.