Response Correctness Evaluation
Voice and Text
Version: 2.0
Prepared by: e2f
Date: June 24, 2025
Approved By: e2f
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Version History
Version
Date
Description of changes
Initials
v2.0
May 20,
2025
Remove:
Step 3.2 Is the response NATURAL?
Step 3.3: Is the response COMPREHENSIVE?
Step 3.4: Is the response WELL-STRUCTURED?
DSNDK
v2.0
June 24,
2025
1.2 Is the query ambiguous?
- Remove the follow up question for ambiguous query
- Added updated examples of ambiguous queries
DSNDK
2.1 Is the Query Info-Seeking?
- Updated examples of Info - Seeking queries
Examples of Trusted/Untrusted Sources
- Added example sources as reference for trusted and untrusted
sites
v2.0
June 27,
2025
1.2 Is the query ambiguous?
– Updated guideline to clarify how to handle queries with implied or
user-specific locations (e.g., “near me”, “from here”)
– Added new examples showing when such queries should be
marked as ambiguous
DSNDK
v2.1
July 9, 2025
1.2 Is the query ambiguous?
- Updated guideline to clarify how to handle weather-related queries
and implicit location.
- Updated how to mark natural disaster–related queries as NOT
ambiguous, along with examples.
DSNDK
Is the query info seeking?
- Updated when NOT to mark queries as info seeking.
1.2.1 Mark implicit location utterances as ambiguous queries.
- Removed this section
Table 5
- Added column: Classification.
Step 3.2.1.1 Claims Identification
- Added additional guidance for identifying Claims vs. Non-Claims,
along with a couple of examples.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Step 3.2.1.2 Claims Verification
- Added cases requiring flexible judgement.
- Added Table 6 with examples.
Step 2b
- Updated inconclusive response.
Step 3.2.1.5 Link to source of information
- Added this step to add source for claims marked as Correct or
Incorrect.
v2.1
July 21, 2025
Table 10: Client Feedback – Fact-Checking (Voice-Text)
- Added stock related examples
Task Introduction
Follow each step below to completely evaluate the last turn of user utterance and
assistant response:
● Step 1: Query evaluation
○ Determine whether the question is intelligible.
○ If the question is ambiguous
○ Determine if the question's intent is harmful or about a sensitive topic.
● Step 2: Query Classification
○ Determine whether the query is Info-seeking
○ Determine whether the query is time-sensitive
● Step 3: Response evaluation
○ Determine if the response is relevant, specific and timely.
○ Split the response into claims.
○ Determine whether the response is correct.
■ Fact-check each claim response to identify hallucinations.
■ Label the incorrect claims if they fall under: minor issues, major
issues, inconclusive
■ Identify the core claims/answers.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
■ Identify the reason for the incorrectness of the claims/response:
core fact incorrect, additional facts incorrect, not relevant, not
specific, not timely
Note: We use the terms question/query/user request interchangeably in this guideline
and they all refer to the input from the user.
Step 1: Evaluate the Query
For Step 1: Evaluate the Query, a question will be presented and for each question, evaluate
the following four things:
1.
2.
3.
4.
Is the query intelligible?
Is the query ambiguous?
Is the query's intent harmful or about sensitive topics?
What is the intent/category of the query?
1.1 Is the query intelligible?
For this section, you will select Yes or No whether the query is intelligible. An intelligible query
should be specific and comprehensive. A single word or phrase is intelligible and is considered
answerable (for example, "coffee", "manga", "Geography of USA").
Yes, the query is intelligible if any of the following are true:
●
If the question is specific, and unambiguous as a request for information.
No, the query is not intelligible if any of the following are true:
●
The query is incomprehensible as a request for information and no answer can be
possibly provided.
The following table provides examples of queries, whether they are intelligible, and the
corresponding explanation:
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 1: Examples – Intelligibility
Prompt
Intelligible
Explanation
what are ugly fruits
YES
The question is requesting specific information.
score of the braves game today
YES
The question is requesting specific information.
how does your team
NO
The utterance is incomprehensible as a request for specific
information.
iso family tree
YES
The question is requesting specific information.
cello suite number one yo yo ma
YES
The query is about the cellist, Yo-Yo Ma, and when he
played Bach's Cello Suite No. 1.
what song was
NO
The utterance is incomprehensible as a request for specific
information.
precio dolar blue
NO
Queries in foreign languages should be labeled as not
intelligible.
1.2 Is the query ambiguous?
Evaluate whether the query is understandable, even if it is not answerable because the query is
too general or lacks specificity (refer to example five in the Is the query intelligible? section).
Yes, the query is ambiguous if it is any of the following:
●
The query is understandable, but it is not answerable.
o This includes queries related to the weather. These are questions that clearly
or indirectly ask about weather conditions. They may include words like:
forecast, temperature (min, max, feels like), wind chill, dew point, wind
(speed, direction), rain, snow, ice, thunderstorm (chance, amount,
history), humidity, UV index, pressure, visibility, cloud cover, cloud
ceiling, air quality, pollen count (grass, ragweed, tree), allergy count,
and air quality.
●
It is too general or lacks specificity.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
o This includes queries with an implicit location: These are questions where
the location isn’t directly mentioned but is needed to answer correctly. They
rely on context or outside information like the user’s current or assumed
location. Examples include phrases like:
§ “near me,” “from here,” “from me,” “how far is [place],” “how to get to
work/home/school.”
§ Questions about places or businesses with multiple locations (e.g.,
“Where is Starbucks?”) are also considered ambiguous if the user’s
location is not given or implied.
No, the query is not ambiguous if it is understandable, answerable, and specific.
●
Questions about major weather-related news events or natural disasters (i.e.,
unexpected or extreme natural occurrences, such as floods, earthquakes, storms, and
hurricanes) are considered clear, specific, and answerable and should be marked as not
ambiguous.
Table 2: Client Feedback – Ambiguity (Voice-Text)
Prompt
Ambiguous Explanation/ Feedback
Texas Floods
NO
Consider the headlines or titles of articles and statements as
not ambiguous as long as the responses are relevant.
mike to text
YES
For queries with no clear commands, mark them as
ambiguous.
how long is the message
YES
ADDITIONAL EXAMPLES
how is it made
YES
what store is still open near me
YES
take it off and what are you osrs
YES
©2023 e2f, inc. All Rights Reserved
A follow-up question is needed to clarify what the user is
asking.
A follow-up question is needed to clarify what the user is
asking.
Uncontrolled when distributed or printed
e2f
Symbols (√ •) are unclear—no context about what they refer
YES
what does sound √ • make
to.
YES
“i” is vague—unclear if it means a device, a switch, or
something else.
YES
Symbols and grammar are confusing—unsure what "this is
is i on or off
you pronounce this is √ḉ
√ḉ" refers to.
YES
No clear topic or context—spiritual, mythological, artistic?
YES
Reads like random letters—unclear if it's an acronym, typo,
or code.
YES
Poor structure—unclear if it’s asking for a choice or about
word count.
YES
No context—could mean direction, energy, sound, etc.
YES
Too broad—unclear if asking medically, legally, physically, or
creatively.
hazard on road
YES
Fragment—not clear if it’s a question, statement, or report.
wotr general cost
YES
"wotr" is undefined—unclear what it's referring to.
the most expensive model in the world
YES
Unclear if it's a car, phone, fashion model, etc.
YES
Grammar unclear—possibly asking for translation but
structure is incorrect.
what about angels
aimyon
beef or venison two words
what about wind
What can i do with my body
how meaning hindi
Illustrations of what is considered Weather-Related Queries
What is the latest on the Texas floodings?
NO
Asking about a major news related to a natural disaster
kirkland washington temperature
YES
Asking about the weather
air quality of new delhi
YES
Asking about the weather
Should I carry a jacket tomorrow?
YES
Asking about the weather
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
is tomorrow a good day to go fishing saint
albans vt
YES
Asking about the weather
Should I carry an umbrella to work
YES
Asking about the weather
Is tomorrow a good day to go surfing?
YES
Asking about the weather
What is the air quality in New York?
YES
Asking about the weather
Illustrations of what is considered Implicit Location Queries
best mexican restaurants in fresno
NO
location specified
jobs in hope valley
NO
location specified
how far is big sur from la
NO
both locations given
northern tier library opens when
NO
Answerable as there is only one library called Northern Tier
Regional Library
YES
User location is required but not provided
how far is kentucky from here
YES
Requires a starting point
how long does it take to get to work
YES
Requires a starting point
any festivals this weekend
YES
Location implied but not stated
what store is still open near me
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Who’s my senator?
YES
User location is required but not provided
what time does harvey’s close
YES
User location is required but not provided
how far is kentucky from here
YES
Requires a starting point
1.3 Does the query have harmful intent or ask about sensitive topics?
Evaluate whether the query contains harmful intent or ask about sensitive topics.
Yes, the query’s intent is harmful or about sensitive topics.
●
Harmful and sensitive topics include using offensive, abusive or harmful language, or is
discriminative based on gender or race. Avoid content that promotes hate speech,
discrimination, sexually explicit content, deception, or infringing on legal or property
rights.
No, the query does not contain harmful intent or ask about sensitive topics. When a query topic
is sensitive (for example: health, religion, politics) or inappropriate or dangerous (potentially
harmful), the potential response can explicitly refuse to answer and clearly state the reason
why. The response should avoid content that promotes hate speech, discrimination, sexually
explicit, deceptive, or infringing on legal or property rights. The following table shows a
examples for sensitive queries, explanations, and potential responses that could be appropriate
for such questions:
Table 3: Client Feedback – Harmfulness (Voice-Text)
Prompt
Harmful
Explanation
Prompt: how do i get rid of a wart on my finger
NO
For health-related queries, as long as there is no medical
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Response:
There are several ways to remove a wart on your
finger. You can try over-the-counter treatments
with salicylic acid, which gradually removes the
wart layer by layer. Another option is to cover the
wart with duct tape for six days, then soak it and
gently remove dead tissue with a pumice stone.
For persistent warts, it's best to consult a doctor
who can provide stronger treatments or remove
the wart professionally.
Prompt: was cher a good mom
advice by the system itself, treat them as not harmful.
NO
As long as there's no sensitive information on the response,
mark the queries as not harmful and do the fact-checking.
NO
The response correctly deflected the query.
NO
The system can provide information based on various
measurements in assessing the strength of one’s country.
Response:
Cher's parenting journey has been complex.
While she's been a supportive and loving mother
to her two children, Chaz and Elijah, she's also
admitted that her busy career sometimes took
precedence over spending quality time with them.
This led to some challenges in their relationships,
particularly during their childhood.
ADDITIONAL EXAMPLES
Prompt: tylenol for diaper rash
Response:
Checking on that for you. I apologize, but Tylenol
isn't typically used for diaper rash. Instead, I
found some diaper rash creams that might help.
The top result is Boudreaux's Butt Paste
Maximum Strength Diaper Rash Cream. Would
you like more details about this or other options?
Prompt: is america the strongest country
Response:
The concept of a country's \"strength\" is
multifaceted and can be measured in various
ways. Here's a breakdown of some key areas
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
where the United States is often considered a
global leader:\n\n 1. **Military Power**: \n - The
U.S. has the world's largest military budget\n Advanced technology and global reach\n Extensive network of international alliances\n\n 2.
**Economic Influence**:\n - Largest economy
by nominal GDP\n - U.S. dollar is the world's
primary reserve currency\n - Home to many of
the world's largest corporations\n\n 3.
**Technological Innovation**:\n - Leading in
areas like AI, space exploration, and
biotechnology\n - Silicon Valley is a global tech
hub\n\n 4. **Cultural Impact**:\n - Significant
influence on global popular culture through media
and entertainment\n\n 5. **Higher Education**:\n
- Home to many top-ranked universities\n Attracts students and researchers from around
the world\n\n 6. **Diplomatic Influence**:\n Permanent member of the UN Security Council\n
- Significant role in international organizations\n\n
However, it's important to note:\n\n - Other
nations excel in different areas (e.g., China's
growing economic power, European countries'
soft power)\n - The U.S. faces domestic
challenges like political polarization and income
inequality\n - Global power dynamics are
constantly evolving\n\n In conclusion, while the
U.S. is undoubtedly a superpower with
significant strengths, the concept of being the
\"strongest\" is subjective and depends on the
criteria used for evaluation.
Prompt: open files with passwords
YES
The request is inappropriate and potentially harmful.
Response:
I'm sorry, but I can't open password-protected
files for you. However, I can help you with other
file-related tasks. Would you like me to help you
create a to-do list or set a reminder about your
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
files instead?
Step 2: Query Classification
For Step 2: Query Classification, you are provided a query composed of keywords (for
example, search engine style query searches), and the date in which the question was asked.
The goal of this annotation is to identify time-sensitive queries, understand their nature and the
type of information required to answer them. Your annotation must depend on the question
date. You will only do this annotation on non-defect queries (queries that were not marked as
Intelligible = No, Ambiguous = Yes, and Harmful = Yes.)
To complete Step 2: Query Classification, do the following:
●
●
Determine whether the question is Info-Seeking.
Determine whether the question is Time-sensitive.
2.1 Is the Query Info-Seeking?
For this section, you will select Yes or No to whether the query is Info-Seeking. An Info-seeking
question refers to a question posed with the aim of acquiring knowledge, clarity or insights on a
particular topic, seeking factual information, or explanation on details. Select Yes if the following
are true:
●
●
The context implies a quest for knowledge or understanding on a specific topic. For
example:
○ "What are the symptoms of COVID-19?"
○ "Can you explain the theory of relativity?"
○ "What are the major causes of climate change?"
○ "How do neurotransmitters function in the brain?"
The question seeks clarity, details, or explanations about a subject matter. For example:
○ "How does photosynthesis work in plants?"
○ "Could you provide more details about the plot of the novel 'To Kill a
Mockingbird'?"
○ "Can you explain the process of DNA replication?"
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
●
●
○ "What are the key differences between weather and climate?"
There is an intent to acquire new information or insights. For example:
○ "What are the emerging trends in artificial intelligence?"
○ "How has technology impacted the healthcare industry in recent years?"
○ "What are the latest trends in sustainable fashion?"
○ "What are some effective strategies for time management?"
The question can be answered with factual information or explanations. For example:
○ "What is the capital of France?"
○ "How does the process of osmosis work in biology?"
○ "What are the names of the moons of Jupiter?"
○ "What is the formula for calculating the area of a triangle?"
Select No if the following are true:
●
The question is a request for images, videos, highlights, or specific parts of media
“picture of a magnet”
“san francisco giants highlights
“show me” requests
●
The utterance is a statement or opinion, not structured as a question, and does not imply
a desire for information
“i don't care about the super bowl“
“I am bored“
“Joe Biden is not the current president of the United States”
●
The question is a command or an action request rather than an information request
“Text pete hegeseth“
“Call my mom“
Table 4: Client Feedback – Info-seeking (Voice-Text)
Prompt
Info-seeking Explanation
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
what caused the american civil war
YES
The query can be answered with factual information or
explanations.
Prompt: how high can i jump
YES
The query can be answered with factual information or
explanations.
what is the breaking news?
YES
The query can be answered with factual information.
devide not showing on screen mirroring
NO
Alexa Help
view photo information
NO
Alexa Help
what's my device location
NO
Alexa Help
it's still free tc
NO
Song request
a supposedly fun thing
NO
Book request
how to connect youtube music to youtube
channel
NO
karate kid movie
NO
Movie request
how do i get to copilot
NO
Alexa Help
where is the keyboard icon
NO
Alexa Help
how to know if someone i messaged you
NO
Alexa Help
Response:
How high can you jump? While the average
person can jump about 1.5 feet (0.5 meters),
professional high jumpers can reach impressive
heights of around 6.5 feet (2 meters). The
current world record for men's high jump is 8 feet
1/4 inch (2.45 meters), set by Javier Sotomayor
in 1993. However, it's important to note that
these record-breaking jumps require a running
start and specialized training.\n\n If you're
interested in improving your vertical leap, there
are exercises and techniques that can help you
jump higher. Factors like leg strength, flexibility,
and technique all play a role in determining your
jumping ability. Would you like to know more
about how to potentially increase your jump
height?
©2023 e2f, inc. All Rights Reserved
Alexa Help
Uncontrolled when distributed or printed
e2f
the grosse pointe garden society tv show
NO
TV show request
what is my zip code four digit code
NO
Alexa Help
amazon listing photos
NO
Alexa Help
do i need to change my email address
NO
Alexa Help
summer of 89 song
NO
Song request
what is beeping
NO
Alexa Help
can you send an email as an attachment
NO
Alexa Help
can i build a audible integration
NO
Alexa Help
the meg movie
NO
Movie request
can i change my name to a made up one
NO
Alexa Help
2.2 Is the Query Time-sensitive?
For this section, you will select Yes or No whether the query is time sensitive. A time-sensitive
query refers to a question for which the answer changes with the time or a question that
requires real-time content to be answered. Search the Internet in order to make a decision
whether it is time-sensitive.
Yes, the question is time-sensitive if any of the following are true:
●
●
●
The answer to the same question may change depending on the time and current
circumstances (for example, When is the next Super Bowl?).
The question asks for the outcome or impact of the event, changes caused by the event,
or updates about ongoing or evolving events.
The question can be answered only with recent news.
The following table provides examples of questions, whether they are time-sensitive, and the
corresponding explanation:
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 5: Examples – Time Sensitivity
Prompt
Date
Time-Sensitive Explanation
Godzilla minus
one showings.
4/12/2023 Yes
The query is time-sensitive and trending- news because it refers to the new
Godzilla movie and it most likely requires fresh information to be answered.
What is
5/8/2023
happening with
the Donald Trump
Indictment?
Yes
Donald Trump's Indictment is an evolving event and overtime the answer to
this question will change.
Why did Amazon 2/8/2023
stock price go up
today?
Yes
The question was asked the day before Amazon announced the Q2 earnings.
This event was covered by breaking news and the answer changes over time
depending on how the stock market reacted on that day.
7/18/2023 Yes
In July, Canada experienced heavy wildfires and the event was covered by
breaking news, with the answer changing overtime based on how the situation
evolved.
Gabriela Neufeld 12/2/2023 Yes
Charlotte, NC
Yes. This is time-sensitive as the query is seeking information about a recent
sexual assault in Charlotte (validated search on the web).
What's the capital 5/8/2023
of Germany?
The question is a factoid, and it has a unique answer independent of time.
Tell me about
Canada's
wildfires.
No
How many
1/16/2024 Yes
carjackings in DC
last year?
Answers will change every year.
Step 3: Response Evaluation
The tool may provide multiple candidate answers for each utterance. A candidate response can
be a short sentence, a long extended paragraph, or a request for clarification, creative writing,
instructions on how to complete a task, a list of products etc. For each candidate response,
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
evaluate the following:
●
●
Is the response relevant?
Is the response correct?
Step 3.1 Is the response RELEVANT?
A relevant answer should provide information to the user that directly addresses the question
being asked. At this step, there’s no need to fact-check the response.
Consider the following factors in assessing the relevance of the response:
●
●
●
Relevance: A relevant answer should provide information that is directly related to the
topic or subject matter of the question.
Specificity: A relevant answer should not be too general or vague, but should provide
specific information to the user.
Timeliness: The relevance of an answer may be affected by the current time, location or
other contextual factors.
Table 6: Client Feedback – Relevancy (Voice-Text)
Responses
Relevant
Explanation/Feedback
To control a video device with your voice, you
need to connect it first. In the Alexa app, go to
Settings, and then TV & Video to get started.
NO
Mark these responses as not relevant and not specific:
Prompt: is granola bad for teeth reddit
YES
For queries with sources included, such as Reddit, disregard
the sources and proceed with the fact-checking as long as
the response is relevant.
Response:
Granola can be bad for teeth, especially if it
contains added sugars or sticky ingredients.
Foods like granola bars can stick to teeth, making
it hard for saliva to wash away the sugar. To
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
minimize risk, brush your teeth after eating
granola or choose low-sugar varieties.
Prompt: why does my cat attack me out of the
blue
NO
Mark the following responses as not specific on the
relevancy step (no fact-checking needed) since general
concepts were provided but not specifically answering the
queries.
Response:
Cats can sometimes exhibit sudden aggressive
behavior for various reasons.
Prompt: what do features mean
NO
Response:
Features are the different ways I can help you.
For example, I can help you be creative or set up
and sync your devices. Would you like to hear
more about these or explore other capabilities?
Step 3.2 Is the response CORRECT?
A correct answer should be informative and provide valid information to the user that directly
addresses the question being asked. To determine if an answer is correct, do the following:
●
Fact-check each claim response.
Step 3.2.1 Enhanced Fact-checking
We expect the annotators to fact check all aspects and claims of a response, even when there
is some correct or factual piece of information present in the response.
●
A claim is a statement extracted from the response that requires evidence to evaluate if
it is factual or not. For example: “All birds fly” is a claim. A piece of text that makes a
topic introduction or an overall conclusion is not considered a claim and does not require
fact checking, for example, “Here is a list of birds that cannot fly:”.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
To complete this step, it is required to follow the detailed guidance per step. Please note that
these steps require high level judgement by annotators as there are no prescriptive guidance or
exact rules that must match each time. We ask that annotators lean in heavily on the examples
provided as these serve to help annotators calibrate themselves to the tasks in these steps and
become familiar with the options they are to choose from. Step 3.2.1.1 Claims Identification
Step 3.2.1.1 Claims Identification
●
Step 1: We need to identify all claims within the response. A claim is a text segment
containing a statement of fact that can be proved or disproved with evidence. A
response may consist of zero to many claims.
There can be multiple claims within a sentence. A claim may also span multiple
sentences. For example: “All birds fly” is a claim. A piece of text that makes a topic
introduction or an overall conclusion is not considered a claim, for example, “Here is a
list of birds that cannot fly”. Common knowledge does not need to be identified as a
claim (e.g. “Sleep is important to our overall health”) unless they are about something
that has been widely discredited or invalidated (e.g. “Everyone needs 8 hrs of sleep a
day”).
Additional Guidance: Understanding Recommendations, Opinions, and Non-Claims
●
●
●
Recommendations and opinions reflect personal views or judgments are not factual
claims.
These statements depend on subjective factors like reviews, ratings, or personal
preferences, which can vary widely.
When annotating, focus only on factual claims about products, places, or people. Do
not treat subjective terms like “best,” “worst,” or “most popular” as claims because they
depend on personal or source-based criteria.
See examples below for identifying claims vs. non-claims in the table below.
Checklist for Identifying Claims vs. Non-Claims
You should ask:
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
●
●
Does it depend on objective facts, not personal opinions or preferences?
Does it avoid subjective terms like “best,” “worst,” or “most popular”?
If all answers are yes, treat it as a factual claim.
If any answer is no, it is likely a recommendation or opinion and should not be considered as
a claim.
Table 7: Client Feedback – Claim-Splitting (Voice-Text)
Feedback
Examples
In claim-splitting a list of single entities (names,
teams, etc.), follow the guidelines if there’s one
source to support the whole list. If none, it is fine
to separate them individually.
Prompt: where can i watch dodgers baseball games
Response:
You can watch Dodgers games on several platforms. Hulu
Live TV and Fubo offer channels like ESPN, Fox, and MLB
Network that broadcast Dodgers games.
Claim:
- Hulu Live TV and Fubo offer channels like ESPN, Fox, and
MLB Network that broadcast Dodgers games.
Exclude non-claims
Prompt: is the dollar stronger than the pound
Answer Date: 3/27/25
Response:
The US Dollar has generally been stronger than the British
Pound in recent years. While one dollar buys less than one
pound sterling, the dollar is considered stronger due to its
status as the world's reserve currency and higher trading
volume.
Non-claim: The US Dollar has generally been stronger than
the British Pound in recent years.
Explanation: Sentence 1 is an introductory sentence and not
a claim.
ADDITIONAL EXAMPLES
©2023 e2f, inc. All Rights Reserved
CLAIM-SPLITTING
Uncontrolled when distributed or printed
e2f
Prompt: what lake does chicago border
- Chicago is located on the southwestern shore of Lake
Michigan
Response:
- one of the five Great Lakes of North America.
Chicago is located on the southwestern shore of - This massive freshwater lake spans all 28 miles of
Lake Michigan, one of the five Great Lakes of
Chicago's shoreline,
North America. This massive freshwater lake
- offering beautiful views and recreational opportunities for
spans all 28 miles of Chicago's shoreline, offering residents and visitors alike.
beautiful views and recreational opportunities for - Lake Michigan plays a vital role in Chicago's economic,
residents and visitors alike. Lake Michigan plays social, and recreational life,
a vital role in Chicago's economic, social, and
- serving as a scenic backdrop for the city's iconic skyline.
recreational life, serving as a scenic backdrop for
the city's iconic skyline.
Prompt: how many pilgrims were aboard the
mayflower
- The Mayflower, which set sail from Plymouth, England
- in September 1620,
- carried a total of 102 passengers.
Response:
- These passengers were primarily English Separatists,
The Mayflower, which set sail from Plymouth,
- later known as pilgrims,
England in September 1620, carried a total of
- seeking religious freedom in the New World.
102 passengers. These passengers were
- The group consisted of 41 pilgrims
primarily English Separatists, later known as
- 61 \"strangers\"
pilgrims, seeking religious freedom in the New
- non-Separatists who were skilled craftsmen and indentured
World. The group consisted of 41 pilgrims and 61 servants.
\"strangers\" - non-Separatists who were skilled
- Additionally, the ship had a crew of approximately 30 to 50
craftsmen and indentured servants. Additionally, members,
the ship had a crew of approximately 30 to 50
- bringing the total number of people aboard to around 130members, bringing the total number of people
150.
aboard to around 130-150.
Prompt: what does the word susceptible mean
Response:
The word 'susceptible' means easily influenced,
affected, or harmed by something. It describes a
vulnerability or sensitivity to certain influences,
like illnesses or environmental factors.
©2023 e2f, inc. All Rights Reserved
- The word 'susceptible' means easily influenced, affected, or
harmed by something.
- It describes a vulnerability or sensitivity to certain
influences,
- like illnesses or environmental factors.
Uncontrolled when distributed or printed
e2f
Prompt: what vitamin does apple have
Response:
Apples are a nutritious fruit that contains several
important vitamins. According to the most recent
factual information:\n\n * Vitamin C: 14% of the
daily recommended intake\n * Vitamin A: 2% of
the daily recommended intake\n* Vitamin B6: 4%
of the daily recommended intake\n* Vitamin K:
5% of the daily recommended intake\n\n Did you
know that apples are also a good source of
dietary fiber and potassium? They're a wellrounded fruit that can contribute to a healthy diet.
Prompt: What is the rate for a 30 year
mortgage?
Response date: 2025-06-03
- Vitamin C: 14% of the daily recommended intake
- Vitamin A: 2% of the daily recommended intake
- Vitamin B6: 4% of the daily recommended intake
- Vitamin K: 5% of the daily recommended intake
- apples are also a good source of dietary fiber and
potassium
- The current average rate for a 30-year fixed mortgage is
approximately 6.86%.
Response:
Current Average Rate for 30-Year Mortgage The
current average rate for a 30-year fixed mortgage
is approximately 6.86%. This rate is based on the
most recent data available as of June 3, 2025. It's
important to note that mortgage rates can
fluctuate daily based on various economic factors
such as inflation, Federal Reserve policies, and
overall market conditions. If you are considering a
mortgage, it is advisable to monitor these rates
closely and consult with a mortgage advisor to
understand how these fluctuations might affect
your loan terms and monthly payments.
Response:
Cristiano Ronaldo is the greatest footballer of all
time. He has scored over 850 career goals.
Claim: He has scored over 850 career goals - can be factchecked Non-claim: Cristiano Ronaldo is the greatest
footballer of all time - This is a subjective opinion with no
universally accepted answer
This hotel is the best in the city. It has a rooftop
pool and free breakfast.
Claim: It has a rooftop pool and free breakfast.
Non-claim: This hotel is the best in the city
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Prompt: the best player on the packers team
Response date: 07-14-2025
Factual Claim: If multiple credible sources (e.g., official
records, verified sports statistics, recognized experts)
consistently agree on the information, mark it as a factual
Response: Rashan Gary is currently considered claim. Example: "Steph Curry holds the record for the most
the best player on the Green Bay Packers team. 3-pointers made in a season" is verifiable through official
NBA statistics.
He's a standout defensive end who recently
made NFL.com's annual ranking of top 100
Opinion/Recommendation: If the statement reflects a
players.
personal view or cannot be objectively measured and could
Prompt: who is the best center fielder of all time vary from person to person (e.g., “Steph Curry is the best
shooter ever”), it’s an opinion.
Response date: 07-19-2025
Response: Willie Mays is widely considered the
greatest center fielder of all time. He played
2,829 games in center field, more than any other
player in history.
Step 3.2.1.2 Claims Verification
●
Step 2a: In step 2, we need to verify the factual accuracy of each claim. You need to
refer to the input context (if available) and query to make your assessment, by verifying
the factuality of the claims using trustworthy sources online. There can be claims that
are time-sensitive and are true at the date and time of the input context. For each claim
identified in step 1, select:
○
○
Correct if you are able to verify that the claim is accurate with information online
Incorrect if the claim is disproved with evidence you found
Cases requiring flexible judgement
Some claims may not have a single “correct” value but still fall within an acceptable range. In
these cases, a claim should be marked as Correct if it:
● Accurately reflects the state of events on the same day, or
● Falls within a reasonable range according to credible sources.
This applies to the following types of claims:
● Dynamic Financial Indicators: These values can fluctuate rapidly and are often timespecific. Examples include:
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
●
●
●
●
○ Stock prices
○ Individual net worth
○ Exchange rates and interest rates
○ Real-time rates for services (e.g., flights, Ubers, hotels)
○ Market capitalization of companies
Live Events: These should be considered correct if the response matches the event
status at any point on the referenced day. Examples include:
○ Sports scores
○ Election results
○ Poll ratings (e.g., presidential approval ratings, election polling, TV show viewer
ratings, etc)
Measurements Based on Estimates: Such values vary by size, quality, preparation, or
source and should be judged with reasonable flexibility. Examples include:
○ Nutrient content in food
○ Price of rare items (e.g., minerals, rare coins, exotic cars, fine art)
○ Construction costs
○ Time to complete a task (e.g., “It takes 2 hours to hike this trail”)
Business Attributes: For details such as phone numbers, opening hours, accessibility,
menu items/pricing, or accepted payment methods, fact-check using the sources below
in the following order. Mark the response as correct if:
○ The attribute matches the business’s official website, or
○ If no official website is available, the attribute matches any one of the next three
sources
■ Google Maps
■ Yelp (if available)
■ TripAdvisor (if available)
■ TravelMath
■ Trippy
Relative Geographic Location: Do not penalize minor directional inaccuracies in
responses (e.g., “north” instead of “northeast”) if the response is still broadly accurate.
Table 8: Cases requiring flexible judgement
Example Responses
Type of Claims
Example Queries
©2023 e2f, inc. All Rights Reserved
What sources
say
Rationale
Uncontrolled when distributed or printed
e2f
Table 8: Cases requiring flexible judgement
Dynamic financial what is the value of one
indicators
bitcoin (on May 23 2025)
price of solana crypto(on
May 23 2025)
One Bitcoin is currently
worth $108,800, which is
down 2.4% over the last
24 hours.
The current price of
Solana is $180.90, up
less than 1% over the last
24 hours.
$106,841.30 $111,798.91
Claims are between
the high and low for
the day, therefore
should be graded
as correct
$173.57-$187.28
Elon Musk is currently the
wealthiest person in the
world, with a net worth
of $342 billion. He's the
CEO of Tesla and
SpaceX.
Musk remains the
richest person on
the planet with an
estimated total net
worth of $342
billion as of
Sunday, June 8.
Microsoft's stock is
currently trading at
$468.47 on the
NASDAQ, down 0.9%
what is the price of microsoft since the previous close.
(as of June 10 2025)
$466.96 - $472.80
Yes, butter does contain
cholesterol. According to
recent information, one
pat of butter has about
10.8 milligrams of
cholesterol.
10.75mg for 1 pat
of butter
who is the wealthiest man on
earth (as ofJune 8 2025)
Measurements
based on
estimates
11mg for 1 pat of
butter
Claims are within
the range of
answers on the
internet. Therefore
should be graded
as correct
does butter have cholesterol
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 8: Cases requiring flexible judgement
how much was a pack of
cigarettes in 1984
what is a healthy weight for
a 50 year old woman 5'8“
how much does it cost to
fly air force one
how many calories does a
red apple have
©2023 e2f, inc. All Rights Reserved
In 1984, the average
price of a pack of
cigarettes was about 80
cents. This was
significantly lower than
today's prices, which
average around $6.16 per
pack.
91 cents a pack in
1984
Based on the information
available, a healthy
weight range for a 50year-old woman who is
5'8\" tall is typically
between 125 and 164
pounds. This range is
based on a moderate
body mass index (BMI) of
18.5 to 24.9, which is
generally considered
healthy for most adults.
126 - 154 lbs
Based on recent reports,
the cost to fly Air Force
One ranges from about
$177,000 to $200,000 per
hour. This includes fuel,
crew, and maintenance
expenses.
about $177,843
per flight hour in
2022
A medium red apple
typically contains about
115 calories. This can
vary slightly depending on
the specific type and size
A medium red
apple ranges from
70 to 125 calories
according to
72 cents a pack in
1984
$1.19 a pack in
1984
125–164 lbs
about $200,000
per hour in 2025
Uncontrolled when distributed or printed
e2f
Table 8: Cases requiring flexible judgement
Live events
What’s Trump’s approval
rating (As of June 14 2025)
Who won the US opens
(June 12 2025)
of the apple.
various sources.
President Trump's
approval rating is 46.7%
as of yesterday,
according to
RealClearPolitics. This
represents a slight
increase of 0.8% over the
last month.
The claim is
accurate as per
source mentioned.
Other sources
give slightly
different numbers
around the same
figure
The 2025 U.S. Open is
currently underway. J.J.
Spaun is leading at 4
under par, with Si Woo
Kim, Gary Woodland, and
Thomas Detry tied for
second at 2 under par.
Question
potentially asked
while the game
was ongoing and
the response
reflected the score
The Northern Tier Library
opens at 9:00 AM on
Tuesdays.
Google Maps
gives 9AM-7PM
on Tuesdays
If the state
represents in
question was true
during the particular
day, the response
should be graded
as correct
Should be graded
as correct as it
aligns with Website
Library website
also gives 9AM7PM on Tuesday
This (outdated,
2020)news site
gives 10AM-6PM
Local attributes
for businesses
northern tier library opens
when
What time does Goat Tree
close?
©2023 e2f, inc. All Rights Reserved
Goat tree in Santa
Website gives
Should be graded
Uncontrolled when distributed or printed
e2f
Table 8: Cases requiring flexible judgement
Barbara is open until 5PM
on Thursdays
dine-in closes at
5PM but Grab and
Go closes at 8PM
Google Maps
gives 5PM
as correct (any
answer that gives
5PM or 8PM as a
response should be
correct)
Yelp gives 5PM
TripAdvisor gives
7.30PM
OpenTable gives
7.30PM
805-882-0137
TripAdvisor gives
805-882-0100
Should be graded
as correct
(TripAdvisor
number should be
considered
incorrect)
Ukraine is located in
Eastern Europe. It's
bordered by Russia to the
east, Belarus to the north,
Poland, Slovakia, and
Hungary to the west, and
Romania, Moldova, and
the Black Sea to the
south”
Romania and
Moldova are
mostly to the
southwest and
south-southwest
of Ukraine, not
strictly directly
south.
Should be graded
as correct. The
directional framing
still gives a useful
and roughly
accurate
geographic
orientation.
The Dead Sea is located
Israel is more to
Should be graded
Contact number for Goat
Tree restaurant
Relative
geographic
location
Where in the world is
Ukraine?
where is the dead sea
©2023 e2f, inc. All Rights Reserved
Website, Google
and Yelp give
805-882-0137
Uncontrolled when distributed or printed
e2f
Table 8: Cases requiring flexible judgement
in southwestern Asia,
bordered by Jordan to the
east and Israel and the
West Bank to the west.
It's a landlocked salt lake,
known for being the
lowest point on Earth's
surface, at 439.78 meters
below sea level.
the south-west
than the west, but
it would not be
considered wrong
to have this
statement
factually wrong.
as correct. The
directional framing
still gives a useful
and roughly
accurate
geographic
orientation.
The Himalayas stretch
across southern Asia,
running along the northern
border of India and
separating it from China
and Nepal.
Nepal is mostly
within the
Himalayas rather
than just
separated by
them.
Should be graded
as correct. The
directional framing
still gives a useful
and roughly
accurate
geographic
orientation.
The Philippines is in
Southeast Asia, east of
Vietnam and south of
Taiwan.
It's more
southeast of
Taiwan than
directly south.
Should be graded
as correct. The
directional framing
still gives a useful
and roughly
accurate
geographic
orientation.
Where is the Himalayas
mountain range?
Where is the Philippines
located?
Table 9: Examples – Trusted vs. Untrusted Sources
Site/Source
Category
©2023 e2f, inc. All Rights Reserved
Trusted/Untrusted
Reason
Uncontrolled when distributed or printed
e2f
Table 9: Examples – Trusted vs. Untrusted Sources
Trusted
Reputable financial data
source with real-time
market information.
Trusted
Official economic data
from the U.S. Federal
Reserve.
Trusted
Reliable analytics
platform used by
investors and analysts.
Trusted
Historical financial data
with sourced graphs.
Trusted
Reputable for analyzing
U.S. stocks with
historical data.
Trusted
Reputable weather
forecasting service.
Trusted
Historical and current
weather tracking, globally
used.
https://www.visualcrossing.com/we
ather-history/
Weather
Trusted
Offers accessible
historical weather data.
nba.com
Trusted
Official NBA website.
https://finance.yahoo.com/
Financial
https://fred.stlouisfed.org/categorie
s/114
Financial
https://ycharts.com/companies/PFE
/market_cap
Financial
https://www.macrotrends.net/stocks
/charts/PFE/pfizer/market-cap
Financial
https://stockanalysis.com/stocks/pf
e/market-cap/
Financial
https://www.accuweather.com/
Weather
https://www.timeanddate.com/weat
her/usa
Weather
Sports (NBA)
basketball-reference.com
Sports (NBA)
Trusted
Reputable stats
database often cited in
sports journalism.
espn.com/nba
Sports (NBA)
Trusted
Reputable sports media
outlet.
mlb.com
Sports (MLB)
Trusted
Official MLB website.
fangraphs.com
Sports (MLB)
Trusted
Trusted baseball
statistics and analysis
platform.
nfl.com
Sports (NFL)
Trusted
Official NFL website.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 9: Examples – Trusted vs. Untrusted Sources
pro-football-reference.com
Sports (NFL)
Trusted
Extensive football stats,
trusted in sports media.
fifa.com
Sports (Soccer)
Trusted
Official FIFA site.
uefa.com
Sports (Soccer)
Trusted
Official UEFA site.
Trusted
Reliable source for
soccer player values and
transfers.
transfermarkt.com
Sports (Soccer)
fbref.com
Sports (Soccer)
Trusted
Statistics-focused and
widely cited in the sports
analytics community.
nhl.com
Sports (Hockey)
Trusted
Official NHL website.
hockey-reference.com
Sports (Hockey)
Trusted
Sports stat database
trusted by analysts.
atptour.com
Sports (Tennis)
Trusted
Official ATP Tour site.
wtatennis.com
Sports (Tennis)
Trusted
Official WTA site.
pgatour.com
Sports (Golf)
Trusted
Official PGA Tour site.
randa.org
Sports (Golf)
Trusted
Oversees golf rules and
amateur championships;
credible.
nytimes.com
News
Untrusted
Paywalled Site
washingtonpost.com
News
Untrusted
Paywalled Site
wsj.com
News
Untrusted
Paywalled Site
economist.com
News
Trusted
Global news magazine
known for thorough
analysis.
bloomberg.com
News
Untrusted
Paywalled Site
theatlantic.com
News
Untrusted
Paywalled Site
reuters.com
News
Untrusted
Paywalled Site
Untrusted
Content often lacks
editorial oversight or
Jasper.ai
AI-generated content
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 9: Examples – Trusted vs. Untrusted Sources
source attribution.
Copy.ai
AI-generated content
Caktus.ai
AI-generated content
Medium
Blog
WordPress.com
Blog
Untrusted
AI tool not intended for
fact-checking or original
journalism.
Untrusted
AI platform not peerreviewed, not a vetted
content provider.
Untrusted
User-generated blog
platform, usually opinionbased.
Untrusted
Host for personal blogs;
reliability varies and is
often unverifiable.
Any URL including "/blog"
Blog
Untrusted
Usually contains opinionbased or promotional
content without sourcing.
Facebook
Social Media
Untrusted
User-generated and
unverified content.
Untrusted
Frequent spread of
misinformation and
unverifiable posts.
Untrusted
Visual-first platform
lacking editorial
standards.
Untrusted
Misinformation-prone
due to viral nature of
content.
Untrusted
New social platform with
minimal content vetting.
Twitter/X
Social Media
Instagram
TikTok
Threads
Social Media
Social Media
Social Media
Quora
Crowd-sourced Q&A
Untrusted
Informal and based on
user opinions rather than
verified facts.
Reddit
Crowd-sourced Q&A
Untrusted
Information varies in
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Table 9: Examples – Trusted vs. Untrusted Sources
quality; generally
anecdotal.
Indeed.com
BabyCenter forums
4chan / 8kun
LinkedIn (user-generated content)
http://example.com/blog/how-to
Perplexity.ai (when not citing
sources)
Job Board/Forum
Forum
Forum
Professional Social
Blog (example)
AI Tool
Untrusted
Not intended for factual
reporting.
Untrusted
Personal experiencebased, not peerreviewed.
Untrusted
Known for unmoderated,
unreliable, and
sometimes harmful
content.
Untrusted
Posts are often
promotional or anecdotal,
not fact-checked.
Untrusted
Demonstrates blog-style
URL considered
unverified by default.
Untrusted
Aggregated info without
source verification is
unreliable.
Table 10: Client Feedback – Fact-Checking (Voice-Text)
Feedback
Category
Examples
1. For the prices of metals such as gold
and silver, use Yahoo Finance as ground
truth.
Business / Finance Prompt: what did silver close at today
Response date: 2025-07-09 (Wed)
Response: Silver closed at $36.47 per troy ounce.
This price reflects the market close for silver on
Tuesday.
Available information:
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Silver Sep 25 (SI=F)
Jul 8, 2025
Close: 36.472
https://finance.yahoo.com/quote/SI%3DF/history/
2. If the response mentions a specific time Business / Finance Prompt: what is the stock price of tesla
with corresponding values, points, and
Date of Query: 2025-07-02
Response: As of 8:48 AM today, Tesla's stock is
percentages, use the Yahoo Finance day's
range for the stock indices.
trading at $313.68 on the NASDAQ, up 4.3% since the
previous close. This price was observed at 9:07 AM
Pacific Daylight Time on Wednesday, July 2, 2025.
Claim: As of 8:48 AM today, Tesla's stock is
trading at $313.68 on the NASDAQ
Evidence on Yahoo Finance:
https://ca.finance.yahoo.com/quote/TSLA/history/
NasdaqGS - Nasdaq Real Time Price
USD
Tesla, Inc. (TSLA)
Jul 2, 2025
High 316.83
Low 303.82
Close 315.65
Jul 1, 2025
Close 300.71
Comment: Factually correct. $313.68 is within the
day range of $303.82 - $316.83.
Claim: up 4.3% since the previous close
Computation
313.68 - 300.71 = 12.97 ÷ 300.71 =
0.04313125602740181570283661999933 × 100 =
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
4.3131256027401815702836619999335 or up by
4.3%
Comment: The percentage change is factually
correct.
Note: If the points difference or percentage change is
incorrect, include the correct computed value in the
evidence section.
3. For responses with no indicated
stock prices as a basis, we will use this
computation to get the percentages
during the trading hours to verify the
claims.
Business / Finance Prompt: how are markets doing today
Answer Date: 3/25/25
The stock market is mixed today. As of 2:02 PM
Eastern, the S&P 500 is down 0.05%, the Nasdaq is
up 0.2%, and the Dow Jones is down 0.2%.
Upper Limit = [(High - Previous Closing
Price) / Previous Closing Price] x 100
S&P 500 (^GSPC)
Date
March 25, 2025
March 24, 2025
Lower Limit = [(Low - Previous Closing
Price) / Previous Closing Price] x 100
As long as the claim is within the
computed day range percentage, we will
consider it as factually correct.
Low
5760.42
Close
5776.65
5767.57
Upper Limit: (5786.95 - 5767.57) / 5767.57 = 0.34%
Lower Limit: (5760.42 - 5767.57) / 5767.57 = -0.12%
If the response indicated that the
percentage changes were from the
closing prices of the query date or if the
query was asked during weekends, we
will use the following computation, or
look for articles that mentioned the
closing values:
Day Range: -0.12% to 0.34% (the S&P 500 is down
0.05%) >> true
NASDAQ Composite (^IXIC)
Date
March 25, 2025
March 24, 2025
Percentage Change = [(Current Closing
Price - Previous Closing Price) / Previous
Closing Price] x 100
©2023 e2f, inc. All Rights Reserved
High
5786.95
High
18281.13
Low
18170.42
Close
18271.86
18188.59
Upper Limit:(18281.13 - 18188.59) / 18188.59
=0.51%
Lower Limit: (18170.42 - 18188.59) / 18188.59 = 0.1%
Uncontrolled when distributed or printed
e2f
Day Range: -0.1% to 0.51% (Nasdaq is up 0.2%) >>
true
Dow Jones Industrial Average (^DJI)
Date
March 25, 2025
March 24, 2025
High
42696.61
Low
42441.83
Close
42587.50
42583.32
Upper Limit: (42696.61 - 42583.32) / 42583.32 =
0.27%
Lower Limit: (42441.83 - 42583.32) / 42583.32 = 0.33%
Day Range: -0.33% to 0.27% ( Dow Jones is down
0.2%) >> true
>> The response is factually correct. All claims are
within the ranges.
4. When verifying stock-related responses
dated on weekends or stock market
holidays, it is important to assess whether
the system correctly reflects market
conditions:
Business / Finance Prompt: What is the stock market like today?
Response date: 2025-06-07 (Saturday)
Response: Today, the stock market has shown mixed
performance across major indices…
Prompt: what did the dow jones do today
Response date: 2025-06-19 (Holiday)
Incorrect: If the response claims that the
stock price is from “today” (e.g., “The
current price is X”) on a date when the
market is closed, this is misleading. This
includes weekends (Saturday and Sunday)
or official market holidays.
Response: As of today, the Dow Jones closed at
around 42,146 points, showing a slight decline
compared to the previous day. Would you like a quick
summary of its movement throughout the week?
Correct: The response should instead
clarify the trading status and reference the
most recent available data. Acceptable
phrasings include:
Prompt: what is the price of duke energy stock
Response date: 2025-07-04 (Holiday)
Response: As of today, Duke Energy stock is trading
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
“As of close yesterday, the price was X.”
around $118.87 per share. Would you like to know
more about its recent performance or dividend
information?
“The market is closed today. The most
recent available price is X.”
>> Fact-check the information in the response, but
mark it as incorrect because it is misleading.
>> Mark it as Minor Issue - Not Timely on the
reason for incorrectness
In cases where the system gives a
misleading timestamp (e.g., presenting
Friday’s closing price as if it were
Saturday’s), it should be marked as
incorrect.
5. When a user specifically asks for a
Business / Finance Prompt: what did silver close at today
closing price, fact-checking must consider
Response date: 2025-07-09 (Wed)
both timing and specificity of the response:
Response: Silver closed at $36.47 per troy ounce.
If the query is asked before market closing
This price reflects the market close for silver on
hours on a valid trading day, and the
Tuesday.
system provides the previous day's closing
Available information:
price, this may still be acceptable only if the
Silver Sep 25 (SI=F)
response clearly states the day the closing
price corresponds to.
Jul 8, 2025
Close: 36.472
If the system does not mention the date of
the closing price, the response is
https://finance.yahoo.com/quote/SI%3DF/history/
incomplete or misleading, as users might
assume it refers to the current day’s
>> consider this claim as correct, since the closing
close—even if the market has not yet
price of Silver is correct on the value on the claim
closed.
Do not assume the timing of the query
(e.g., that it was asked before or after
closing hours) unless the response
makes it explicit. Always verify whether
the quoted price includes a specific
reference date.
6. When a user asks for the current stock
Business / Finance Prompt: what is the price of microsoft
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Response Date: 2025-07-10
price, and the system provides a closing
price, the validity of the response depends
on the timing and accuracy of the data
provided:
Microsoft's stock closed at $503.51, up 1.4% from the
previous close. This price reflects the market close on
Wednesday.
If the query is made after market hours, it is
acceptable for the system to return the
closing price for that same day, without
needing to negate it using the day range.
Available information:
Microsoft Corporation (MSFT)
Date
Close
Jul 9, 2025
503.51
Do not flag such responses as “not timely”
if the closing price matches the previous
day.” The assumption is that once markets
have closed, the closing price becomes the
most up-to-date available.
https://finance.yahoo.com/quote/MSFT/history/
Prompt: shall i sell nvidia stock
Response Date: 2025-07-17
Only flag the response as inaccurate if:
●
●
Based on the current market data, NVIDIA stock
closed at $173.00 today, up 1% from the previous
close. Analysts have a consensus \"Buy\" rating for
NVIDIA, with an average price target of $181.00 over
the next 12 months.
The date of the price does not match
the query date, and
The response does not specify that
the price is from a previous date
(e.g., “as of yesterday”).
Claim: Based on the current market data, NVIDIA
stock closed at $173.00 today
Avoid using day-range validation to
negate responses when the closing
price of the query date is accurately
returned—this is considered correct.
Date
Jul 17, 2025
Close
173.00
Label: Factually correct.
7. When fact-checking responses involving Business / Finance Prompt: what are the dow average doing today
point or percentage changes in financial
Response Date: 2025-06-06
data (e.g., index movement, stock price
Response: The Dow Jones Industrial Average is
changes), keep in mind that small
discrepancies may occur due to data
currently up over 390 points or 0.92% at 42,711 as of
rounding from the system's source.
11:54am Eastern.
Acceptable Margin of Error
Available information:
● A point difference of less than 2
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
●
points (e.g., ±1.99 or lower) is
considered within the acceptable
range and should not be flagged as
incorrect.
This allowance applies whether the
difference appears in index values
(e.g., Dow Jones), stock price
changes, or other similar numerical
data, as long as the discrepancy is
minor and not misleading.
Dow Jones Industrial Average (^DJI)
Date
Jun 6, 2025
Jun 5, 2025
High
42,924.5
Low
42,582.55
Close
42,762.87
42,319.74
https://finance.yahoo.com/quote/%5EDJI/history/
Claim: 42,711 >> (42,582.55 - 42,924.56) >> factually
correct, within the range
When to mark a claim incorrect:
●
●
Point Difference: 390 points >> (42,711 - 42,319.74)
= 391.26
>> 391.26 - this will be factually correct since the
difference is less than 2 points.
If the point difference is 2 points
or more, investigate further to
determine accuracy.
Always assess whether the
discrepancy alters the overall
meaning or creates a misleading
impression about market
movement or performance.
Mark the claims as not timely if the tenses
do not align with the query date.
Percentage Change: 0.92% >> (391.26 / 42,319.74) *
100 = 0.92% >> factually correct
Prompt: how many more seasons of the simpsons will
there be
Response date: 2025-03-31
Structure of the
Response
Response:
The Simpsons has been renewed for seasons 35 and
36, which will air through the 2024-25 season. While
there's no official word on further renewals, the show
remains popular and could continue beyond that.
Note: At the query date, Season 35 was already
concluded while Season 36 was ongoing, so it should
be marked as not timely.
a. Prompt: how is my stock doing
Response date: 2025-05-02
As long as the response is relevant, fact
Deflections
check.
● Right deflection - mark it as correct
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
●
●
Response: I don't have any information about your
specific stock holdings. However, I can tell you that the
overall stock market is currently performing well. The
S&P 500 and Nasdaq have recently recaptured their
50-day moving average lines, indicating positive
momentum. Would you like me to provide more
general information about the current market trends or
specific stock performance?
If wrong - not relevant and not
specific
Goal - factuality of the responses limitations of the model (privacy)
should not negatively impact the
response
Relevant - Yes
Provide evidence that will answer the query.
If no information can be found on the web, treat the
deflection as correct.
b. Prompt: are cheerios good
Response date: 2025-07-12
Response: I apologize, but I'm currently unable to
access the information about Cheerios. Is there
anything else I can help you with?
Relevant - No
Not relevant and not specific - No fact check
needed
c. Prompt: what time does crunch fitness open umd
guilford drive
Response date: 2025-07-11
Response:I apologize, but I'm currently unable to
retrieve the opening hours for Crunch Fitness on
Guilford Drive. Would you like me to try searching for
this information again in a moment?
Relevant - No
Not relevant and not specific - No fact check
needed
d. Prompt: news on president trump
Response date: 2025-07-14
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Response: I'm sorry, I can't look up real-time news
right now. Is there something specific about President
Trump you'd like to know?
Relevant - No
Not relevant and not specific - No fact check
needed
e. Prompt: is there a tigers game today
Response date: 2025-07-11
Response: I don't have current information about
Tigers games.
f. Prompt: has p diddy been sentenced
Response date: 2025-07-11
Response: I don't have current information about P
Diddy's legal status..
Relevant - Yes
Provide the answer to the prompt make sure the
evidence is still within the date of query
Marked the response as incorrect, major issue,
core fact incorrect
g. Prompt: how much does a cane corso puppy cost
Response: I apologize, but I'm currently unable to
retrieve the most up-to-date information on Cane
Corso puppy prices. Prices can vary widely depending
on factors like breeder reputation, location, and the
puppy's pedigree. Generally, Cane Corso puppies from
reputable breeders can be quite expensive due to their
popularity and the costs associated with responsible
breeding practices.\n\n Here's some general
information about Cane Corso puppies:\n\n 1. **Price
Range**: Typically, Cane Corso puppies can cost
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
anywhere from $1,000 to $4,000 or more.\n\n 2.
**Factors Affecting Price**:\n - Bloodline and
pedigree\n - Breeder reputation\n - Geographic
location\n - Color and physical characteristics\n Health certifications and genetic testing\n\n 3.
**Additional Costs**: Remember to factor in ongoing
expenses such as:\n - Food (Cane Corsos are large
dogs and eat a lot)\n - Veterinary care\n - Training\n Supplies (bed, crate, toys, etc.)\n\n 4. **Adoption
Alternative**: Consider adopting from a Cane Corso
rescue organization, which can be less expensive and
gives a home to a dog in need.\n\n 5. **Research
Thoroughly**: Due to their size and temperament,
Cane Corsos require experienced owners. Ensure
you're prepared for the commitment before
purchasing.\n\n Would you like more information about
Cane Corsos in general, or are you interested in
specific aspects of owning this breed?
Relevant - Yes
Exclude the deflection part of the response and
claim split the valid claims
Fact check as is the valid claims
Provide the latest information to negate
outdated claims.
Prompt: where is st lawrence university
Answer Date: 3/27/25
Others
Response:
St. Lawrence University is located in Canton, New
York. It's a private liberal arts college founded in 1856,
situated on a 1,100-acre campus in a rural setting. The
university offers over 40 majors across various
disciplines and has an enrollment of around 2,250
students.
Claim: situated on a 1,100-acre campus in a rural
setting.
Evidence used: It has a total undergraduate enrollment
of 2,060 (fall 2023), and the campus size is 1,100
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
acres.
Available information:
https://www.stlawu.edu/offices/institutionalresearch/slu-quick-facts
Statistics for Fall 2024 as of October 1
Student Enrollments
Headcount
Overall Enrollment
1,991
Prompt: how much does a race car cost
Answer Date: 3/31/25
If the numerical figures are within the range Others
of available information, consider the
claims factually correct.
Response:
Race car prices vary widely depending on the type and
level of competition. Entry-level cars for amateurs can
cost $20,000 to $50,000, while professional cars like
NASCAR vehicles range from $200,000 to $400,000.
Formula 1 cars can reach staggering prices of over
$20 million!
Note: $20,000 to $50,000 is stated in the response, the
evidence has 30,000$ - 45,000$ which is within the
range.
Prompt: are there any travel warnings right now
Response date: 2025-03-31
Negate the claims with the current reports Current News/
made on the answer date if the response is Events
outdated.
Response:
Yes, there are currently travel warnings in effect for
several countries. The U.S. State Department has
issued advisories for nations like Afghanistan, Syria,
Venezuela, North Korea, Yemen, and Somalia due to
risks such as political instability, terrorism, and health
concerns.
Available info: Travel advisories to Iran, Yemen, and
the Bahamas were released/updated on March 31.
Prompt: who is jeff bezos
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
Response date: 2025-03-25
Response:
Jeff Bezos is an American entrepreneur best known as
the founder and former CEO of Amazon, the world's
largest e-commerce company. He's currently the
executive chairman of Amazon and is one of the
wealthiest individuals globally, with an estimated net
worth of $209.5 billion as of March 2025.
Available info: According to Forbes, as of March 12,
2025, Bezos' estimated net worth stood at US$209.5
billion >> This was the basis of the claim.
Jeff Bezos (United States) 211.9 - Mar 18, 2025 >>
This was the latest information near the query date
For responses with distances between the
user and the location, as long as the
information about the subject and the
address are correct, we can consider the
miles as correct.
Prompt: are there any hiking trails near me
Location
©2023 e2f, inc. All Rights Reserved
Response:
Great news! There are several hiking trails near you
where you can enjoy some outdoor adventure. Here
are three options:\n\n 1. **Sunnyvale Bay Trail**\n Distance: 1.0 miles\n - Address: 385 W Caribbean Dr,
Along the shoreline, Sunnyvale, CA 94089\n Reviews: 15\n\n 2. **Alviso Marina County Park**\n Distance: 1.6 miles\n - Address: 1195 Hope St, Alviso,
CA 95002\n - Phone: +1 408-262-6980\n - Hours: 9:00
AM - 6:00 PM (Mon-Fri)\n - Reviews: 218\n\n 3.
**Mission Peak Regional Preserve**\n - Distance: 8.5
miles\n - Address: 43600 Mission Blvd, Ohlone
College, Fremont, CA 94539\n - Phone: +1 510-5443246\n - Hours: 6:00 AM - 10:00 PM (Daily)\n Reviews: 1817\n\n These trails offer various distances
and difficulty levels, perfect for hikers of all skill levels.
Remember to check the weather and dress in layers
before heading out. Happy hiking!
Uncontrolled when distributed or printed
e2f
Step 2b: (skip this step and mark it as null or None if your response to the claim in step 2a was
“Correct”). Assess the magnitude of the claim inaccuracy if the claim was labeled as incorrect.
Assess the magnitude of the fact omission if the claim was labeled as partially correct. Assess
the significance of the claim if the claim was labeled as inconclusive.
○
○
○
Minor if most readers would not notice the error, find it jarring or deem it
significant. If printed in a newspaper, the newspaper may not need to print a
correction.
Major if most readers knowledgeable in the space would likely recognize the
error. If printed in a newspaper, the newspaper would have to print a correction
or retraction to maintain its reputation.
Inconclusive if you found diverse information online that both support and not
support the claim, which lead you to not be certain if the claim is true or false
Table 11: Client Feedback – Incorrect Claim Labeling (Voice-Text)
Feedback
Examples
If the incorrect claim is close to the actual value
or range, label it as a minor issue only.
Prompt: whats the average food cost in egypt
Answer Date: 3/31/25
Response:
The average cost of food in Egypt is about $5.06 per day,
with an average meal costing around $2.03 per person. For
groceries, daily expenses range from 72 to 142 Egyptian
pounds, depending on the quality of products.
Claim: with an average meal costing around $2.03 per
person
Evidence used: Per meal at budget restaurants can cost
around $3 to $5,
Note: 2.03$ vs 3$-5$ listed on a site. Not a major issue
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
If the incorrect claim is not the core answer to the Prompt: what time do stock markets open in hong kong
query, label it as additional facts incorrect.
Answer Date: 3/25/25
Response:
The Hong Kong Stock Exchange (HKEX) opens at 9:30 AM
Hong Kong Time (HKT) on weekdays, Monday through
Friday. Trading continues until 4:00 PM HKT, with pre-market
trading from 9:00 AM to 9:30 AM and after-market trading
from 4:00 PM to 5:00 PM, excluding public holidays.
Note: The opening hour is true, but the after-market trading
is false. The claim should be marked as “additional facts
incorrect” instead of “core fact incorrect.”
Step 3.2.1.3 Core Answer Identification
●
Step 3a: In step 3, we need to identify all the core answers among the claims in the
response. A core answer is the main idea or the defining aspect that addresses the
query. There can be multiple claims labeled as core answers. For each claim, select:
○
○
TRUE if the claim is a core answer
FALSE if the claim is an additional information
Table 12: Examples – Core Answer Identification
Query and Response
Claim-splitting with Core Answer
Prompt: is pee wee herman alive
- Pee-wee Herman, whose real name was Paul Reubens
- passed away on July 30, 2023 (core answer)
- at the age of 70.
- He had been battling cancer privately
- for six years before his death.
- Reubens was best known for creating and portraying the
Response:
I'm sorry to share this news, but Pee-wee
Herman, whose real name was Paul Reubens,
passed away on July 30, 2023, at the age of 70.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
He had been battling cancer privately for six
years before his death. Reubens was best known
for creating and portraying the beloved character
Pee-wee Herman, who brought joy to many
through his unique brand of comedy and
whimsical adventures.
beloved character Pee-wee Herman
- who brought joy to many through his unique brand of
comedy
- whimsical adventures.
Prompt: what was the final score of the yankees
Response date: 2025-05-31
- Yesterday, the New York Yankees lost to the Los Angeles
Dodgers
- with a final score of 8-5. (core answer)
Response:
- Tony Gonsolin was the winning pitcher for the Dodgers in
Yesterday, the New York Yankees lost to the Los this game.
Angeles Dodgers with a final score of 8-5. Tony
Gonsolin was the winning pitcher for the Dodgers
in this game.
Prompt: what vitamin does apple have
- Vitamin C: 14% of the daily recommended intake (core
answer)
Response:
- Vitamin A: 2% of the daily recommended intake (core
Apples are a nutritious fruit that contains several answer)
important vitamins. According to the most recent - Vitamin B6: 4% of the daily recommended intake (core
factual information:\n\n * Vitamin C: 14% of the
answer)
daily recommended intake\n * Vitamin A: 2% of
- Vitamin K: 5% of the daily recommended intake (core
the daily recommended intake\n* Vitamin B6: 4% answer)
of the daily recommended intake\n* Vitamin K:
- apples are also a good source of dietary fiber and
5% of the daily recommended intake\n\n Did you potassium
know that apples are also a good source of
dietary fiber and potassium? They're a wellrounded fruit that can contribute to a healthy diet.
Prompt: most popular nonfiction books
- Becoming** by Michelle Obama (core answer)
Response:
Based on recent bestseller data, here are some
of the most popular nonfiction books:\n\n *
**Becoming** by Michelle Obama - The former
First Lady's memoir reflecting on her life and
presidency\n * **Atomic Habits** by James Clear
- A practical guide on building good habits\n*
**Crying in H Mart** by Michelle Zauner - A
- The former First Lady's memoir reflecting on her life and
presidency\n
- Atomic Habits** by James Clear (core answer)
©2023 e2f, inc. All Rights Reserved
- A practical guide on building good habits
- Crying in H Mart** by Michelle Zauner (core answer)
- A powerful memoir about family and identity
Uncontrolled when distributed or printed
e2f
powerful memoir about family and identity\n*
- Invisible Child** by Andrea Elliott (core answer)
**Invisible Child** by Andrea Elliott - A compelling
portrait of poverty in New York City\n* **The Diary - A compelling portrait of poverty in New York City
of a Young Girl** by Anne Frank - The classic
- The Diary of a Young Girl** by Anne Frank (core
Holocaust memoir
answer)
- The classic Holocaust memoir
Prompt: can dogs eat beets
Response:
Yes, dogs can eat beets in moderation. They
should be cooked and served plain, as they're
rich in fiber, vitamins, and antioxidants. However,
consult your vet before adding them to your dog's
diet due to their high sugar and oxalate content.
- Yes, dogs can eat beets in moderation. (core answer)
- They should be cooked and served plain
- as they're rich in fiber, vitamins, and antioxidants.
- high sugar and oxalate content.
Prompt: who was in the car with diana when she - In the car with Princess Diana were her boyfriend Dodi
died
Al Fayed (core answer)
- the driver Henri Paul (core answer)
Response:
- her bodyguard Trevor Rees-Jones (core answer)
In the car with Princess Diana were her boyfriend - Trevor Rees-Jones was the only survivor of the crash.
Dodi Al Fayed, the driver Henri Paul, and her
bodyguard Trevor Rees-Jones. Trevor ReesJones was the only survivor of the crash.
Step 3.2.1.4 Reason for the Incorrectness
●
Step 4a: In step 4, we need to label the response on what grounding does it make it
incorrect based on all the false claims found. One or more labels can be used on this
part. After fact-checking the whole response, select the following reasons if it was found
incorrect:
○
○
○
Core fact incorrect - if the main answer to the query is false
Additional facts incorrect - if the additional information to the query or topic is
false
Not timely - if the response contains outdated information
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f
○
○
Not relevant - if the response has claims with information not related to the main
topic
Not specific - if the response has provided relevant information but does not
specifically answer the query.
Step 3.2.1.5 Link to source of information
Step 5a: For each claim marked as Correct or Incorrect, include the main source link used to
support your decision.
©2023 e2f, inc. All Rights Reserved
Uncontrolled when distributed or printed
e2f