>> Eric Horvitz: Okay. We're honored today to have Joel Robertson, founder and CEO
of Robertson Technologies. Joel is a scientist and what is seen as biosocial entrepreneur.
That's a nice phrase. He's developed quite a few healthcare programs using computing as
its foundation.
Robertson Technologies is dedicated to making -- having impact on healthcare around the
world, and it's a -- I guess it's a network of nonprofit and for-profit organizations with a
kind of a global perspective.
Joel is the -- also the visionary behind NxOpinion, written N-X Opinion, a healthcare
platform that is based on probabilistic methods for doing diagnosis and recommendation.
That application over the years has been known as an application built on Microsoft
technology that's been featured in a variety of ways, and it earned Robertson a 2004
Computerworld Honors Award nomination.
And I always like bios that end like this. I think it's really nice. Dr. Robertson founded
his organization on the principles of compassion, respect, and hope. Three nice words.
So today Joel will talk a bit about NxOpinion: A Novel Integrated and Predictive
Solution for Global Healthcare Delivery.
Joel.
>> Joel Robertson: Thanks, Eric. The first thing that I'll do is apologize because when
you take an hour and a half and you have to talk about vision you have to talk about
science and you have to talk about applications. You can't go deep into all of them.
And so for those of you who want more information, more than willing to share with you,
so we're going to try to do that in an hour and a half, talk about all of those various
aspects of NxOpinion, so that you can see this is a product that has gone from R&D and
definitely we usually say you can see David Heckerman all over it.
David has been actively involved from the beginning in helping us to design the
mathematics, which also go back to Eric Horvitz' and David's philosophies that have
come into this application.
And it's very definitely has a Microsoft stamp on it because of that aspect of those guys
and their research and thinking.
They're the minds behind it. We kind of put it into making it work in the real world.
Robertson Technologies is a -- we refer to it as a Robin Hood theory of medicine. You
make money where there's money and you give it to the poor. And so we have profits
and nonprofits, 501(c)(3) public charities. We work with various organizations such as
the Bill & Melinda Gates Foundation, Intel World Ahead, Johnson & Johnson, Project
Unite, a lot of those nonprofits.
As well as in the profit world, you're going to see this fall a launch out of UPG with the
OneApp. We'll be on the OneApp platform most notably towards South Africa and
India.
So that the whole theory is how can we make a significant difference in world medicine
is what we look at. And it comes through partnering. And partnering with people such
as Microsoft in order to be able to deliver a solution.
There is -- all of us, when you look at it in the world of the developed country, there is an
ethical responsibility for us to care for the poor. In 2002 is when, you know, I was kind
of setting there and had a successful publishing career in behavioral medicine area and
began to look at, you know, what about the rest of the world.
And I've done enough international traveling to say we do have a responsibility for us to
look at the poor around the world and especially if the solution lies within technology,
which we do believe, at looking at how can we get there.
In India alone there's 512 million who will never see a physician. And I'll say very
simply: Should everyone see a physician? Yes. But the reality is it can't happen.
So how can you take medicine, safe with clinical supervision, to the poor without having
quacks delivering the care but having good, solid medicine and getting good metrics out
of it? The only way you can do it is through technology.
And so that was our solution. That's what we looked at. And we said we have to have it
innovative.
One of the most important things about a solution for the poor is it has to be more
sophisticated than a solution for the wealthy.
When you take people who don't know enough to ask the question, you have to be smart
enough to ask the question that they don't know to ask.
So that was one of our challenges, to say how do you get software that's that smart that it
can get into people who are untrained and yet get a right answer.
So in 2002 we started NxOpinion. And we started it in a nonprofit environment for two
reasons: Another guy and I invested in it and we didn't know if we could do it. In fact,
we had doubts, so we at least wanted a tax write-off.
But we had three questions, you know: Could we do it, could technology do it, and does
anyone want it.
And so we did those three questions in a nonprofit, and when we came up with the
answer to that solution, then we kind of moved into an area that said let's add the profit to
be able to move on.
So could it be done. One of the things that was very important is there had been a lot of
programs out there, QMR, a lot of programs that had been out there in the area of clinical
decision support and things of that nature. But one of the first things that we had to do is
we said is there the ability to be able to be smart enough again, if you don't know a
question, to be able to get to the answer.
And let me use an example of how this works, and I give you a big picture how it works,
and we'll take it from there.
Let's say you're a 19-year-old female. It's February. You're in Banaganapalli, Andhra
Pradesh, India, and you have a cough. And you have a health worker that that's all they
know. This software has to be smart enough to know what are the most likely diseases at
that time of the month in that location that are going to show up with a cough and begin
to ask the question to differentiate between the top diseases.
Then it has to give you a confidence level that you have that disease. And then if it's a
disease, one you're able to treat because you've been permitted to treat it, such as malaria
may be able to be treated by a rural wealth worker, and you have a confidence level, then
you get a green light to move to the treatment. And that treatment might be chloroquine.
Next what's important is a few days later you ask the person are they taking their
medication and not are they getting better but are the chills better, are the intermittent
fevers, because you're looking at the symptoms, so you have to have a findings-based
diagnostics, and then you have to have a follow-up based reverse thinking, asking
findings as to whether they're clearing up.
That's the general gist of what this is all about.
So we found that we needed multiple diagnostic engines. If we started just with Bayes,
for example, which is probabilistic, you'd run down a lot of bunny trails.
We also found that there wasn't a database that existed to support that kind of data.
There's millions and millions of relationships that have to be in there, relationships that
are regionalized, what is -- exists in the U.S. is not at all relevant to something that might
exist in South Africa or Rwanda, is might be very different than the mountains of
Uttaranchal in India.
So you really had to have a database. And there wasn't anything there. So we said, well,
I guess we have to develop it. And that was fun.
When we began to test it, we tested it through a lot of different dynamics, and it's been
well tested. It's in the commercialization phase.
And I think Bill Crounse here, director of worldwide health, in a recent program that I did
with him said: The result of this work, NxOpinion, is one of the most significant,
accurate, responsive and intuitive diagnostic programs I've seen.
And that's pretty much what we get.
Could technology support it in 2004? Steve Ballmer said that we cut our costs by 90
percent, our development time by 58 percent, and we need like 35 to 40 percent as much
code. So better solutions delivered quickly, more maintainable, supportable, in a very,
very mission-critical environment.
That is essentially what we see ourselves. Our space is what Steve Ballmer had talked
about. That's what we do best. We partner with people that do other stuff better because
the solution on such a vast problem has to be a working and cooperation between various
organizations.
Good technology. We had lots of questions. You know, what can you do. There's
connectivity issues and, you know, you've got all kinds of mobile platforms and how do
you get data in on a mobile platform and how do you make it be able to integrate all the
way up through a hospital HIS system, all communicate together.
And so what we decided is that we developed what's called a platform. It's a database
management platform. Multiple ways of getting data in, multiple ways of getting data
out, all using an integrated system.
Needs to be modular. Modular, for example, a level-one worker is somebody who
doesn't know much. So why ask them the same questions you're going to ask a specialist
physician.
So modular in that when you publish that out to a phone only the questions that they can
ask are asked of them for usability issue. And if they don't have labs, don't ask them
about labs. But when they get up into a PHC, a primary health center, and they may have
the availability of labs, what labs are available and what can we put the data in to enhance
the diagnostics.
And more importantly could we tell them what would be important to help them
differentiate the diagnosis.
So we really had to have a multiple platform. And I think the biggest challenge it has to
be agnostic. There are so many databases out there that we have to be able to read to,
pull from, and write back to those platforms because those databases, if we're going to do
global medicine, people already have various -- you know, Africa is strong in OpenMRS.
We've got a lot of different systems.
But if we want to look at global medicine, we have to be able to agnostically look and be
able to jump into the health vaults, jump into the open MRSs in many different areas and
to be able to read in and out.
Question: Would anyone want it? What happened when we finished our tests and we
went through various tests around the world, 90 different test sites in 14 different
countries. And basically it came back and our conclusion at this particular point isn't
does anyone want it but how in the world do we partner with people to support the
enthusiasm of what has been developed.
One of the examples that we talk about, does it work for them. And we recently just got
an article where the physician has diagnosed in a rural village in India that the -- I mean,
the multipurpose health worker diagnosed indigestion, and they ran it through the
NxOpinion. It came back and said it looks like it could be an MI. And they referred the
person up and it was an MI.
That's an example of the type of stuff that that patient would not have been treated
because there was not the information available.
So we have thousands of ongoing clinical encounters.
So one of the questions comes up, how did we do it? And so because we're here with
Microsoft Research, we want to talk a little bit about what's the mathematics logic behind
it as well as our vision for the use of it.
So the first thing is we had to develop an engine. And as I said, we developed four
engines. One bombed. And so we ended up with three that work. And they all work
simultaneously in one diagnostics. So it's rather interesting in how we were able to do
that.
But the key thing is it needed to mimic clinical reasoning. It needed to think like a
doctor. And it needed to be predictive. And then the database had to be modular, it had
to be global, and it had to be regionalized.
In order to do this, if you want to do it on a world basis, you have to regionalize.
Regionalize includes language, but it includes terms. There are four different terms for
diarrhea in the state of Andhra Pradesh alone in India.
So how do you do that? What are the terms that they refer to? Because information in -or diagnostics out is only as good as information in.
And then most importantly taking a complex, extremely complex program, needs to be
user friendly.
So we started out with Bayes. And I think that's why you hear terms such as Heckerman
and Horvitz, because when you think Bayes, that's who you think of.
So we started out with Bayes and we said I like this concept. It says Bayes' theorem can
then be understood as specifying how an ideally rational person responds to evidence.
Now, we're making an assumption that physicians are ideally rational, which is stretch
sometimes. But if we use that concept, we say how do we do that.
So it really comes back, our first thing in saying how do we take medical facts, put them
in a clinically weighted. You need to know how important is that symptom to that
disease and how important is that symptom related to that. How do we do that.
So we have a probability factor, and we went through a lot of discussions with David.
We talked about sensitivity, specificity, positive predictive values, negative, and we came
up with something that said there is a clinical predictive value.
And so what we use, and I'll talk about it later, is what we call the CPV. Very important.
A clinically predictive value.
Prevalence data, which you also need in order to run Bayes. You're going it's scattered,
it's spotty, it's slanted. You know, you may get data on a hospital in the U.S., but where's
the relevance elsewhere.
And it's really looking at what is the clinical presentation probability. And so we use this
clinically weighted prevalence. And what it really says is clinicians agree with it, you
can rapidly regionalize it, and most importantly at this particular point in science, that
data is subjective, until you get enough data back to turn it into empirical.
And so it needs to be a learning process that says this is what we believe to be the
clinically weighted prevalence. But in time we'll validate that or change it. So this is our
best guess. That's what you have to look at it, because that's where we're at in this field.
Sensitivity. You know, is this a test, is it a symptom, is it a sign. So we went into the
clinical data descriptive aspect and we said, you know, physicians typically think in terms
of I often see this, I rarely see it, or, you know, in our experience.
So we had to take this clinically semantic approach and do what we called a CPV, which
is a clinical predictive value. And that really means how much does X evoke a suspicion
of Y.
So now you've got those two terms that we use, CPV and CWP. And we developed
what's called buckets. And this, again, we tried to go through, and there is no specific
numbers that exist. So what are the buckets that exist.
So with the CWP you'd say is it a very rare? Is it a rare? Is it common? Very common?
You can get consensus on that.
CPV. You can get consensus on this is rare, it's often, it's typically associated with it.
The strength of using the bucket is that you've got a broad clinical agreement and you can
also regionalize fairly rapidly. You can say these are the top ten diseases that we see in
rural medicine globally, but here in Sub-Saharan Africa would you add some to that
bucket or would you take them away.
And then the same thing you can do with findings. These findings are often associated
with these diseases. Would you add something to that bucket or take it away.
The next thing that was very important was leaks. One of the things that I always
struggled with clinical decision support systems is you have to know what to put in. It's
almost like you need to know the answer before you put the data in.
And what if we were going to take data, for example, an electronic health record, and we
want to incorporate all the data into the diagnostics? Everything that's there. Your
disease-to-disease relationships, your preexisting diseases, whether you have a village
that has mosquito-infested malaria diseases or whether you have bad water or whether
you have previously had an MI.
All of that's important in a physician's mind, but they can run -- make Bayes run on a
bunny trail. If I'm putting in that I have a headache, I'm dizzy, and I have a sore foot,
there's three facts. But two of them are related and one is not.
So the challenge of how do you get irrelevant data not to mess you up and take you down
a bunny trail if the assumption is we need to make this work with nontrained individuals
who don't know the answer as to what's relevant.
And that includes our consumer. That includes sometimes a physician who's not
specialized because a disease is specialized. And it often occurs in very rare diseases that
are not seen very frequently. The physician may be the one who says I don't know
enough to ask that question.
In order to do that, we had to create what's called a multiple engine. And the first thing
when you enter in data in this particular system is it uses a rules-based engine on
templates.
What it is is we are gathering up and building prior probabilities. Because at that
particular point by asking questions directly to get prior probabilities prior to Bayes
kicking in, you have taken related symptoms and build up enough that Bayes can then
start into prediction. But if you did not do that, you'd have a multiplicity of data and
never be able to get Bayes to work.
So we created chief complaint templates that are related things. So if you say cough, you
may want to know is there sputum production, you may want to know whether that's
cardiovascular oriented, or you'd want to know, you know, whether it's respiratory
related. So you ask a few questions to get the related probabilities together and then let
Bayes kick in.
In addition to that, Bayes is a great probabilistic engine, but sometimes there are possible
diseases that Bayes will miss, so we ran simultaneously an occurrence-based engine, so
that when you see our software, it will say this is probably what's in existence, but you
can kick in and say what is possible.
And then Bayes can take a possible disease and make it into a probable disease with a
specific finding. So you can go to possible, click on it, and then enter in the findings that
make it into a probable disease.
So that was the creative thing. So what we had to do is we had to create suggestions. We
call it one more question. It needs to look through, and when you get to a certain point in
Bayes that you've got five, or in our case we use ten top diseases, let's ask the questions
of you that will differentiate those top ten diseases. Because you may not know enough
to ask it. So we'll ask that.
In clinical use it's rather fascinating because you can put a finding in. Let's say you can
put a result of a CAT scan in and see what it changed in your probability before you spent
the money on the CAT scan. So as you can see, there's a lot of uses for this sort of thing.
Let me describe something as you go through. I want you to connect this. We feel what
we've built is a combustion engine that can be used in a weed whacker or it can be used
in a Ferrari.
And so the application of where it is used is varied upon the user, where they're looking
at. Are they looking at it from a healthcare cost containment predictive, are you looking
at a diagnostic, are you looking at a rural health in order to be able to do that.
So we developed this platform. This is a way that we did some early engine testing
where we just would put in probabilities and we look at diseases and we put it in front of
expert witnesses and we'd -- expert witnesses -- expert panels and we put it in front of -run use cases that may be in literature or something of that nature to see if we were on the
right track.
The database then became rather interesting. Because now you're talking about when
you add lab and when you add imaging and you add findings and you add various
locations and extraordinarily complex database.
So when you look at the data, you can start out and you can say in general textbooks I can
find some relations. And if you look at most of the clinical decision support, you'd be
able to see them, you know, connect in with Griffith's 5-minute clinical diagnosis, things
like that. It's nothing more than these symptoms follow with this. There's no prevalence,
there's no sensitivity, but there's a relationship.
And then the next thing that you do is you have to go through and you have to author
certain specialty texts. And in our particular case, because we felt that we were on the
road of building something rather interesting, every single finding has to be documented
of where that finding came from and what piece of literature, and you have to have a
standard on what literature is legitimate and what is not.
So all of that comes with our -- what we call our content management team.
So then you go general, then the next level you go specialty text, and then the next thing
you have to do is go to literature review. And there's where you have some real standards
of saying what is legitimate information, what is a legitimate study, and then the criteria
before that data can be considered a positive relational and what we call published data.
Then we go through medical expert reviews. And these reviews, what we did was
originally on a global aspect just what are the disease relationships, what are the finding
relationships that goes through these internists, specialists, and then you go next to the
international.
So let's say that there are approximately 850 diseases that you may look at outside of
psychiatric and orthopedics. And so you'd look at that. And then there are, for example,
212 diseases that are seen frequently in rural medicine. And so you have a global what
we call rural medicine diseases of 212. Then you might go to India and take that
specialist and they'll say these are the 212 within India. And then you might go to Africa
and say these are the ones that fall within there and add and subtract to it.
So you're taking this data and not only on the relationship that exists with the original
diagnosis but the relationship that exists within that region.
So we developed, then, what we call the NxOpinion knowledge management platform.
And early on we met with the team from Microsoft from content management and some
technology people to describe our content because we said can we buy a content
management system.
And the conclusion back in 2004 or '5, I believe it was, was there isn't anything that exists
that can do this. So we had to build it on our own. And with a lot of, again, support from
Microsoft.
So we developed these things that are called a rapid regionalization. And we have a
finding name, a regionalized name, a disease name, and then various ways that we can
very quickly go in and regionalize a disease.
After we got the clinical decision support and we began to do some studies on the
accuracy and efficiency and the quality of the diagnostics, we found that there are a
couple of things that were very important.
One of them is that we found that a clinical decision support without certain
infrastructure and electronic health record storage, data, the ability to do metrics on it was
somewhat -- there wasn't a purpose for it. It was great technology, but as Eric and I were
talking prior, technology putting it into the rubber meets the road and making it work, we
found that we required that every encounter that a person had has to build on a health
record.
If you see a person and, for example, you see a pregnant woman and you put her in the
database and she's eight months pregnant, you have two people in the database. You
have a woman who's pregnant and you have a minus-one-month child who's going to be
needing to be asked were they born alive or not in a month.
So you need to have the data that says every encounter creates it, and you have to be able
to monitor the metrics of that. So we included a suite. So we kind of look at the
platform. And you're going to see when I describe the platform a diagnostic engine, an
event engine, a search engine, but you're also going to have an electronic health record
and a reporting in metrics that are all associated in this suite that is a platform.
How you access that could be through, for example, our Microsoft UPG OneApp, might
be a PC, it might be Web based, whatever it is you access this. Where it's stored could be
proprietary. And we'll talk about some of the challenges that come in working globally.
So NxOpinion version 1, the database took us four years and more than $6 million just to
put the numbers in.
And if you look at that, that's actually a very, very efficient number, for those of you who
have been involved in this field. And we did it by using a very, very skilled group of
people. And because we are a small company, low overhead, you're very agile. And so
that was what we did.
The next thing we had to do was look at clinical UIs. And Microsoft actually sent a team
with us to the Dominican Republic when we first started, and this was kind of the first
time I'd gone from the theory into the practicality.
And we'd take the NxOpinion and put it in front of doctors, and no one would tell them
how to use it. And Microsoft team would watch and see where they got stuck.
What I can tell you today is it takes less than an hour for a person to really use this
software. Now, there's a lot of features that will take longer, you know, metrics and stuff,
but it's a very, very user friendly.
As said, we had become one of the top five finalists in the world in 2004 as the best
medical solutions as nominated through the Computerworld awards.
We've done some academic and clinical studies. I just put a couple of them there. We
did Improvement of Diagnostic Performance of Indian General Practitioners. We found
that general practitioners increase their diagnostic accuracy between 25 and 41 percent as
compared to the specialist physician diagnosis. And that was in GI diseases.
We of course ran it against some of the powerhouse, and we'll show you some of those
studies. But we also ran it with who can use it so that you looked at use case of can
[inaudible] use it, what can [inaudible] do, what are the limits of these people. Can some
of them only gather data. Can some of them make diagnostics. What are the
multipurpose health workers come in. Did you improve not only their data collection and
integrity of the data, did you improve their diagnostic skills.
So all of those tests have been done, and I won't bore you with those. But here's an
example, for example, of side-by-side comparison of some of the diagnostic programs.
And I'd like you to look at the line 4 where NxOpinion's about a 94 percent, and you take
some of these like Epocrates and Isabel and they're around 83 to 84.
But one of the most important things is that's if they put the data in, and you have to
know the data to put in. If you spell it wrong, it doesn't get in. If you don't know what to
enter, it doesn't get in.
So even though you put all the data in side-by-side were more accurate. What's more
important is because we ask for data from an end user, most of those programs will be
around 60 percent accurate, where we'll still maintain our 90 percent or so, because we're
asking questions, where those aren't.
And you can put in multiple system symptoms. You could put in cephalgia and headache
in most of them and get two facts. We will not. The way that we've develop it is that's
one fact.
We did some pilot studies. Well, many pilot studies. Dominican Republic, one of the
things that we did is just threw it into a college student to see if they could gather
information. The quote/unquote back was this was easier than falling out of bed.
That was very important for us to be able to say can data collection, which is very
important -- because, you know, in global medicine, we have to know who exists, you
know. And getting data in, very easily done.
In Congo we worked at it with various teaching hospitals and in Uttaranchal and various
places, Andhra Pradesh, done a lot of the pilot studies initially to see if we were on.
Looking at the current features. One of the things that I think is very important is that
NxOpinion has a personal health record. Now, I'm going to use the term differently,
personal health record, than most people.
And I come from a behavioral medicine background, and so from that standpoint I'm
going to make a statement that comes from that background.
And I say, you know, personal health records, when you think of us as consumers, where
you're getting data in, you're typing data in, I make the comment sometimes that says,
you know, that data is probably not going to be entered unless you have a little bit of an
OCD in you. If you have a little bit of obsessive compulsive disorder, you're going to
keep your health record. Because why am I going to keep my health record?
We use the phrase I will keep a health record if I can interact in it, if I can ask good
questions.
You take the technology of NxOpinion, you could go into your health record with more
technology. But I'm saying the engine does this. I could say can I drink Red Bull, and it
will go in and say you have hypertension, you're on this antihypertensive, it is not
indicated -- it is contraindicated. No, you can't. Are you looking for energy?
This software right now with the supporting systems can do that. That is a health record
to me. That's an interactive personal health record. That is something that will be used
universally, because I now have a reason to have that and interact with that.
So we look at it and we say we needed to be able to access this health record from a
cellular device. Our first cellular device, of course, has come through Microsoft UPG in
the OneApp, and it is very, very cool. And that platform what it can do is really slick.
And so that's something that will sometime around November be released.
Personal PCs, obviously. But it also needs to be able to be incorporated into the public
health and into the hospital information systems. We have an HIS dashboard that I'll
show you that uses all of this sort of stuff.
Your health record also has to have vital data registration and integration. To give you an
example of how simple it is, let's say using the UPG phone OneApp, if you were data
collecting and you went into a village, the first thing it might ask the rural health worker
is a village questionnaire. You know, might be asking questions about water quality, it
might be asking questions about airborne diseases. And that questionnaire is filled out on
that phone one time.
The next time you go and you enter in a patient and create a patient record, all that data is
entered into their health record because that's important in making diagnoses.
So the ability to be able to integrate. The ability to be able to go in and say if I have a
mother with a history of diabetes and I create a son, that son has a family history of
diabetes, I don't need to enter that in. So that we're looking at being able to get in so the
data entry is so easy that we're going to get good data and vital statistics.
The other thing important with this clinical decision support is it needs to be used with
skilled or nonskilled user interfaces. We use three different levels. Level one is a person
who really can only ask symptoms and only in a limited basis on a handful of diseases.
And there are five major diseases, for example, that create a significant number of deaths
around the world, and there are ways to be able to get a fairly high confidence level with
symptoms. They might be a level 1.
Level 2 says I have some rudimentary laboratory and imaging. I might have a chest x-ray
and I have a few labs available. And I can ask those next-level questions. That might be
an accredited nurse, midwife, or it might be a multipurpose health worker or something
of that nature.
And you need to be able to plug in various devices, blood pressure monitors, et cetera, so
that you take away the training of the individual in various organizations or companies
have those.
And then you have level 3, which is highly sophisticated. I've got a specialist, I've got a
lot of information. So each interface and each question has to be very different. Every
template has to be different.
For example, with the lowest level, you go through a chief complaint. The next level you
might go I can go to disease and I can start entering my findings very quickly and then
find out my predictive diagnostic. Various ways you can go in when you get up to the
higher levels.
You need to be able to customize management and treatment for each of those levels.
And when you take a rural health worker and they can't -- they're away from a hospital
and they can't have the access to the medication, then you have to give them various ways
of stabilizing disease and transferring.
We believe what we have provided is something that is extremely important. You now
have a documented clinical supervision of rural health workers. Every event is
documented. Their decision is documented. The findings that put in, the treatment they
put in, the follow-up to their treatment is all documented electronically now in a very
user-friendly fashion.
And so now you to have for the first time physicians can look at these people and
statistically find out in very easy reporting systems do they need more training, are they
treating their diseases, are the diseases being treated properly, is there a resistance
occurring.
And all of that on data that right now is not captured. If it is, it's written down, but it's not
able to be done on the fly. And it provides a standard of care.
The monitoring and evaluation piece is very interesting because even on the mobile
phone one of the really -- again, I keep talking about OneApp -- is that we're able to
customize just our ability to monitor and evaluate. And it can be done on the fly with the
forms and immediately uploaded into the phone. Just very simple. So you can do a
monitoring and evaluation.
One of the cool things is let's say you're in a village in this particular organization,
perhaps like an organization like World Vision wants to know something as simple as is
there electricity. You can add that to the form and gather that data and immediately get
other information. And then it comes in and NxOpinion knowledge management can run
stats. Obviously that data is important in diagnostics, necessarily, but it does allow us to
do some monitoring and evaluation.
We can monitor and evaluate not only follow-up with medications, but pregnancies. You
can through, for example, the OneApp. When a child is born you can send out a note that
says to the health worker see if the child is able to nurse. Is the child drinking water.
You can ask these questions and if all the answers are hitting the standard that is
appropriate, you got a green light and it just shows up green: Keep doing what you're
doing. It shows up yellow, then what happens is immediately an alert is sent to a -- the
next higher level supervisor that says we need intervention in this area or we need an
immunization or whatever it is. All of that is programmable very easily on this system.
One of the things that we found very early on is the diagnosis accuracy -- not only in U.S.
medicine, there's various studies that say there's 7 to 20 percent misdiagnoses, but in the
rural medicine is people make diagnoses based upon their level of knowledge. And so
they allowed and make a diagnosis and it's considered a diagnosis and all of our public
health systems count that as a diagnosis.
What we've done is we said we need to be more granular. We need to go down to the
finding. We need to gather the findings because the grouping of findings may validate a
diagnosis or it may validate that this is not a diagnosis. This is not the appropriate
diagnosis because they didn't have enough information to make that diagnosis.
So now I think that we can do better early outbreak. We can tolerate misdiagnoses. We
can look at various reporting functions because technology has now allowed us to get so
granular and run these reports.
One of the things using the architecture that we have is it provides a common medical
data architecture. It doesn't matter how the data is stored. It allows us to be able to pull it
in, interpret it, and dump it back into your system. Very important from that standpoint.
It allows the whole broad data accessibility, whether you want to pick it up on a phone,
whether that person goes through multiple levels of treatment, their records are still
available on every event in which they were seen and can be reported back.
We also believe in a transparency of data analysis. Our job is to be able to get data in.
Our job is to share that data in the public health system, to allow other people to be able
to go in and figure out what solutions exist. And so we are very much in the public
health data is so important to be available to all and be transparent.
The regionalization of NxOpinion was one of the greatest challenges we had. And so I
think that what we looked at is we took global and regional. And what I -- I refer to it as
what I call a fallback system.
India, for example, has some diseases that we can overall say these diseases exist or
Sub-Saharan Africa, but then India may be broken down into regions that have winters -or mountains with winters, mountains without winters, coastal and plains.
So as we gather data and as we learn, we can go more granular on our prevalence and
sensitivities, but it always falls back to what we know. So that that data is not yet there, it
goes back to what is known. And then as we learn, we can add the data in.
And so it uses that ability to regionalize. What that means is, for example, the creative
group at UPG, we can do a data collection in a new region in a matter of hours. Takes
hours to be able to take it into another area. To be able to do the diagnostics and reports
is a matter of just days to be able to convert it regionally.
Now, to do the full diagnostic in a specialty language takes a lot longer. But we have the
ability to say that this finding shows up in this fashion, and you can put it into a different
language. The database doesn't care the language; it cares that that finding get filled into
that field.
So it's very interesting. And if you use voice-directed technology, not voice recognition,
you can ask questions in a native tongue in a very fast method using NxOpinion's clinical
decision support system.
One of the other challenges that you have in global medicine is publish set. Everybody
wants to own their data and nobody wants to share with anyone else. Not only do
countries do that, but organizations do that. And sometimes even within organizations
they won't share from one department to the other.
So one of the things that we decided to do was to create what's called published set. You
can go into our database and create a subset of data that says for this organization for this
location, this is the data that is published in as your database. We keep a mirrored
D-identified image.
They can own that data. They can use it for financial support. They can do whatever
they want with it. We with the mirrored data can run all of the statistics, all of the public
health information and can pinpoint it back to an individual. They can connect it to the
individual.
So by creating this we've allowed people to say we don't want your data, we don't want to
make the money off of the database; we want a global database that is what is necessary
to really do public health.
So they own it. They control the data. We manage the data in a D-identified.
As I talked about before, regionalization has a user skill level and it has a structure of
being able to fall back.
Let's talk about NxOpinion as a platform. I've given you a little technology. Let's start
moving into what do you do with this.
So as a platform we look at four different areas. There's -- yes, sir.
>>: What's the scalability of what you've built?
>> Joel Robertson: That's exactly where we're going. Exactly. Because that is a huge
challenge, scalability. And that's exactly where we're going.
>>: I'd like to know what are some of the interfaces to organization to prevent, and the
third part is what about -- I mean, you mentioned briefly that if someone's collecting
imaging data, that that can prove a whole set of [inaudible] ->> Joel Robertson: That's right.
>>: Right? Are you looking just at x-rays? Are you looking at just being able to take
visual example with the cell phone camera or being able to do ultrasound or ->> Joel Robertson: Yeah. There's -- again, based upon the interface and the level of the
program being used, cell phone or PC. For example, if you're looking at a cell phone and
you have an x-ray finding, you'll only have the result showing. If you go into our HIS
system, you're going to see the x-ray come up, and you're going to see what that looks
like. So it goes through.
And you're absolutely right. With the cell phone, you frequently can take a picture, get
that data. There are some -- some areas where they're beginning to look at automatic
diagnostics through the imaging. But it can always go back to the PC -- the PHC or
higher and get it and put it back in. So you're able to transmit.
And the OneApp is really cool. That's one of the things we liked, is we added imaging to
that piece that they had not at UPG previously -- I mean, picture imaging, for that various
reason. Dermatological. It's a great way to do that in rural areas.
So -- and I'll show you, for example, in our HIS system you hover over a finding, it will
pull up an x-ray right in front of you or whatever imaging you need.
But scalability is a huge issue. Do we have an answer to all the problems? The answer is
no. And one of them is, has to do with why I'm here, why I'm interested to be here, is
there's still problems to be solved, and that needs to be solved by some people brighter
than I am in those areas.
And so what we're looking at is saying this is our domain knowledge, but there are other
domain knowledges that have to build in this if we are going to change world medicine.
So we aren't the solution; we are part of a solution.
So like you said here, we've got a whole data collection system that you have to talk
about. Data interpretation, data storage and data reporting.
Here is the platform that we have built. I want you to think of this. You've got this
global databases of all the aggregated data. You've got a personal health record which
whoever wants to own the her. The platform that says the diagnostic engine, event
engine and search engine, that's what we do.
The diagnostic engine is I'm going to help to diagnose a disease. An event engine is I'm
going to take such things as follow ups, I'm going to ask for alerts, I'm going to ask for
various events. And the search engine has the ability to be able to go in and get the data
all the way down into granular basis.
Out of that we have the monitoring and evaluation. Messaging engine, UPG, for
example, with the OneApp, is a messaging engine. That's using that. There may be an
SMS methodology that may come out of there. There maybe various aspects that come
in.
We look at ourselves as a platform. We say other people build. You can build some
interfaces on us. We've got this piece put together that is a huge effort. And how it's
built and how it works in. Monitoring and evaluation is another area we're weak at. We
need lots of people to be involved in that. Epidemiological studies.
So I think the whole ability to dump into this system and to use this system to come back
out, there are some aspects.
>>: So if I understand your vision correctly, it's almost like you're making a case for
publishing KPIs to your [inaudible] ->> Joel Robertson: Absolutely.
>>: Being pretty open.
>> Joel Robertson: Absolutely.
>>: [inaudible] business as well as a technical problem. But essentially you're talking
about an open source.
>> Joel Robertson: I'm talking ->>: [inaudible]
>> Joel Robertson: -- you're absolutely right. Now, it's very interesting because what's
fascinating is -- like I said, we have a 501(c)(3) and we have a business model. And
essentially what we're saying is, you know, we're not going to give away our code to our
engine, but we are really building on the open source concept. We are really building on
saying, you know, this is what we want to do and we're going to do this. We're going to
give you a lot of APIs to be able to commit.
We have to do that. This -- you know, that's why we kind of used open MRS as our first
mapping.
Our NxKM database is actually -- can do more and has got a lot of better things. But you
know what? It makes it proprietary. But if you got open MRS, you can come in and do
that sort of stuff and get to us.
So, you know, we really are looking at that aspect. So you're absolutely right that we felt,
again, when Eric described my mission, my mission is not to build a proprietary database
diagnostic system; it's to build a platform that other people can use because unless we all
work together, unless it is open to people who are brighter than I am and our company is
in other areas, the ultimate solution is saving lives. And that's the only way you can do it.
>>: Do you have an STK, then?
>> Joel Robertson: Not yet. We aren't at that point. But you're absolutely -- that's our
next -- we are close. December maybe. That's where we're at right now.
This is just the Cojax. This is the OneApp application which is out of the UPG. And it's
just various ways of -- there's two elements to what we've built with UPG. One is called
a health manager, which is accessible by consumers to be able to go in on a cellular
platform, to be able to get regionalized diagnostic data, health tips, all of that, without
using the term -- we're kind of the UPG's WebMD of developing countries.
So that's like a November release. And we'll be looking in South Africa and India. And
that's our role. We've got 15 countries or so that we're geared towards. And that's the
consumer base.
>>: [inaudible]
>> Joel Robertson: We're running all the back end. Yes. So when you go to OneApp,
you'll see whatever the name is, but Robertson Health Manager or NxOpinion Health
Manager, et cetera. And then in the back end we're doing all the health tips that are
regionalized, we're doing the diagnostics, et cetera, storing the data.
The other part of that is data collection, which we look at with a lot of humanitarian
efforts. Gathering data at the HEW level, pulling data in using the cell phone. And so
those two components are both ready to come out of -- in November.
>>: [inaudible] who are your carrier partners?
>> Joel Robertson: What was that?
>>: Who are your carriers?
>>: Operators. Telecom. Telecom operators.
>> Joel Robertson: I'm not -[multiple people speaking at once]
>> Joel Robertson: Oh, yeah, yeah, yeah, yeah, yeah. You know, like I think Microsoft
has relationships with blue label telecom and Oxygen in India.
We also are going to be looking at independently we can work with companies such as
Tata in India and all of those things from a distribution standpoint. So company based,
corporate based, as well as we've got distributors.
We've got some meetings on humanitarian groups to release and use it also. Sorry.
This is the Microsoft OneApp, just kind of a -- this is a mid-level phone. And just kind
of the way that it might work out where you just can kind of go in and click a few buttons
and put a few things in and the data comes in.
One of the things that we thought was kind of an interesting challenge is in dealing with
rural India, for example, names, you know, are rather interesting because you have some
very difficult names to put in on a cell phone.
One of the simple solutions is you take a paper, write your name on, hold it up, snap a
picture, and a person that's on a keyboard can enter the data in. You've already got a
unique identifier with it.
So there's many different ways that you can find some practical solutions to be able to get
some of the problems.
>>: But what exactly [inaudible] one of the biggest problems for healthcare, particularly
in India, is that the government doesn't even know what their population count is. So
how do you make sure that everybody is uniquely identified?
>> Joel Robertson: What we have been able to do with our NxOpinion is to have an
identifier that is associated -- if it is a particular ASHA or health worker that has a phone,
that phone is connected to a region and to a village. And so what you have is a beginning
of a numerical solution to this is the village, this is the state, this is the region.
That numerical, which they don't have to enter because it's automatically done, and then
when you create the record you have an additional unique identifier. That unique
identifier can be mapped to a national ID or anything.
Because they don't have it, we create it and will map it to however they want to do it, so
that we have had to create the unique identifiers that can be mapped. You're absolutely
right.
>>: [inaudible]
>> Joel Robertson: Yeah. And we do work with the governments as far as seeing if -what's their -- do they have a plan or what is the plan, et cetera.
This is an example of data gathering. Now, this is just the pilot study. Now, don't think
of this as the user interface, but this is kind of how NxOpinion works.
On the left side is where you have patient data. On the right side is general reference. So
that means anytime I go in and do a search, where NxOpinion is very different than your
typical WebMD Mayo Clinic searches, is if I am on the right -- left side and I am a
52-year-old male with a history of diabetes, et cetera, and I go search the database, you
won't find me able to get diseases that are only common to females. You won't find me
finding diseases that are unique to pediatrics. It filters your findings based upon what is
existent in your health record.
On the right side is generic. I can go in and find out anything. So that's part of the search
process. You have some various ways of getting in epidemiologically. Then what we
have on even at the PHC is you have what's called quick picks.
On the right-hand side here, you'll see at the bottom you have all the different diseases,
and it's either do I have it or does my family have it. And it has a smart search. But on
the top are quick picks. Quick picks are what are most likely to exist in your area the
most common diseases.
And it's a user usability interface. So that what comes up is it's very much -- they can
stay up in that top area most of the time, click it in, and it's very much of a user. And
that -- usability issue. And that is part of our regionalization.
You can go through and all of the findings, whether there's symptoms, whether there's
labs, whether their imaging and all the various things, you not only have a description but
you have a link that they may have with other diseases and/or findings.
Now, data interpretation. I'm going to just show you a couple of screen shots of how
NxOpinion works. And, again, this is the research tool, not the user interface. On the
phone it works different, et cetera.
Here is a series of symptoms were entered, and at the bottom you'll see that there are 11
pieces of information and evidence.
And those 11 pieces, you know, being male or female, age, are pieces of evidence. And
various aspects, former history, whether you take -- whether you've had an immunization
or not is in evidence.
But all of a sudden we've run in some findings through a chief complaint and we have -whoops, I skipped a slide, didn't I? No. Well, we have a mis-slide.
What you'll see here, though, is I've got malaria and I have a confidence level. Now, with
the end user level 1 and 2, that's a dot: green, yellow, or red. We don't give them the
number. With a more sophisticated user, we may give the number. So that it comes this
way. A green disease says the group of physicians feel at your level you're qualified to
treat with the finding or the treatment that's here.
For a disease to reach green, it has to be you're qualified to treat, we're confident you
have it, then you get a green.
If you're qualified to treat, we're not confident you have it, you have a yellow, which says
you need more information. Over here on the right is what are the different information.
Those are suggestions that help us differentiate your disease.
If it is a yellow disease, meaning you aren't qualified to treat at your level but we are
confident, then you have a different statement: It's likely you have this but you need to
refer the person. A red disease says: There's no way, get him in for help immediately.
So we're able to use green, yellow, red not only on diagnostics but on alerts, on
follow-ups, on all of those various things.
>>: Did you say there were two kinds of yellows?
>> Joel Robertson: Yes. Yellow is I'm not qualified but I'm confident I have it. Or
I'm -- it's you're not qualified and I don't know if you have it.
>>: You are qualified.
>> Joel Robertson: Or you are qualified but I don't know that you have it. Both of those
would be yellows. Both would have a different statement. There is a qualifying
statement that says, you know, you probably have this disease, but you aren't qualified,
you need to have somebody treat it. That's a good telemedicine call type of disease.
This, going back to your imaging, is a dashboard that we created that integrates with any
HIS system. Now, we've created a clinical dashboard in which to put in perspective our
phrase that we use is doctors should be information interpreters, not information
gatherers. You need the data when you want it, where you want it, and how you want it.
So we have a dashboard where basically using this engine, if you are an infectious
disease doctor, there will be a template that comes out and it says most infectious disease
doctors would like to know platelet counts -- I mean WBC counts, temperature charts,
and whatever it is, this is what you want.
NxOpinion, when you go in and pull up that -- your patient, it will show you your graphs
of temperature, it will show you the medications, it will show you any alerts of anything.
This is a chest x-ray. And if you hover over it, it shows the chest x-ray. You don't have
to go get it. It shows up.
What's most important is let's say you prescribe a gentamicin, for example, for an
infection and we know that it has a high side effect of renal disease. NxOpinion in a
suggestion will say you might want to run a creatinine or a BUN.
So this is something that will be ready in January. But it goes back to that core
technology of different use. This is kind of a Ferrari where the OneApp is a weed
whacker. Although it's a pretty powerful weed whacker. So, again, using it from that
standpoint.
You'll also notice that in the corner or in the right-hand side -- and these are Widgets, so
you can move them around wherever you want them, you're going to see NxOpinion
setting there with the probability of the disease.
What we feel is it needs to be sitting in the background -- this is mostly a hospital
information system; this is at a secondary hospital or things of that nature -- is it should
be running in the background. It shouldn't be interfering.
But if all of a sudden you look and you say I've only got an 80 percent diagnosis, I'm
ready to go, you can go over there and it will say here's some ways to get you up to 90
percent.
It will also be able to go -- you could say I want to run a CAT scan or an MRI, and it will
give you which one is most likely to add to the diagnosis. And in our version 2, it not
only will say in that hospital how long will it take you to get a CAT scan run and an MRI,
but what are the different cost effects, or the different effects of that cost.
So you might be able to get something done in two days less expensive and it adds almost
the amount of diagnostics to it. So it has a little bit of workflow to it. That will be ready
in June of next year.
NxOpinion is a product. You know, I really look at NxOpinion, one, as a plug-in to HIS
systems. Once again, to steal a phrase from another company, BASF, "we don't do
anything ourselves; we make other people better," that's kind of what we see ourselves.
We look at, for example, there be Almalga HIS system and say, you know, we can make
you better. There may be another system out there, we can make you better.
Because what we say is this is our particular strength. This is what we do. And we want
to stay within that realm, but we want others to work with us.
We especially have spent a significant amount of time into the rural and remote
healthcare. Our passion is to take technology which is essential to take quality medicine
into the rural area.
And by using NxOpinion you have clinical supervision, you have standard of care, you
have diagnosis in the area of the nonphysician yet you have the supervision of that. You
have a treatment standard. You have follow-up, you know it's done, and you've
monitoring and evaluation and the ability to provide alerts.
Many of the challenges we that we've had -- my next slide is going to talk about the
challenges we haven't met -- but at this particular point.
It allows us to do telemedicine. Yes, sir.
>>: So are you just running one instance of this gigantic thing that everybody feeds into?
>> Joel Robertson: No, no.
>>: Or lots of instances? And what kind of business relationship do [inaudible]?
>> Joel Robertson: We are in the process of -- you know, you asked the question are we
running one instance, and the answer is in each of these pilot areas, the answer is yes.
The technology that we're doing now is moving that out of that. So that will be I think
October is coming out of that system.
We, again, philosophically believe the cloud is the answer. There are some hesitancies
with various organization groups, et cetera. So in that case you run instances on their
local server and then put it back into the cloud on a D-identified data.
So our business model, that's what we look at. That's also what we look to Microsoft to
say, okay, we've done this, help us with this problem. That is a problem that we come
and say we need you to answer that.
You know, one of the reasons why I'm really pleased to be here is there are problems that
are above our skills. That's one of them. What is the infrastructure to support this
realtime? And I think it's a great question.
So when we're ready to roll right now, we have to do it as we can at this particular point.
But if we're going to do a really worldwide global database, where does HealthVault fit,
where does Amalga fit with -- you know, and I'm going to go into that.
Where do we go with the infrastructure?
>>: So currently someone would just license the shrink wrap [inaudible].
>> Joel Robertson: Absolutely. Absolutely.
>>: And you do it as a nonprofit.
>> Joel Robertson: It depends on who it is. You know, for example, an area in which we
have gained a tremendous amount of demand of small hospitals in the U.S. for an HIS
system because they can't afford them. And it's very affordable. That comes through our
profit system.
India hospitals are very much liking us because we have a regionalized clinical decision
support. That's out of the profit.
On the other hand, if, you know -- sometimes even go government contract can be profit,
depending. But if you're setting there in a -- especially Sub-Saharan Africa or in areas
where there's just -- you know, certain states in India, Chhattisgarh, there's no money. So
then we run it through our nonprofit.
But it is our ability for us to be sustainable to say nonprofit, profit, various things of that
nature.
I believe strongly that we -- the next movement, I personally would like to see that
because I have a relationship pretty heavily for years with Microsoft to take something
such as HealthVault and make an interactive personal health record.
The technology exists. Google's playing with it with predictive. They've got one guy
running around doing some stuff. We are so far beyond that if we are ready at this
particular point for medicine to go to interactive personal health records, that is an
extraordinarily, I believe, important aspect.
Also case management resource utilization. We've been contacted by insurance
companies that are saying can you keep healthcare costs down. Well, obviously, if you
can pinpoint a specific test to be run to help you differentiate versus a battery of tests, that
helps. Various ways of dealing with that.
Here you go. Scaling. Big problem. This comes from a book that Kristin Tull
[phonetic], David Heckerman, Delda Hart [phonetic], a chapter that Kristin Tull, David
Heckerman, Delda Hart and myself wrote. And it's called "How Do You Scale
Healthcare." How do you take this and scale it.
Well, I'm going to give this to -- this phraseology to Kristin, because Kristin created this.
We call it scale up. How do you take medical databases that are hundreds of thousands
and turn it into millions. Problem No. 1.
I don't have a solution necessarily. I think you guys do.
Problem No. 2 is scale out. How do you regionalize to multiple country and regions with
language, regulations, organizations. We have a good solution there.
Scale across. Multiple diseases and programs, how do you coordinate HIV, you know,
diarrheal diseases, maternal care with the Gates Foundation, et cetera. We've got the
technology. It's a matter of those joint cooperations to put those programs in.
Scale in. One of the things that I think is extremely important about what we do is being
able to get down to a granular level. I can tell you having a great deal of experience in
rural health, if you can't scale in and you can't get down to that patient finding, you're just
making decisions on inaccurate data without any validity. We need to validate the data
we're making decisions on.
So when I look at that, I say one of the major problems that we see is this scaling up. The
scaling up. It's an infrastructure system that's far beyond our small company. It's an
infrastructure that I think ties in several companies and several solutions.
And that is one of the challenges. We can run this and we can run it to a point, and then
we will stall without that scaling up.
What I have here is a global database and some touch points and where we look at
Microsoft, for example, and we look at other partners and say where -- what can we do
and where do we need help.
You have this user and then you do have a method of data collection. OneApp, for
example, is a great data collection system. I think OneApp is fantastic. I love the way it
goes. It's limited from an SMS text standpoint. That means certain areas of the world
can't use OneApp. So the question is where does that technology come from that's going
to be necessary.
I can argue -- I can argue the OneApp platform as being a superior platform for data
collection. But if you don't have the infrastructure to use that mobile phone system, you
can argue all day that it's superior, but if it doesn't have the ability to work in that system,
then you need another solution.
So there's an area where we need a solution.
By using the API system that we have, many people can use data collection systems in
through our system. So we say, you know, we've created a foundation; let's get other
people to be able to do that.
We can build for them on OneApp. We can build for them, et cetera. But there are other
people that need to be able to look at that.
Data storage, I tell yeah, this is a nightmare for me. This is an absolute nightmare from
the standpoint of we can gather data. We can store data. We have to deal with the
regulatory and most important we have to deal with the global aspect of this. Has anyone
ever tried a database system that could potentially be this large and manage it.
So I take that one and I say you know what? I can brainstorm with you. I have no
solutions because you are far over my head on that one. I do not know how to do that.
And that is our weight limiting step at this particular point.
Data interpretation. You know, I look at Amalga. And I'm intrigued by that technology,
for example. And I think of what we do and then with that, the Amalga system, that
there's some ability to be doing even greater data interpretation. Especially because they
have similar languages, probably because we have a couple of guys that are pretty heavy
in that area.
Data feedback. I think not only gathering data but going back and reporting data back
and being able to use that, whether it's through the mobile OneApp platform or whether
it's through a HealthVault system.
We are in a position that I want what our fall release is is I want to give you a new term.
It's called push medicine. We believe in what's called push medicine. Pull medicine is -pull medicine is what people do now. They go to the Internet and they say I have
diabetes; I want to find out about diabetes. Or they go and they say my doctor said I have
this; I want to find out about it.
Push medicine says if I take the database that NxOpinion has plus one of our companies
called Robertson Wellness, which is another technology that we do, I can go in and say
because you're diabetic, obese, and male, here's a new piece of information you need to
read. I can push it to them. I can push information to them, only relevant information to
them.
To me, that is a whole new way of doing medicine. Pull medicine means I have to have
something that asks me to go dig it out, whether it's all of a sudden I got a lab test back, I
may be months late. Push medicine says I'm going to give you new information, pay
attention to it. Here's a new way of doing it, here's new research, here's something that
we know about your medication. It's a whole new way of doing it.
NxOpinion can do that. But it needs the data feedback system. OneApp we will be
doing that.
Data learning. Again, I kind of look at the Amalga system in saying ooh. I salivate with
taking the data that we have, our data learning, but Amalga's data learning, for example,
or some system like that can really help us change medicine.
And, again, the analytics that are provided from Amalga. You keep seeing -- hearing me
say Amalga. I think it's the future. It's a semantic database. We're a semantic database.
So there's a lot of fascinating things that I find.
So when I kind of look at this in conclusion, and then I'll kind of open it up for a little
discussion, you know, we don't have all of the answers. We have done a lion's share of
and met some of the solutions and some of the challenges in dealing with global
medicine.
We now are in the position of saying the solution needs to deal with some very, very
important infrastructure issues, with being able to work with other people who do things
specifically better.
But it is fascinating because from my perspective, and it's been with the help of
Microsoft, for the first time, you know, you take -- Eric, how long have you been dealing
with Bayes? I mean ->> Eric Horvitz: Since birth.
>> Joel Robertson: Since birth. So 25 years, huh?
>> Eric Horvitz: [inaudible]
>> Joel Robertson: I mean, you take -- you take guys like Eric and you take guys like
David that have just been digging into it, and it's kind of neat that -- you know, I feel that
we're very fortunate to be able to take those pioneers and take their stuff and put it into a
system that offers a solution.
But we are only a piece to the puzzle. And I think one of the things you'll find about our
attitude is we need help. We are not the solution. We are a piece to the puzzle. And
from that it's very interesting that we look at and say this is what we're not.
My father was very wise when he said to me know what you're good at and do it, and
then find people who are better at things and ask them to do it.
We need help in epidemiology. We need help in metrics. We need help in infrastructure.
We need help in analytics. We can't be everything. And for that we need to have good,
solid partners with research-valid information.
Any questions? Eric.
>> Eric Horvitz: [inaudible] catch up with David on the qualitative [inaudible].
>> Joel Robertson: CWP.
>> Eric Horvitz: It would be kind of nice to take for a set of diseases [inaudible] go to
the population and do some sort of qualitative or quantitative evaluation, whether it be an
actual set of runs with real patients or you have a proxy database coming in from its
reporting model in India or any place you want to -- New York City, and see how well
the system is actually working. And that would be a really nice slide 17 to have in your
presentation.
>> Joel Robertson: We have three studies that I did not put in because I didn't want to -I, you know, again, trying to get vision, trying to get technology, et cetera.
The first study that we did had to do with 300 patients in which we were looking at did
we make general practitioners better and were we accurate. So that was the basis of that
first study.
The second study was what level of skill -- and this was a Gates Foundation-funded
study, was what are the skill levels of the people who can even be involved in clinical
decision support and who can be involved in data collection.
And we were somewhat surprised that that is a major hurdle that some of the people that
are the front-line workers don't read -- either they're -- can't write their own language and
so therefore have a difficult time.
So voice directed helps. If you can ask a question in their native language and they can
answer yes or no. But that was a challenge. We have a partial solution.
The third thing was to take 2,000 multipurpose health workers and look at two things.
One is the collection skills of their present system versus does NxOpinion get what I refer
to as valid data. Because I'm a stickler that says when you make conclusions on data
that's subjective is a difficulty.
So we looked at that in just two pieces of data collection capabilities. And the other thing
is how did they rate with those 2,000 patients compared to the physician.
And that study has not yet been published, but it's complete, correct? That's been
complete.
Those were done using Nizam Institute of Medical Sciences as our supervising. We went
through all of the standards that are appropriate for India medical services and -- or
medical society and all the protocols so that it could be, quote/unquote, a publishable
article. And so from that standpoint we used all of those criteria on all three of them.
But there's a lot more stuff. I mean, it's just continuous. Continuous.
>>: Is the study on data collection that you did in India for the Gates Foundation, has
that been published?
>> Joel Robertson: They have not published it yet. We finished it in December. I have
not seen it published. It's something that's available. We can give you a summary of it, I
believe, without getting in trouble.
>>: [inaudible].
>> Joel Robertson: Because we were somewhat surprised. When we first -- you know,
of course, Microsoft owns -- I don't know if they own totally, but have a relationship with
Tellme. And Tellme being voice recognition and we said, you know, that's not a -- that's
a great application perhaps with HealthVault interactive personal health records, which is
phenomenal, but voice directed, using our rules-based -- what's really cool is using our
rules-based template to get predictors in, you can ask questions. You can set it up where
somebody just has to ask the question and you answer yes or no before Bayes kicks in.
Once Bayes kicks in, you have a series of questions that can be asked and answered. So
it's really adaptable way of being able to get to those people who are illiterate ->>: [inaudible] along the same lines.
>> Joel Robertson: Yeah. I've heard about that. And, you know, ->>: Is it only funded [inaudible].
>> Joel Robertson: Okay. See, this is -- this is where, you know, our philosophy, of
course, is -- and this is what we do. But there are so many solutions that can build on
this. And for us to try to find all of those things is ridiculous. We're just, again, kind of
in the attitude of saying we're really -- we really think we have something cool here. It's
been tested enough. It's got some solid theory and testing. And now it just needs to be
made better by partners.
>>: [inaudible] opportunity at the [inaudible] health summit for discussion and
interaction in a lot of mobile health solutions from around the world. And so I do hope
that you'll have an opportunity to come and participate as well.
>> Joel Robertson: I was just invited to September 15th to a meeting in India by Intel
just today. And ->>: Big notice.
>> Joel Robertson: Yeah. And similar with the India public health system, and so that I
will see if I can get there, because that -- it is a matter of cooperation.
>>: I will not get there. That's too short of notice.
>>: One of the things that we were talking about, too, is that when you use these kind of
things, and you have developed some plug-and-play analytics for unskilled users
[inaudible] or whatever. So even someone [inaudible] validated tools, validated data, and
start doing some studies on their own, both publishable, because [inaudible] models that
you know about.
>> Joel Robertson: I think there's another thing, too. I know you guys at Microsoft
Research had a maternal care database or -- I mean, those are the types of things that we
say, you know, if we get to the point where people think what we do is legitimate, we
need data. And grabbing that data and putting that in in a format that will be used, those
are very important things that we look at from that standpoint.
>>: [inaudible] scaling up [inaudible] but just from a business point of view, is it this -is it a way for people to sort of put sustainable businesses on the map? Right? So people
have to start making money. If there isn't money being made or providing some services,
then there's an incentive to sort of keep things up-to-date.
>> Joel Robertson: You know ->>: [inaudible]
>> Joel Robertson: The practicality of it is is it has to be sustainable and it has to be
something that people work with. Blue Label Telecom, which is one of your ventures, I
talked with one of the guys last week, some really creative ideas of how to use a micro
business with this process.
We have some strong corporate relationships in India that provide some of those things.
So I think there's a whole strategy like you say of not only data collection but business
management, maintaining a business model, infrastructure model. There's just so much
that needs to wrap around it.
You're fortunate that you're the first people that have heard our vision -- well, not the first
people that hear our vision, but the first group from the standpoint of us saying this is
where we're at now in our deployment and here are some of our problems. And I can't
think of a better place to throw our problems than at Microsoft Research and see if you
have any solutions.
>> Eric Horvitz: I think we should stop there [inaudible]. Thanks very much.
>> Joel Robertson: Great. Thank you.
[applause]