Information for Energy Group, Energy Institute,
New Cavendish Street, London
Thursday 13 May 2010
Advanced Internet Searching for Energy
Information & Market Research
karen.blakeman@rba.co.uk , http://twitter.com/karenblakeman
http://www.slideshare.net/karenblakeman
http://www.rba.co.uk/ , http://www.rba.co.uk/wordpress/
This presentation is licensed under a Creative Commons Attribution 3.0 License
Photo: http://www.flickr.com/photos/rbainfo/3313765895/
Search is changing
• How people look for and find information is changing
• Social media and networks are now an essential part of the search
mix
• Facebook for market research and customer feedback e.g. the marmite
bar http://www.facebook.com/MarmiteBars
• Business students assignment on advertising history of Cadbury’s
Crème Egg and Dairy Milk – Youtube!
• Twitter and blogs used as an essential part of reputation monitoring
• What’s happening at conferences - Twitter, Slideshare, Flickr, videos,
podcasts
• Search engines now including social media and professional
networks in search results
Search on a person
Blogger profile
Twitter profile
Facebook profile
LinkedIn profile
14 March 2016
Karen Blakeman www.rba.co.uk
3
Search on a topic – first page results
Results from
Google News
Latest results rolling feed
from Twitter,
blogs,
discussions
14/03/2016
www.rba.co.uk
4
Search on a topic – first page results
Images
Blog posting
listing videos
Wikipedia article
YouTube video
Results from my “social
circle”: Twitter, RSS feeds,
FriendFeed, Google Wave
14/03/2016
contacts etc.
www.rba.co.uk
5
Social media
• It is where many companies and organisations market themselves,
provide customer support and distribute information
• It is where an increasing proportion of the population grow up & they
don’t differentiate between the different means of communication
• It is where an increasing number of people feel comfortable and
share information
• It’s impossible to ‘gag’ social networks
– BBC NEWS | Politics | When is a secret not a secret?
– http://news.bbc.co.uk/1/hi/uk_politics/8304908.stm
For serious research, social media and professional networks
are not optional extras - they are an essential part of your
research strategy
14 March 2016
Karen Blakeman www.rba.co.uk
6
Types of search tools
• Humans
– is a colleague or a friend already working in, or knowledgeable
about the subject area?
– who have you met at meetings, conferences?
– discussion lists, bloggers, LinkedIn, Facebook,
trade/professional associations, Twitter etc. (social “stuff”!)
• Search engines
– Google, Yahoo, Bing,
• So called “deep web” and specialist tools, evaluated listings,
subject listings, customised search engines
• Databases and peer reviewed sources
• Multi search engine tools, federated search
– search lots of search tools or databases at once
– or type in your search once and click on each search tool in turn
14 March 2016
Karen Blakeman www.rba.co.uk
7
How up to date are search tools?
• Google, Yahoo, Bing etc.
– you are searching an out of date index of the web and not the
web itself
– may take days to months for a site to be added to the index
– hierarchy of sites for updating
• Real time, social media search tools
– Twitter, blogs, discussion boards, RSS feeds
– what is happening NOW
14 March 2016
Karen Blakeman www.rba.co.uk
8
What are you looking for?
•
•
•
•
Authoritative facts?
Official or corporate announcements/press releases?
News articles?
Views and opinions on a topic?
– from experts
– from consumers
– from people who don’t know what they’re talking about!
• Spur of the moment rants about failed services or products?
• Rumour, speculation, gossip?
• Or do you want to gather new data?
– e.g. set up surveys on Facebook, Survey Monkey, Twtpoll.com
14 March 2016
Karen Blakeman www.rba.co.uk
9
Before we start - is your Google web history on?
• Switched on by default
• Google adjusts your search results according to what you
have clicked on in the past
• See Karen Blakeman's Blog - Your Google results are
about to get weirder
• http://www.rba.co.uk/wordpress/2009/12/17/your-googleresults-are-about-to-get-weirder/
14 March 2016
Karen Blakeman www.rba.co.uk
10
A search engine’s results may vary
• In content and presentation
– from one minute to the next
– different server being used
– testing out different search and ranking algorithms
• Country versions
–
–
–
–
–
14 March 2016
different emphasis
local content
different interface
different search features
use if researching a person, company, subject located or
based in another country
Karen Blakeman www.rba.co.uk
11
14 March 2016
Karen Blakeman www.rba.co.uk
12
General search techniques
• By default, the main search tools look for all of your terms
in a page
• Use double quote marks around phrases or names
• To exclude pages containing a term, precede the term with
a minus sign (-) but use with care
• Focus your search in areas of the page
- use advanced search screens or commands
- inurl: for example UK inurl:carbonemissions
• looks for your terms/phrase in the URL
- intitle: or allintitle: for example UK intitle:"carbon
emissions"
• looks for your terms/phrase in the title of the page
14 March 2016
Karen Blakeman www.rba.co.uk
13
General search techniques
• Search sites or domains using the advanced search screen or site:
command
– wind turbine energy generation site:statistics.gov.uk
– wind turbine energy generation site:gov.uk
• Imagine what you would like to appear in your ideal document and
include those terms in your strategy
- renewable energy generation UK wind solar wave hydro
• Repeat your key search terms in your search strategy
– tar sands EROEI
– tar sands EROEI EROEI EROEI
• different results
• Change the order of your terms
– biodiesel fuel UK buses efficiency
– biodiesel fuel efficiency UK buses
• different results
14 March 2016
Karen Blakeman www.rba.co.uk
14
File format search
• Use advanced search options to limit your search to file
types or format:
– pdf or doc for government or industry/market reports
– xls for data and statistics
– ppt or pdf for presentations, experts on a topic
• Use Google and Yahoo Advanced Search
• Bing.com sometimes picks up unique content
– use command filetype: in your search strategy e.g.
filetype:ppt car ownership UK
14 March 2016
Karen Blakeman www.rba.co.uk
15
Docjax
• http://www.docjax.com/
• searches pdf, xls, doc, ppt in Yahoo and Google
14/03/2016
www.rba.co.uk
16
Presentations
• For :
– find experts on a topic
– what are companies doing
– introduction to a topic or a general overview
– data and market statistics
– company structures, who’s who
• Presentation search:
– Slideshare http://www.slideshare.net/
– Authorstream http://www.authorstream.com/
– Slideboom http://www.slideboom.com/
– Slidefinder http://www.slidefinder.net/
14/03/2016
www.rba.co.uk
17
Slideshare.net
Search: biofuels UK public transport
14/03/2016
www.rba.co.uk
18
Slidefinder
• http://www.slidefinder.net/
• Identifies individual slides within a presentation
14/03/2016
www.rba.co.uk
19
Slidefinder
• Can limit search to universities
• Also Advanced Search to search areas of slides
14/03/2016
www.rba.co.uk
20
Google
•
Additional search features previously hidden under “Show options” now in left
hand side menu
•
Most are self explanatory
•
A few that aren’t....
Updates = Twitter
14/03/2016
www.rba.co.uk
21
Google
• Wonderwheel
14/03/2016
www.rba.co.uk
22
Google
• Timeline
– but Google can get it wrong
?
14/03/2016
www.rba.co.uk
23
Unique Google search features
• Similar or related pages
– looks for pages that are similar in type and content
– click on the Similar link next to the page in your results list
– or use the related: command e.g. related:www.bwea.com
14/03/2016
www.rba.co.uk
24
Unique Google search features
• Automatically looks for variations on your terms
– to stop it, precede your terms with plus signs
e.g. +Norne oil field to stop Google looking for ‘Horne’
– or use double quote marks around the term or phrase “Norne”
– (hat tip to Even Flood for this example)
• Numeric range search
– can be weights, distances, years, prices, measurements of any
sort
– use Advanced Search screen
– or the search box on the Google home page
– search term(s) first value..second value unit of
measurement
– toblerone 1..6 kg
– world oil production forecasts 2010..2015
14 March 2016
Karen Blakeman www.rba.co.uk
25
Unique Google search features
•
Proximity
– use the asterisk (*) to stand in for one or more terms
– useful when searching for people
– separates the terms by one or more words
– Karen * Blakeman picks up:
• Karen Blakeman
• Karen Mary Blakeman
• Karen Sands Blakeman
• Karen Marie Blakeman
– solar * panels picks up:
• solar heating panels
• solar electricity panels
• solar thermal panel
• solar voltaic panels
• solar concentrator panels
14 March 2016
Karen Blakeman www.rba.co.uk
26
Google Finance
• http://www.google.co.uk/finance
14/03/2016
www.rba.co.uk
27
Yahoo!
• http://www.yahoo.co.uk/
• very busy interface!
• http://search.yahoo.co.uk/
• simpler, Google like interface
14 March 2016
Karen Blakeman www.rba.co.uk
28
Yahoo!
• http://search.yahoo.co.uk/
• Advanced Search hidden under “More”
• Similar options to Google Advanced Search
14 March 2016
Karen Blakeman www.rba.co.uk
29
Yahoo!
•
•
•
•
http://www.yahoo.co.uk/
http://search.yahoo.co.uk/ http://search.yahoo.com/
Deal with Bing
Some country versions already using Bing web search, rest will
follow during the year
• Now includes options in the left side menu for limiting your
seearch
14 March 2016
Karen Blakeman www.rba.co.uk
30
Yahoo Finance
• http://uk.finance.yahoo.com/
14/03/2016
www.rba.co.uk
31
Bing
• http://www.bing.com/ (was live.com)
• “decision engine”
• Results tend to be more consumer oriented
• Seems to have the most up to date database and possibly the
most extensive database of web pages
• Results vary considerably depending on the country version
• Supposedly incorporating real time, social media results into
main search
• Advanced Search screen ‘minimalist” but can use commands
(see Search Engine Summary and Comparison Chart
http://www.rba.co.uk/search/compare.pdf)
14 March 2016
Karen Blakeman www.rba.co.uk
32
Compare search engines
• TripleMe
– http://www.tripleme.com/
– compares Google, Yahoo and Bing side by side
• Blindsearch
– http://blindsearch.fejus.com/
– compare Google, Yahoo and Bing results for web or images
– vote on which are the best without seeing the branding
• Zuula
– http://www.zuula.com/
– runs your search through a range of search tools one by one
– order can be customised
• Browsys Finder (was Intelways/Crossengine)
– http://www.browsys.com/finder
– runs your search through a plethora of search tools one by one
14 March 2016
Karen Blakeman www.rba.co.uk
33
iSEEK
•
•
•
http://www.iseek.com/
Clusters results into topics, people, places, organisations, date & time
“Education” option – more research oriented pages
14 March 2016
Karen Blakeman www.rba.co.uk
34
Biznar
• http://www.biznar.com/
• Real time federated search
of selected business
resources, some of them
“hidden web” or “deep web”
• Click on Advanced Search
to see a list of the sources
14 March 2016
Karen Blakeman www.rba.co.uk
35
Intute
•
http://www.intute.ac.uk
•
Funding to be discontinued but has been given a stay of execution
•
Excellent starting point for quality resources in all subject areas and industries
14/03/2016
www.rba.co.uk
36
TechXtra
•
http://www.techxtra.ac.uk
•
ICBL and the Library at Heriot-Watt University, Edinburgh
•
Engineering, mathematics and computing
•
Free information and pay per view
14 March 2016
Karen Blakeman www.rba.co.uk
37
Alacrawiki spotlights
• http://www.alacrawiki.com/
• Key resources for major industry sectors
• Most pages have not been updated for 2-3 years  BUT much of
the information still valid
14/03/2016
www.rba.co.uk
38
Alacrasearch
• http://www.alacra.com/alacrasearch/
• Google custom search engine
14/03/2016
www.rba.co.uk
39
LGSearch
• http://lgsearch.net/
• Another Google custom search engine
• Searches UK public sector web sites
14/03/2016
www.rba.co.uk
40
Directionlessgov.com
14/03/2016
www.rba.co.uk
41
http://www.search4oil.com/
• New!
• Information on the oil and gas sectors
14/03/2016
www.rba.co.uk
42
http://www.search4oil.com/
• People, Feature
14/03/2016
www.rba.co.uk
43
Statistics
• Some general starting points
– OFFSTATS - The University of Auckland Library
http://www.offstats.auckland.ac.nz/
• by country or subject or both
• excellent list of energy web site stats including the BP Statistical
Review of World Energy (2009)
– RBA Energy Information Sources
http://www.rba.co.uk/sources/energy.shtml (under revision!)
– Google Public data explorer http://www.google.com/publicdata/
• See review Google Public Data Explorer- fine as far as it goes
http://www.rba.co.uk/wordpress/2010/03/27/google-public-dataexplorer-fine-as-far-as-it-goes/
– Guardian datastore and datablog http://www.guardian.co.uk/datastore and World Government Data http://www.guardian.co.uk/worldgovernment-data
• the latter is a better way of searching http://data.gov.uk/
14/03/2016
www.rba.co.uk
44
Guardian World
Government Data
14/03/2016
data.gov.uk
www.rba.co.uk
45
Energy Export Databrowser
http://mazamascience.com/OilExport/
14/03/2016
www.rba.co.uk
46
Free industry and market research
• Individual publishers
– tables of contents, executive summaries, free sample
chapters
• Government bodies
– US http://www.export.gov/, http://www.bisnis.doc.gov/
– regulatory bodies e.g. in the UK - Competition Commission,
Ofcom
•
•
•
•
Management consultants
Academic sector
Trade, industry, manufacturing associations
Trade and industry press
14/03/2016
www.rba.co.uk
47
Priced industry and market research
• Aggregators may not be comprehensive
– many publishers embargo reports for weeks or months
– many publishers withhold reports from aggregators
– check the publisher’s own web site or email them to see if there is
additional or more recent research available
– may get a better deal by going to the publisher direct
– use aggregators as an index to see who is publishing on your
topic, for example
• http://www.marketresearch.com/
• http://www.mindbranch.com/
• http://www.researchandmarkets.com/
• http://www.alacrastore.com/
• http://www.reportbuyer.com/ (go to Energy and Utilities)
• For emerging markets http://www.securities.com/
14/03/2016
www.rba.co.uk
48
Priced industry and market research
• Many sites specialising in energy reports
– just search on energy market research reports
– Energy market research, analysis, databases, forecasts and
energy market consulting http://www.absenergyresearch.com/
– Energy Market research http://www.energy-market-research.info/
– GBI Research http://www.gbiresearch.com/ (Go to the Oil and
Gas section)
14/03/2016
www.rba.co.uk
49
News
•
•
•
•
•
•
•
Newspapers, journals, trade press
Blogs
Twitter
Video
Podcasts
Set up alerts and RSS feeds
Google News
–
–
–
–
http://news.google.co.uk/
good regional, national, international and trade press coverage
free sources only so misses some important publications
useful way of identifying industry publications
14/03/2016
www.rba.co.uk
50
Chipwrapper
http://www.chipwrapper.co.uk
• Google Custom
Search engine
• Searches everything
available on 15 free
UK News Sites
• “Time slices” for last
24 hours, past week,
past month, past year
14/03/2016
www.rba.co.uk
51
Silobreaker
http://www.silobreaker.com/
• covers free resources
• news, blogs, video,
tweets
• geographical location
of stories
• people
• networks
• use as a starting point
for in depth research
• has an energy section
14/03/2016
www.rba.co.uk
52
RSS for ...
• Current awareness and alerts
– newspapers, blogs, twitter
– customised alerts on topics or as part of reputation
monitoring/management
• Read feeds in
–
–
–
–
14/03/2016
an online feed reader e.g. Google Reader
desktop client e.g. FeedReader, RSS Bandit, Omea
Outlook (2007 and later)
Start pages e.g. Netvibes, My Yahoo, iGoogle,
www.rba.co.uk
53
RSS feeds (in Omea)
14/03/2016
www.rba.co.uk
54
Netvibes.com
14/03/2016
www.rba.co.uk
55
Netvibes.com
• New dashboard feature
– type in your keywords and Netvibes automatically searches
across news, social media, videos etc.
– looks in places you might not think of
14/03/2016
www.rba.co.uk
56
Video & Audio
•
•
•
•
Search engines’ video/audio tabs
YouTube, Vimeo.com, iTunes U
Blinkx - http://www.blinkx.com/
Truveo – http://www.truveo.com/
– country versions available
• Zuula - http://www.zuula.com/
– click on the video tab
• Browsys Finder (formerly Intelways/Crossengine)
– http://www.browsys.com/finder/
– click on the video tab
• Podcasts associated with news stories on e.g. BBC, The
Guardian, New Scientist, Nature etc.
• Odeo - http://www.odeo.com/
14 March 2016
Karen Blakeman www.rba.co.uk
57
Blogs and blog search tools
•
Blogs are a very easy and quick means of publishing
•
New postings are picked up within minutes - sometimes seconds – of
publication
•
RSS feeds for alerts and keeping up to date
•
Google left side menu
– Blogs
– By date and time
• Latest, Past 24 hours, Past week include large proportion of forums,
news and press releases, blog postings
•
Blog search tools
– Look for the blogroll or list of links on a known, trusted blog
– Google blogsearch http://blogsearch.google.com/
– Blogpulse http://www.blogpulse.com/
– Icerocket http://www.icerocket.com/ (also Twitter and MySpace)
– Technorati http://www.technorati.com/
14 March 2016
Karen Blakeman www.rba.co.uk
58
Blogs
• Many researchers and industry experts run blogs
• Some have lengthy articles, analysis and serious discussion e.g.
The Oil Drum (http://www.theoldrum.com), Energy Balance
(http://ergobalance.blogspot.com/)
14/03/2016
www.rba.co.uk
59
Searching Twitter and tweets
• To use Twitter’s Find People you need to be signed in to
Twitter
• Alternatively search for people or organisations using
Tweepz (http://www.tweepz.com/)
• Searching public tweets
– http://search.twitter.com/, http://www.twazzup.com/
• Backtweets.com – enter the URL of a web site and see
who has referred to it in a tweet
• Phil Bradley's weblog: Twitter Search - 20 alternative
search engines
– http://philbradley.typepad.com/phil_bradleys_weblog/2009/03/
twitter-search-20-alternative-search-engines.html
14 March 2016
Karen Blakeman www.rba.co.uk
60
Twitter search
14 March 2016
Karen Blakeman www.rba.co.uk
61
LinkedIn.com
• Check up on individuals, companies, groups
14 March 2016
Karen Blakeman www.rba.co.uk
62
Design your own search engine
• For
– regularly searched sites
– selected sites on a topic
– searching sites on a reading list
• Google Custom Search Engines
– http://www.google.com/cse/
– at least hundreds of sites, maybe thousands!
– can import lists of sites
• Need a Google account to set one up
• Cannot search password protected sources or sites where
you have to fill in a form to access the information
14 March 2016
Karen Blakeman www.rba.co.uk
63
Host it on Google
• Keep it private or make it public
• URLs are horribly long but you can create a TinyURL, Bit.ly
URL etc. to make it easier for your users
14 March 2016
Karen Blakeman www.rba.co.uk
65
Disappearing pages
 Search engine cache copies
– Google, Yahoo, Bing
 UK Web Archive
 http://www.webarchive.org.uk/ukwa/
 Wayback machine
– http://www.archive.org/
– from 1996 to about 1 year ago
– navigate the archived site or type in the full URL of the
document if known
14/03/2016
www.rba.co.uk
66
UK Web Archive of
http://www.berr.gov.uk/energy/index.html
14/03/2016
www.rba.co.uk
67
Wayback Machine
14 March 2016
Karen Blakeman www.rba.co.uk
68
Further information & keeping up to date
• Information World Review (IWR) and IWR Blog
– http://www.iwr.co.uk/, http://blog.iwr.co.uk/
• Phil Bradley’s Blog
– http://philbradley.typepad.com/
• I want to
– http://philbradley.typepad.com/i_want_to/
• Web 2.0 tools and applications: 1100+ Web2 apps listed, annotated
– http://www.philb.com/iwantto.htm
• Karen Blakeman’s Blog
– http://www.rba.co.uk/wordpress/
• Annual Online Information Exhibition, London, usually at the
beginning of December
– http://www.online-information.co.uk/
– Extensive programme of free seminars, updates and master
classes on the exhibition floor.
14 March 2016
Karen Blakeman www.rba.co.uk
69