A Project
On
Web Chat Application
for the award of the degree of Bachelor of Technology
In
Computer Science and Engineering
By
Ajit kumar
(1904920100008)
Aditya kumar Gupta
(1904920100005)
Aditya Prakash
(1904920100006)
Under the guidance of
Professor DR. Sanjay Kumar
Dept. of Computer Science
Department of Computer Science and engineering
KCC INSTITUTE OF TECHNOLOGY AND MANAGEMENT
(Affiliated to Dr. A.P.J. Abdul Kalam Technical University, Approved by AICTE)
Knowledge Park-III, Greater Noida, Uttar Pradesh 201306
KCC INSTITUTE OF TECHNOLOGY AND MANAGEMENT
(Affiliated to Dr. A.P.J. Abdul Kalam Technical University, Approved by AICTE)
Knowledge Park-III, Greater Noida,
Uttar Pradesh 201306
Department of computer science and Engineering
This is to certify that the project entitled “Chat Application” being submitted
by Ajit Kumar (1904920100008),Aditya Prakash (1904920100006) and
Aditya kumar Gupta (1904920100005) partial fulfillment of the requirements
for the award of the degree of the Bachelor of Technology in Computer Science
and Engineering to KCC Group of Institutions is a record of bonafide work
carried out by them under my guidance and supervision from September 2022
to December 2022.
The results presented in this project have been verified and found to be
satisfactory. The results embodied in this project report have not been submitted
to any other University for the award of any other degree or diploma.
Internal Guide
Guide Name (Asst. Professor, Dept of CSE)
Dr. Sanjay kumar,
Professor and Head, Dept. of CSE
External Examiner
ACKNOWLEDGEMENT
It is our privilege and pleasure to express a profound sense of respect, gratitude
and indebtedness to our Professor Dr. Sanjay Kumar, Dept. of Computer
Science and Engineering, KCC Group of Institutions for his/her indefatigable
inspiration,
guidance,
cogent
discussion,
constructive
criticisms
and
encouragement throughout this dissertation work.
We express our sincere gratitude to Dr Sanjay, Associate Professor & Head,
Department of Computer Science and Engineering, KCC Group of Institutions,
for his suggestions, motivations and co-operation for the successful completion
of the work.
We extend our sincere thanks to Mr. Deepak Gupta, Chairman, KCC Group of
Institutions for his encouragement.
Ajit kumar- 1904920100008
Aditya kumar Gupta- 1904920100005
Aditya Prakash - 1904920100006
DECLARATION
We hereby
declare
Application” submitted
that
to
the
project
the KCC
work
Group
of
entitled “Web Chat
Institutions in partial
fulfilment of the requirements for the award of the degree of Bachelor of
Technology (B. Tech) in Computer Science and Engineering is a record of an
original work done by us under the guidance of Sanjay kumar, Assistant
Professor and this project work have not been submitted to any other university
for the award of any other degree or diploma.
Ajit kumar (1904920100008)
Aditya kumar Gupta (1904920100005)
Aditya Prakash (1904920100006)
ABSTRACT
Messaging has become a part of our everyday lives in part due to its convenience
for real-time chat communication and simple-to-use functionality. For instance, an
iOS or text message on an iPhone or Android device from a friend, an email from a
co-worker on Microsoft or Gmail, a team chat in a Slack or Microsoft Teams
workspace, or even instant messaging through social media. These messaging and
real-time chat applications play an important role in how the world interacts today,
due to their immediacy and vast capabilities.
A chat application makes it easy to communicate with people anywhere in the
world by sending and receiving messages in real time. With a web or mobile chat
app, users are able to receive the same engaging and lively interactions through
custom messaging features, just as they would in person. This also keeps users
conversing on your platform instead of looking elsewhere for a messaging
solution. Whether it’s private chat, group chat, or large-scale chat, adding
personalized chat features to your app can help ensure that your users have a
memorable experience.
More people are embracing virtual experiences as a way to connect with one
another. From gamer group chat messaging in a live chat to e-learning and team
communication using chat rooms and file sharing among co-workers, online chat
applications have grown largely in popularity because of their ability to retain the
feel of a real-time conversation, virtually. However, when thinking ahead to how
you want to build your web app, a vital thing to consider is your customer's
experience. What are the essential features and functionalities that are needed to
make an engaging real-time chat app that will lead to user retention? Let’s dive in.
A digital chat app that’s composed of real-time messaging features enables users to
have an authentic and interactive experience.
Features like message reactions, stickers, emojis, GIFs, and voice calls and video
chat, provide a way to engage your users directly on your app instead of external
platforms–creating a more connected experience.
Other functionalities like identifying active users, push notifications, and message
history—to name a few—also add to that immediacy by automatically detecting
the presence of users in a real-time chat application.
Being able to focus on your customers' needs helps foster retention on your
messaging platform. A real-time chat platform solution, like Pub Nub, gives you
the flexibility to customize the user experience any way that you want with our
unique APIs, SDKs, and pre-built chat components that can be easily incorporated
into the way you prefer to build. This means you can add innovative features that
other chat APIs won’t have, such as push notifications, user presence, geolocation,
and more.
Table of Contents
S.NO
PAGE NO
1. Introduction
1
1.1. Key observations
2
1.2 The Dawn of Social messaging era
2
1.3. Objectives and Methods
3
2. Analysis and Regional demographic stronghold
3
3. Analysis
5
3.1. Regional strongholds
5
3.2. Demographic
5
3.3. Product Features
6
4. Industry Challenges
4.1Key players and case Studies
5.Design a Chat system
5.1. Understanding the problem
7
7
12
12
5.2. propose high level deign and get buy in
5.3. polling and long polling
5.4. Web Sockets
5.5. Design deep dive
6.Major Tech stacks used
17
7. Results and Screenshots
19
9. Conclusion and Future Enhancements
20
10. Reference
23
INTRODUCTION
Messaging apps now have more global users than traditional social networks —which
means they will play an increasingly important role in the distribution of digital journalism
in the future. Drawing upon our interviews and case studies, we identify a number of
opportunities and challenges for organizations using—or hoping to use—messaging apps
for news. We argue that to devise a successful messaging app strategy, publishers must
understand regional strongholds, user demographics, and popular features of each app. As
happened after the early days of social media, before which a proliferation of services
(some with regional strengths) led to intense competition for user attention, we expect to
see some eventual consolidation among chat apps. Elsewhere, we conclude that issues
around information, privacy, personal security, and mobile data penetration will unfold in
different ways around the world; apps like Telegram and Fire Chat are among those at the
forefront of addressing and solving these problems. In developing editorial strategies for
some of these wide-ranging messaging platforms, news organizations are not just helping
to future-proof themselves, they are also venturing into online spaces that could enable
them to reach hundreds of millions of (often young) people with whom they have never
engaged before.
Messaging apps now have more global users than traditional social networks—which
means they will play an increasingly important role in the distribution of digital journalism
in the future. While chat platforms initially rose to prominence by offering a low-cost,
web-based alternative to SMS, over time they evolved into multimedia hubs that support
photos, videos, games, payments, and more.
While many news organizations don't yet use messaging apps, digitally savvy outlets like
BuzzFeed, Mashable, The Huffington Post, and VICE have accompanied a more
traditional player in BBC News by establishing a presence on a number of these platforms.
To complement our research, we interviewed leadership at multiple news outlets and chat
platforms, thereby synthesizing key lessons and presenting notable case studies reflecting
the variety of creative and strategic work taking place within the messaging space. Most
publisher efforts around messaging apps are still in a formative, experimental stage, but
even those have often proven effective in diversifying traffic sources for digital content.
Drawing upon our interviews and case studies, we identify a number of opportunities and
challenges for organizations using—or hoping to use—messaging apps for news. We argue
that to devise a successful messaging app strategy, publishers must understand regional
strongholds, user demographics, and popular features of each app. Advantages to the chat
ecosystem include huge, untapped audiences; high engagement through push notifications;
unique products like stickers and “chatbots” (see glossary for definitions); and the
opportunity to build community through chat rooms and crowdsourced storytelling.
Meanwhile, challenges include limited analytics tools and a fragmented social landscape
boasting roughly a dozen messaging apps, each with over 50 million registered users.
Our case studies illustrate a number of ways in which major news outlets have utilized
various messaging apps, each with its own niche characteristics. In the past two years,
many platforms—including Snapchat, Viber, Kik, LINE, WeChat, and Telegram—
introduced official channels that publishers like CNN, The New York Times, The
Huffington Post, and Cliff Central now leverage for content distribution and user
engagement. Other players, like WhatsApp, have no official offering for media owners,
but this has not deterred organizations—most notably the BBC—from launching
experimental campaigns.
Our research indicates that one of the greatest benefits of chat apps is the opportunity to
use these platforms as live, sandbox environments. The chance to play and iterate has
helped several news organizations develop mobile-first content and experiential offerings
that would have proved difficult in other digital environments. As these services
primarily—and in some cases exclusively—exist on mobile phones, editorial teams have
learned to focus purely on the mobile experience, freeing themselves from considerations
about how content will appear on desktop websites or other broadcast mediums.
As happened after the early days of social media, before which a proliferation of services
(some with regional strengths) led to intense competition for user attention, we expect to
see some eventual consolidation among chat apps. While Facebook Messenger and Skype
do not garner much focus in this report, we believe they could become significant players
in this space over the coming year as they figure out the right proposition for their
platforms to partner with media organizations.
Elsewhere, we conclude that issues around information, privacy, personal security, and
mobile data penetration will unfold in different ways around the world; apps like Telegram
and Fire Chat are among those at the forefront of addressing and solving these problems.
They, in conjunction with similar applications, are likely to see an increase in user uptake
as they meet needs that other major chat apps are unable to serve.
In developing editorial strategies for some of these wide-ranging messaging platforms,
news organizations are not just helping to future-proof themselves, they are also venturing
into online spaces that could enable them to reach hundreds of millions of (often young)
people with whom they have never engaged before.
1.
Key Observations
The key findings of this report can be summarized as follows:
•
Messaging apps offer strong opportunities to engage new or difficult-to-reach
demographics. For example, Snapchat, a very popular app among millennials, has
been credited with engaging and informing young audiences on complex issues like
the Iran nuclear deal.
•
Innovations such as bespoke (or customized) stickers and emojis can help news
outlets quickly build significant audiences via messaging apps.
•
Messaging apps enable news outlets to gather (potentially exclusive) usergenerated content and can become a major source during breaking news situations
(e.g., as in the BBC's coverage of the second Nepal earthquake in 2015).
•
News outlets may need to experiment with a variety of chat apps to decipher which
content type best suits the audiences of each app. BuzzFeed, for example, had
considerably more success using LINE for soft news than for hard news.
•
Messaging apps can provide unique opportunities for giving audiences direct
access to content and publishers, be it through tailormade WeChat platforms (Cliff
Central) or Public Chats on Viber (The Huffington Post).
•
Messaging apps provide a space for news outlets to engage their audiences with
different—possibly lighter—types of content (e.g., The Washington Post has used
Kik to distribute games, quizzes, and chat adventures).
•
Even apps that are not as geared toward publishing as some of their competitors,
such as WhatsApp, hold great potential for news outlets if used strategically.
•
Messaging apps not only facilitate communication with eyewitnesses in areas
where other forms of communication aren't functioning (e.g., during extreme
weather conditions), they can also provide a platform for people who don't feel safe
speaking on the telephone. Additionally, apps such as Telegram focus strongly on
encryption and security.
•
Messaging apps are not used solely to drive traffic to websites (e.g., BuzzFeed is
using LINE as part of a more distributed strategy and aims to provide as much
information as possible inside the app, rather than direct users to its website).
•
When sourcing user-generated content, apps like WhatsApp, which are tied to a
user's phone number, can significantly speed up the verification process as they
provide a direct line to the content creator/eyewitness.
•
Messaging apps are relatively new and evolving at a rapid pace. Consequently,
many news outlets are still in the process of establishing strategies to best utilize
these platforms.
•
There is emerging evidence that messaging apps may drive traffic back to outlets'
other, more traditional platforms (e.g., CNN's Snapchat Discover drove people to
seek out longer-form content on CNN's TV news channel).
2.
Introduction: The Dawn of the Social Messaging Era
“The social media phase of the Internet ended,” declared Fred Wilson—a leading venture capitalist
and early investor in Twitter, Foursquare, and Tumblr—in his 2014 “What Just Happened?” yearin-review. He continued:
While Wilson's views may be surprising to some, statistics support them. Business Insider
Intelligence released an infographic in April titled “Messaging Apps Have Caught Up to
Social Networks,” indicating that major chat apps (WeChat, Viber, WhatsApp, and
Facebook Messenger) now have more combined users than the top social networks
(Twitter, LinkedIn, Facebook, and Instagram). 2 And according to Mary Meeker's “Internet
Trends 2015” report, messaging apps make up 60 percent of the top 10 global apps by both
usage and number of sessions. 3
Like any presumed overnight success, WhatsApp’s rise was actually part of a broader
movement that took years to coalesce within a zeitgeist moment. Facebook's $22-billion
purchase of the app merely marked the day that everyone, even those with limited interest
in technology, learned that the era of social messaging had arrived.
Any user now downloading the world's 10 biggest chat apps for the first time may be
struck by their similarities. Dig deeper into their histories, though, and you'll see that each
company conquered the globe with one unique, core innovation before incorporating some
of its competitors' strongest ideas: LINE popularized stickers, Viber perfected free mobile
calling, Snapchat invented ephemeral messaging, Kik pioneered mobile chat anonymity,
Telegram supplied chat encryption, Tango delivered mobile video chat, WhatsApp
introduced the simplicity of free texting, and WeChat created the messaging app as mobile
web portal. Facebook Messenger, for its part, forged an amalgam of all those apps, and
now looks to differentiate itself with the introduction of M, an artificial intelligence chat
assistant.
3.
Objectives and Methods
In this guide, we explore the critical role chat apps can play in the distribution of digital
journalism today and in the years ahead. As mobile traffic and referrals from social
platforms continue to grow for the majority of news organizations, chat apps present a
profound opportunity for audience development and engagement.
According to Pew Research Centre’s “State of the News Media 2015” report, 78 percent of
the top 50 digital news websites receive more visits from mobile devices than desktop
computers. And analytics firm Parse.ly, which delivers audience insights to over 400 news
organizations including Wired, The Atlantic, and Reuters, claims 43 percent of referrals to
its publisher network now come from social media (outpacing search at 38 percent).
Messaging has emerged as the new frontier of social on mobile, and the sheer size of
audiences on the top global chat apps is too big to ignore. These apps also present an
opportunity to diversify mobile traffic sources and to minimize vulnerability should
Facebook or other platforms decrease traffic for publishers.
For our research, we conducted interviews with leadership at numerous major messaging
app companies, as well as early-adopter news organizations. We selected companies and
case studies that demonstrate the diversity of opportunities within the ecosystem, while
also highlighting the uniqueness of each chat app platform.
While publishers generally indicated optimism and excitement for their work on chat apps,
nearly all pointed out that as an industry we are still in an early, exploratory phase. Most
major chat apps spent the last few years perfecting their user experience, only recently
turning their attention to media-owner partnerships. With that in mind, we urge readers to
not only learn from the case studies presented, but also to initiate experiments of their own
to find the right strategy for any editorial team.
Modern instant messaging and SMS both began their march to prominence in the early and
mid-1990s. The difference between the two is subtle: SMS (the acronym for “short service
message”) allows mobile phone users to send each other text messages without an Internet
connection, whereas instant messaging enables similar functionality via the web. The first
SMS message was sent over the Vodafone GSM network in the United Kingdom on
December 3, 1992, with the words “Merry Christmas.” Israeli firm Mirabilis released the
first widely used online messenger, ICQ (short for “I Seek You”), in 1996.
Precursors to ICQ date as far back as 1961 when MIT's Computation Centre built the
Compatible Time-Sharing System (CTSS), allowing up to 30 users to log-in concurrently
and share text messages. CompuServe's CB Simulator, released in 1980, is generally
recognized as the first dedicated online chat service; it required users to pay monthly fees
for membership.
In 1985, Commodore launched Quantum Link (or “Q-Link”), an online service for
Commodore 64 and 128 that enabled multi-person chat, file sharing, electronic email,
games, and news via modem connection. Quantum Link changed its name to America
Online (AOL) in 1991, and by the mid-90s was the leading U.S. Internet service provider
and portal to the web.
The company launched AOL Instant Messenger (AIM) in 1997 and purchased competitor
ICQ in 1998 to consolidate its primacy over instant messaging. Along with a few
competitors, it also pioneered chat robots like Study Buddy and Smarter Child that
provided information and played games with users.
In 2006, AIM controlled 52 percent of the instant messaging market, but it struggled to
monetize and went into rapid decline in the face of competition from services like Google
Talk, Yahoo! Chat, MSN Messenger, and Skype. The growing popularity of BlackBerry
Messenger in the late 2000s also pointed to a bright future for mobile messaging.
By the time mobile chat apps like WhatsApp and Kik arrived in 2009, SMS was king.
Mobile texting became a key mode of global, personal communication, earning billions of
dollars for telecommunications companies.
But time and technology did not prove kind to telephone-service companies. As
smartphones began to proliferate, messaging apps were an increasingly accessible solution
to a simple problem: SMS is expensive in most countries, so why not text or call much
more cheaply, or for free, via the mobile web?
SMS volume peaked in 2012, and chat apps surpassed SMS in global message volume for
the first time in 2013. By January 2015, WhatsApp alone hosted 30 billion messages per
day compared to about 20 billion for SMS.
As Wi-Fi and high-speed mobile networks hit critical mass in many markets, chat apps
quickly became multimedia hubs where users could easily share videos, photos, stickers,
games, articles, live streams, and more. Between 2013 and 2014, many messengers turned
their attention to monetizing their massive audiences, and in doing so introduced tools for
publishers and brands.
.
Despite the boldness of Zuckerberg's repositioning, Facebook was responding to trends
more than shaping them. A large swath of chat app innovation continues to originate in
Asia on platforms like WeChat and LINE, and Western apps often appropriate those
platforms' best products.
WeChat, in particular, has a monolithic presence in China reminiscent of late 1990s
Internet portals like AOL and Yahoo (with a modern mobile twist). Users not only talk to
friends and consume news on the app, but also make purchases, pay utility bills, book taxis
and doctor’s appointments, enrol in brand loyalty programs, monitor traffic and air
pollution, and report incidents to the police.
That dominance of the mobile web in countries like China might not be as likely in other
markets, but it has served as inspiration for ambitious Western chat apps plotting
roadmaps for growth and monetization. Like WeChat, Japan's LINE is also incredibly
advanced in its product offerings, grossing $656 million in revenue in 2014. By contrast,
while leading Western messenger apps may boast multibillion-dollar valuations, they often
have profit and loss sheets in the red.
Time will tell if the same winning strategies will work halfway around the world. But for
news organizations in North America and Europe looking for a glimpse of how the market
may look in one to two years, downloading WeChat and LINE is the best place to start.
4.
Analysis and Regional and Demographic Strongholds
When devising your own strategy for messaging apps, it's vital to select the right platform
mix for your organization, based on three core criteria:
1. Regional Strongholds: Only a small group of apps like WhatsApp, Facebook
Messenger, and Viber can be said to be truly global—and even those platforms
struggle in certain countries. Meanwhile, messengers like WeChat, LINE, and
Kakao Talk completely dominate specific markets but have negligible traction in
others.
2. Demographics: It's a common misconception that messaging apps are a uniformly
millennial phenomenon. Some apps like Snapchat and LINE skew both young and
female, but others like Tango (which boasts 100 million monthly, active users, by
our estimate) predominantly appeal to those aged 25–54 and strongly over-index
with Hispanic and African-American users.
3. Product Features: If your primary goal is to drive traffic directly back to your
website, that won't be possible with apps like Snapchat that don't support clickable
URL links. And if you want to launch a sticker/emoji-pack campaign, LINE and
Viber are great options, but you can rule out both WhatsApp and Snapchat. Each
app has a particular set of features, so make sure to choose one that supports the
activation you have in mind.
5.
Industry Challenges
1. Fragmentation: The social media landscape is entering a period of hyperfragmentation that may be a challenge to publishers: Facebook, Twitter, and
Instagram continue to loom large, but social media managers can now launch
official channels on roughly 10 chat apps with over 50 million monthly, active
users each.
BuzzFeed, a perennial innovator in digital audience development, provides a great
example of what it looks like to be on nearly all of these platforms. Starting with
the launch of a share-to-WhatsApp button on mobile in February 2014, BuzzFeed
has also built an official presence on seven additional major messengers, as seen in
the timeline below.
2. Analytics: For organizations accustomed to robust, real-time data, the lack of good
analytics tools for messaging apps remains a major deterrent to adoption. The
challenge is twofold: Strong analytics dashboards take time to build, and many
messengers are privacy-centric by nature.
The latter issue has led to an existential crisis for apps keen to remain true to their
roots while also attracting brands and appeasing investors. To the far left,
companies like WhatsApp and Telegram have made public promises to shun
personal data collection and advertising. Closer to centre, Snapchat and Kik collect
basic information like age and location for ad targeting, but denounce “creepy,”
hyper-targeted ads. Other platforms like Pinger seem more inclined to offer rich
data collection and hyper-targeting of anonymized users. Specifically, Pinger
leverages keyword data to let advertisers reach users based on their interests.
Still, in a landscape increasingly driven by programmatic ambitions and datacentric decision making, chat apps currently deliver comparatively limited
information in basic dashboards, or in some cases via analogy reporting. Most are
mobilizing to remedy the situation quickly, and a host of third-party services are
also beginning to create dashboard solutions based on the major apps' willingness
to grant API access.
3. Growing Pains: As messaging apps respond to the demands of spectacular growth,
publisher partners we've spoken to have at times found it difficult to get consistent
support or answers to questions about platform capabilities.
Apps like Tango have also made large changes to their channel format for
publishers— starting with a news feed that helped drive traffic to content, but
transitioning to chat rooms geared toward discussion. While changes like this are
understandable as part of a process for looking to find experiences that resonate
with users, the volatility can be unsettling.
Other platforms like WhatsApp currently have little interest altogether in providing
tools for publishers. Their emphasis, for the time being, is purely on connecting
users to friends and family. And while this may not technically qualify as a
growing pain, it certainly poses logistical challenges for companies attempting to
adapt the app to their needs.
6.
Key Players and Case Studies
For publishers acting as early adopters, 2014 and 2015 has been a period of great
experimentation. In addition to learning the strengths and weaknesses of various platforms,
news organizations have also explored different strategies for adapting their professional
voice to an ecosystem filled with emojis, stickers, games, memes, and more.
We've taken a look at some of the main messaging platforms and how news organizations
use them.
Audience: 900 million monthly, active users
Top Markets: India, South Africa, Malaysia, Spain, Mexico
Demographics: Broad age range, not limited to those aged 16–34
Key Features: Free messaging for the first year, then an annual subscription of $1; free
WhatsApp calls; WhatsApp web interface; voice memos, as well as audio, image, and
video files; groups and broadcast lists.
In early September of 2015, co-founder of WhatsApp Jan Koum announced that the
company had passed the 900 million marks for monthly, active users.
The platform is likely to break the one billion active users mark by the end of 2015 or soon
afterward.
While it may not be the dominant chat app in the United States, WhatsApp rules across
many parts of the world as the number one downloaded app of any kind (let alone chat
app). One of the key reasons for its dramatic, global uptake is the free alternative it
provides to expensive SMS charges that exist in many parts of the world. In addition to
text messages, it can also facilitate voice memos, images, and video clips—and, most
recently, Internet calls.
Another big factor in its huge, global success is the simplicity of the app itself. Once
downloaded, it scans a user's address book and immediately lets him or her communicate
with any personal contacts already on WhatsApp. It doesn't require the creation of a
username or password—an impediment which remains a significant barrier to entry for
many people in emerging markets when it comes to other apps and social media platforms.
While many instant messaging platforms tend to skew toward a younger demographic of
users, WhatsApp has a much broader appeal globally.
BBC News was the first to experiment with editorial content on WhatsApp in 2014, most
notably with its Ebola WhatsApp “lifeline” information service targeting those in West
Africa.
However, WhatsApp is not engineered to work efficiently as a mass-push distribution
service, and much of BBC News's more recent strategy on the platform has focused on
audience engagement through user-generated content and newsgathering.
Early in 2015, the UGC and social media Hub, situated in the heart of the BBC's global
newsroom, set up a central WhatsApp number and encouraged news audiences across
online, TV, and radio to use it as a means to share their content with the newsroom.
Soon after launching the account, it quickly proved its value: “For us, WhatsApp has
proved a key way to get in touch with people in areas where other forms of communication
just don't work, so it was naturally the source we turned to following the earthquakes in
Nepal,” said Natalie Miller, a senior journalist at the UGC Hub.
Miller explained why WhatsApp became the primary communication tool for many in
Nepal at the time of the earthquake:
Phone lines were down in the affected areas and we can be waiting a long time for an
email to be received and answered, whereas WhatsApp is quick and allows people to share
their views, pictures, and videos with us all via the same channel. In some cases,
WhatsApp allows us to speak to people who just don't feel safe talking on the phone.
Miller also explained that directly verifying content when people submit it is a much
quicker process inside WhatsApp—as all inbound communication comes with a mobile
number attached to it. “On other social media channels, it can take some time for us to
message people and get a response,” she said. “With WhatsApp it's often instant as they
see our message popping up immediately on their phone. If we need to follow up with a
call, we have their mobile number and so these speeds up the process considerably.”
The launch of WhatsApp Web, which allows the app on a user's phone to pair with a
computer screen, has helped the UGC Hub manage incoming content much more
effectively, too. Miller recommended using an Android handset, as it is easier to transfer
the media content coming in via the subfolder menus on the Android platform via a USB
cable onto a computer; iPhones don't have the same ability. “We normally assign one of
our producers each day to manage the WhatsApp account in addition to their other duties.
The web interface means they can keep an eye on any incoming content, respond to any
messages, and also continue to do other work on their computers,” Miller added.
“In the case of the second earthquake in Nepal, we received almost 50 WhatsApp
messages, including pictures and videos from within the country and dozens more from
elsewhere talking about the quake,” she said. “It became the main source of content for the
live news blog on the BBC News website that day.”
The team also produced a separate story on the website detailing the experiences of those
affected on the ground.
The potential for using WhatsApp is clearly significant for the BBC. In addition to the
UCG Hub, a number of its news programs have set up their own accounts to solicit
audience opinion and contributions. Several of its language services also use WhatsApp to
source material around major stories. However, this broad type of use remains relatively
limited among other news organizations—particularly those with global audiences—and is
likely to be an area of increasing focus in the coming months.
Designing A Chat System
Step 1 - Understand the problem and establish design
scope
It is vital to agree on the type of chat app to design. In the marketplace, there are one-on-one chat
apps like Facebook Messenger, WeChat, and WhatsApp, office chat apps that focus on group
chat like Slack, or game chat apps, like Discord, that focus on large group interaction and low
voice chat latency.
The first set of clarification questions should nail down what the interviewer has in mind exactly
when she asks you to design a chat system. At the very least, figure out if you should focus on a
one-on-one chat or group chat app. Some questions you might ask are as follows:
Candidate: What kind of chat app shall we design? 1 on 1 or group based?
Interviewer: It should support both 1 on 1 and group chat.
Candidate:
Is
this
a
mobile
app?
Or
a
web
app?
Or
both?
Interviewer: Both.
Candidate: What is the scale of this app? A start-up app or massive scale?
Interviewer: It should support 50 million daily active users (DAU).
Candidate:
For
group
chat,
what
is
the
group
member
limit?
Interviewer: A maximum of 100 people
Candidate: What features are important for the chat app? Can it support attachment?
Interviewer: 1 on 1 chat, group chat, online indicator. The system only supports text messages.
Candidate:
Is
there
a
message
size
limit?
Interviewer: Yes, text length should be less than 100,000 characters long.
Candidate:
Is
end-to-end
encryption
required?
Interviewer: Not required for now but we will discuss that if time allows.
Candidate:
How
long
shall
we
store
the
chat
history?
Interviewer: Forever.
In the chapter, we focus on designing a chat app like Facebook messenger, with an emphasis on
the following features:
•
A one-on-one chat with low delivery latency
•
Small group chat (max of 100 people)
•
Online presence
•
Multiple device support. The same account can be logged in to multiple accounts at
the same time.
•
Push notifications
It is also important to agree on the design scale. We will design a system that supports 50 million
DAU.
Step 2 - Propose high-level design and get buy-in
To develop a high-quality design, we should have a basic knowledge of how clients and servers
communicate. In a chat system, clients can be either mobile applications or web applications.
Clients do not communicate directly with each other. Instead, each client connects to a chat
service, which supports all the features mentioned above. Let us focus on fundamental operations.
The chat service must support the following functions:
•
Receive messages from other clients.
•
Find the right recipients for each message and relay the message to the recipients.
•
If a recipient is not online, hold the messages for that recipient on the server until she
is online.
Figure 2 shows the relationships between clients (sender and receiver) and the chat service.
Figure 2
When a client intends to start a chat, it connects the chats service using one or more network
protocols. For a chat service, the choice of network protocols is important. Let us discuss this
with the interviewer.
Requests are initiated by the client for most client/server applications. This is also true for the
sender side of a chat application. In Figure 2, when the sender sends a message to the receiver via
the chat service, it uses the time-tested HTTP protocol, which is the most common web protocol.
In this scenario, the client opens a HTTP connection with the chat service and sends the message,
informing the service to send the message to the receiver. The keep-alive is efficient for this
because the keep-alive header allows a client to maintain a persistent connection with the chat
service. It also reduces the number of TCP handshakes. HTTP is a fine option on the sender side,
and many popular chat applications such as Facebook [1] used HTTP initially to send messages.
However, the receiver side is a bit more complicated. Since HTTP is client-initiated, it is not
trivial to send messages from the server. Over the years, many techniques are used to simulate a
server-initiated connection: polling, long polling, and WebSocket. Those are important
techniques widely used in system design interviews so let us examine each of them.
Polling
As shown in Figure 3, polling is a technique that the client periodically asks the server if there are
messages available. Depending on polling frequency, polling could be costly. It could consume
precious server resources to answer a question that offers no as an answer most of the time.
Long polling
Because polling could be inefficient, the next progression is long polling (Figure 4).
In long polling, a client holds the connection open until there are actually new messages available
or a timeout threshold has been reached. Once the client receives new messages, it immediately
sends another request to the server, restarting the process. Long polling has a few drawbacks:
•
Sender and receiver may not connect to the same chat server. HTTP based servers
are usually stateless. If you use round robin for load balancing, the server that
receives the message might not have a long-polling connection with the client who
receives the message.
•
A server has no good way to tell if a client is disconnected.
•
It is inefficient. If a user does not chat much, long polling still makes periodic
connections after timeouts.
WebSocket
WebSocket is the most common solution for sending asynchronous updates from server to client.
Figure 5 shows how it works.
WebSocket connection is initiated by the client. It is bi-directional and persistent. It starts its life
as a HTTP connection and could be “upgraded” via some well-defined handshake to a
WebSocket connection. Through this persistent connection, a server could send updates to a
client. WebSocket connections generally work even if a firewall is in place. This is because they
use port 80 or 443 which are also used by HTTP/HTTPS connections.
Earlier we said that on the sender side HTTP is a fine protocol to use, but since WebSocket is
bidirectional, there is no strong technical reason not to use it also for sending. Figure 6 shows
how WebSocket’s (ws) is used for both sender and receiver sides.
By using WebSocket for both sending and receiving, it simplifies the design and makes
implementation on both client and server more straightforward. Since WebSocket connections are
persistent, efficient connection management is critical on the server-side.
High-level design
Just now we mentioned that WebSocket was chosen as the main communication protocol
between the client and server for its bidirectional communication, it is important to note that
everything else does not have to be WebSocket. In fact, most features (sign up, login, user profile,
etc) of a chat application could use the traditional request/response method over HTTP. Let us
drill in a bit and look at the high-level components of the system.
As shown in Figure 7, the chat system is broken down into three major categories: stateless
services, stateful services, and third-party integration.
Stateless Services
Stateless services are traditional public-facing request/response services, used to manage the
login, signup, user profile, etc. These are common features among many websites and apps.
Stateless services sit behind a load balancer whose job is to route requests to the correct services
based on the request paths. These services can be monolithic or individual microservices. We do
not need to build many of these stateless services by ourselves as there are services in the market
that can be integrated easily. The one service that we will discuss more in deep dive is the service
discovery. Its primary job is to give the client a list of DNS host names of chat servers that the
client could connect to.
Stateful Service
The only stateful service is the chat service. The service is stateful because each client maintains
a persistent network connection to a chat server. In this service, a client normally does not switch
to another chat server as long as the server is still available. The service discovery coordinates
closely with the chat service to avoid server overloading. We will go into detail in deep dive.
Third-party integration
For a chat app, push notification is the most important third-party integration. It is a way to
inform users when new messages have arrived, even when the app is not running. Proper
integration of push notification is crucial. Refer to "Design a notification system" chapter for
more information.
Scalability
On a small scale, all services listed above could fit in one server. Even at the scale we design for,
it is in theory possible to fit all user connections in one modern cloud server. The number of
concurrent connections that a server can handle will most likely be the limiting factor. In our
scenario, at 1M concurrent users, assuming each user connection needs 10K of memory on the
server (this is a very rough figure and very dependent on the language choice), it only needs
about 10GB of memory to hold all the connections on one box.
If we propose a design where everything fits in one server, this may raise a big red flag in the
interviewer’s mind. No technologist would design such a scale in a single server. Single server
design is a deal breaker due to many factors. The single point of failure is the biggest among
them.
However, it is perfectly fine to start with a single server design. Just make sure the interviewer
knows this is a starting point. Putting everything we mentioned together, Figure 8 shows the
adjusted high-level design.
In Figure 8, the client maintains a persistent WebSocket connection to a chat server for real-time
messaging.
•
Chat servers facilitate message sending/receiving.
•
Presence servers manage online/offline status.
•
API servers handle everything including user login, signup, change profile, etc.
•
Notification servers send push notifications.
•
Finally, the key-value store is used to store chat history. When an offline user comes
online, she will see all her previous chat history.
Storage
At this point, we have servers ready, services up running and third-party integrations complete.
Deep down the technical stack is the data layer. Data layer usually requires some effort to get it
correct. An important decision we must make is to decide on the right type of database to use:
relational databases or NoSQL databases? To make an informed decision, we will examine the
data types and read/write patterns.
Two types of data exist in a typical chat system. The first is generic data, such as user profile,
setting, user friends list. These data are stored in robust and reliable relational databases.
Replication and sharding are common techniques to satisfy availability and scalability
requirements.
The second is unique to chat systems: chat history data. It is important to understand the
read/write pattern.
•
The amount of data is enormous for chat systems. A previous study [2] reveals that
Facebook messenger and Whatsapp process 60 billion messages a day.
•
Only recent chats are accessed frequently. Users do not usually look up for old chats.
•
Although very recent chat history is viewed in most cases, users might use features
that require random access of data, such as search, view your mentions, jump to
specific messages, etc. These cases should be supported by the data access layer.
•
The read to write ratio is about 1:1 for 1 on 1 chat apps.
Selecting the correct storage system that supports all of our use cases is crucial. We recommend
key-value stores for the following reasons:
•
Key-value stores allow easy horizontal scaling.
•
Key-value stores provide very low latency to access data.
•
Relational databases do not handle long tail [3] of data well. When the indexes grow
large, random access is expensive.
•
Key-value stores are adopted by other proven reliable chat applications. For example,
both Facebook messenger and Discord use key-value stores. Facebook messenger
uses HBase [4], and Discord uses Cassandra [5].
Data models
Just now, we talked about using key-value stores as our storage layer. The most important data is
message data. Let us take a close look.
Message table for 1 on 1 chat
Figure 9 shows the message table for 1 on 1 chat. The primary key is message_id, which helps to
decide message sequence. We cannot rely on created at to decide the message sequence because
two messages can be created at the same time.
Message table for group chat
Figure 10 shows the message table for group chat. The composite primary key is (channel_id,
message_id). Channel and group represent the same meaning here. channel_id is the partition key
because all queries in a group chat operate in a channel.
Message ID
How to generate message_id is an interesting topic worth exploring. Message_id carries the
responsibility
of
ensuring
the
order
of
messages.
To
ascertain
the
order
of
messages, message_id must satisfy the following two requirements:
•
IDs must be unique.
•
IDs should be sortable by time, meaning new rows have higher IDs than old ones.
How can we achieve those two guarantees? The first idea that comes to mind is the
“autoincrement” keyword in MySQL. However, NoSQL databases usually do not provide such a
feature.
The second approach is to use a global 64-bit sequence number generator like Snowflake [6].
This is discussed in the “Design a unique ID generator in a distributed system” chapter.
The final approach is to use local sequence number generator. Local means IDs are only unique
within a group. The reason why local IDs work is that maintaining message sequence within oneon-one channel or a group channel is sufficient. This approach is easier to implement in
comparison to the global ID implementation.
Step 3 - Design deep dive
In a system design interview, usually you are expected to dive deep into some of the components
in the high-level design. For the chat system, service discovery, messaging flows, and
online/offline indicators worth deeper exploration.
Service discovery
The primary role of service discovery is to recommend the best chat server for a client based on
the criteria like geographical location, server capacity, etc. Apache Zookeeper [7] is a popular
open-source solution for service discovery. It registers all the available chat servers and picks the
best chat server for a client based on predefined criteria.
1. User A tries to log in to the app.
2. The load balancer sends the login request to API servers.
3. After the backend authenticates the user, service discovery finds the best chat server for User
A. In this example, server 2 is chosen and the server info is returned back to User A.
4. User A connects to chat server 2 through WebSocket.
Message flows
It is interesting to understand the end-to-end flow of a chat system. In this section, we will
explore 1 on 1 chat flow, message synchronization across multiple devices and group chat flow.
1 on 1 chat flow
Figure 12 explains what happens when User A sends a message to User B.
1. User A sends a chat message to Chat server 1.
2. Chat server 1 obtains a message ID from the ID generator.
3. Chat server 1 sends the message to the message sync queue.
4. The message is stored in a key-value store.
5.a. If User B is online, the message is forwarded to Chat server 2 where User B is connected.
5.b. If User B is offline, a push notification is sent from push notification (PN) servers.
6. Chat server 2 forwards the message to User B. There is a persistent WebSocket connection
between User B and Chat server 2.
Message synchronization across multiple devices
Many users have multiple devices. We will explain how to sync messages across multiple
devices. Figure 13 shows an example of message synchronization.
In Figure 13, user A has two devices: a phone and a laptop. When User A logs in to the chat app
with her phone, it establishes a WebSocket connection with Chat server 1. Similarly, there is a
connection between the laptop and Chat server 1.
Each device maintains a variable called cur_max_message_id, which keeps track of the latest
message ID on the device. Messages that satisfy the following two conditions are considered as
news messages:
•
The recipient ID is equal to the currently logged-in user ID.
•
Message ID in the key-value store is larger than cur_max_message_id.
With distinct cur_max_message_id on each device, message synchronization is easy as each
device can get new messages from the KV store.
• User login
The user login flow is explained in the “Service Discovery” section. After a WebSocket
connection is built between the client and the real-time service, user A’s online status
and last_active_at timestamp are saved in the KV store. Presence indicator shows the user is
online after she logs in.
• User disconnection
We all wish our internet connection is consistent and reliable. However, that is not always the
case; thus, we must address this issue in our design. When a user disconnects from the internet,
the persistent connection between the client and server is lost. A naive way to handle user
disconnection is to mark the user as offline and change the status to online when the connection
re-establishes. However, this approach has a major flaw. It is common for users to disconnect and
reconnect to the internet frequently in a short time. For example, network connections can be on
and off while a user goes through a tunnel. Updating online status on every disconnect/reconnect
would make the presence indicator change too often, resulting in poor user experience.
We introduce a heartbeat mechanism to solve this problem. Periodically, an online client sends a
heartbeat event to presence servers. If presence servers receive a heartbeat event within a certain
time, say x seconds from the client, a user is considered as online. Otherwise, it is offline.
In Figure 18, the client sends a heartbeat event to the server every 5 seconds. After sending 3
heartbeat events, the client is disconnected and does not reconnect within x = 30 seconds (This
number is arbitrarily chosen to demonstrate the logic). The online status is changed to offline.
• Major Tech stacks used
1.
React js
ReactJS is a declarative, efficient, and flexible JavaScript library for building
reusable UI components. It is an open-source, component-based front-end library
responsible only for the view layer of the application. It was created by Jordan
Walke, who was a software engineer at Facebook. It was initially developed and
maintained
by
Facebook
and
was
later
used
in
its
products
like WhatsApp & Instagram. Facebook developed ReactJS in 2011 in its newsfeed
section, but it was released to the public in the month of May 2013.
Today, most of the websites are built using MVC (model view controller)
architecture. In MVC architecture, react is the 'V' which stands for view, whereas
the architecture is provided by the Redux or Flux.
A ReactJS application is made up of multiple components, each component
responsible for outputting a small, reusable piece of HTML code. The components
are the heart of all React applications. These Components can be nested with other
components to allow complex applications to be built of simple building blocks.
ReactJS uses virtual DOM based mechanism to fill data in HTML DOM. The
virtual DOM works fast as it only changes individual DOM elements instead of
reloading complete DOM every time.
To create React app, we write React components that correspond to various
elements. We organize these components inside higher level components which
define the application structure. For example, we take a form that consists of
many elements like input fields, labels, or buttons. We can write each element of
the form as React components, and then we combine it into a higher-level
component, i.e., the form component itself. The form components would specify
the structure of the form along with elements inside of it.
2.
Node js
Node.js is
an open-source server
environment.
Node.js
is cross-
platform and runs on Windows, Linux, Unix, and macOS. Node.js is
a back-end JavaScript runtime
environment.
Node.js
runs
on
the V8 JavaScript Engine and executes JavaScript code outside a web
browser.
Node.js lets developers use JavaScript to write command line tools and
for server-side scripting. The functionality of running scripts serverside produces dynamic web page content before the page is sent to the
user's web browser. Consequently, Node.js represents a "JavaScript
everywhere" paradigm, unifying web-application development around
a single programming language, rather than different languages for
server-side and client-side scripts.
Node.js has an event-driven architecture capable of asynchronous I/O.
These design choices aim to optimize throughput and scalability in
web applications with many input/output operations, as well as
for real-time
Web applications
(e.g., real-time
communication programs and browser games).
The Node.js distributed development project was previously governed
by the Node.js Foundation, and has now merged with the JS
Foundation to form the OpenJS Foundation. OpenJS Foundation is
facilitated by the Linux Foundation's Collaborative Projects program.
3.
Html, CSS, JavaScript
An overview:
•
HTML provides the basic structure of sites, which is enhanced
and modified by other technologies like CSS and JavaScript.
•
CSS is used to control presentation, formatting, and layout.
•
JavaScript is used to control the behaviour of different
elements.
1.
HTML
HTML stands for Hypertext Markup Language.
"Markup language" means that, rather than using a
programming language to perform functions, HTML uses tags
to identify different types of content and the purposes they each
serve to the webpage. Markup languages work in the same way
as you just did when you labelled those content types, except
they use code to do it -- specifically, they use HTML tags, also
known as "elements." These tags have pretty intuitive names:
Header tags, paragraph tags, image tags, and so on.
Every web page is made up of a bunch of these HTML tags
denoting each type of content on the page. Each type of content on the
page is "wrapped" in, i.e., surrounded by, HTML tags.
2.
CSS
CSS stands for Cascading Style Sheets. This
programming language dictates how the HTML
elements of a website should actually appear on the
frontend of the page.
HTML provides the raw tools needed to structure
content on a website. CSS, on the other hand, helps to
style this content so it appears to the user the way it was
intended to be seen. These languages are kept separate
to ensure websites are built correctly before they're
reformatted.
3.
JavaScript
JavaScript is a logic-based programming language that
can be used to modify website content and make it
behave in different ways in response to a user's actions.
Common uses for JavaScript include confirmation
boxes, calls-to-action, and adding new identities to
existing information. In short, JavaScript is a
programming language that lets web developers design
interactive sites. Most of the dynamic behaviour you'll
see on a web page is thanks to JavaScript, which
augments a browser's default controls and behaviour’s
Results and Screenshots
Conclusion and Future Enhancements
The main objective of the project is to develop a web Chat Application. I had taken a
wide range of literature review in order to achieve all the tasks, where I came to know
about some of the products that are existing in the market. I made detailed research in that
path to cover the loop holes that existing systems are facing and to eradicate them in our
application. In the process of research, I came to know about the latest technologies and
different algorithms.
The portability of the application has been achieved by using node js.
As there is always a room for improvement in any app. Right now, we are just dealing
with text communication. There are several apps in the market that have way more
functionality but we are headed to extend more functionalities to our app in the future
like,
Voice messaging
Audio messaging
Video call etc...
By making this project we have learnt a lot and wish to further increase this knowledge in
the future.
References
1. https://www.ijeat.org/wp-content/uploads/ papers/v9i5/E9578069520.pdf
2. http://indusedu.org/pdfs/IJREISS/ IJREISS_3661_55346.pdf
3. https://thescipub.com/pdf/jcssp.2015.723.729.pdf
4. https://www.ijrte.org/wp-content/uploads/ papers/v7i5s2/ES2063017519.pdf
5. https://core.ac.uk/download/pdf/187726106.pdf