What is the DOM? The Document Object Model Explained in
Plain English
D.M. Oladele
Modern web pages are dynamic. This means we need a suitable and
convenient way to modify and manipulate a web document's structure.
This modification in an HTML document, for instance, usually takes the form of creating, adding,
or removing elements in the document.
In this article, you'll learn what the Document Object Model (DOM) is, a bit about its history,
and how you use it to manipulate web documents, especially HTML documents.
What is the Document Object Model (DOM)?
The DOM is a web interface, developed and released by the World Wide Web Consortium
(W3C). This organization was founded to establish standards for the World Wide Web.
The DOM is a web API that is language-neutral. This means that you can implement and adopt it
in any programming language.
The DOM represents the structural pieces of a web document as objects that can be accessed and
manipulated. In other words, the DOM allows you as a software developer to do the following:
•
Create and build web documents.
•
Navigate the structure of web documents.
•
Add, modify, or delete elements and content within web documents.
History of the DOM
The history of the DOM is relative to JavaScript and JScript as the first widely used scripting
languages. These languages helped make web pages interactive.
Prior to the development of a standard DOM specification by the W3C, JavaScript and JScript
had different ways of enabling access to manipulating HTML documents.
These limited methods and interfaces that let you manipulate HTML documents in this way
became the DOM Level 0.
In 1998, the W3C completed its draft of the first standard DOM specification, which became the
recommended standard for all browsers. This standard DOM specification became DOM Level 1.
The DOM level 1 provided a comprehensive model for manipulating both HTML and XML
documents.
In 2000, the W3C released DOM Level 2, which introduced methods such as getElementById(),
as well as a standardized event model and support for XML namespaces and CSS.
The DOM Level 3, released in 2004, added support for XPath and keyboard event handling. And
in late 2015, the latest DOM specification, DOM Level 4, became a published standard.
What is the DOM Tree?
The structural representation created by the DOM is very much like a tree. It has several objects
in it known as nodes.
The browser uses the DOM tree representation it builds from an HTML document to determine
what to render on a web page. For example, a visual representation of a DOM tree will look like
this:
The DOM Tree
The HTML document of the above DOM tree looks like this:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>TITLE</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<div>
<h1>HELLO WORLD</h1>
</div>
<a href="link text">Document Object Model</a>
</body>
</html>
Nodes vs Elements in the DOM
Often, developers confuse nodes with elements. So we should distinguish between the two early
in this article to avoid confusion.
Nodes are all the components a web page document is made up of. In other words, a web page is
collection of nodes.
An element is a type of node within a document. For instance, the DOM
property nodes.childNodes when used on a parent node will return all the different nodes
contained within that specified parent node.
In the example below, the childNodes property is used on the <body> element node of the HTML
document given above:
//javascript content
//select the <body> element node with the DOM method querySelector
const body = document.querySelector('body')
//select the children nodes with the <body> element node with the DOM property node.childNodes
const childrenNodes = body.childNodes
//console log the children nodes
console.log(childrenNodes)//NodeList(5) [text, div, text, a, text]
Notice that there are five items in the nodeList. This is because we have another type of node, the
text nodes, different from the element nodes within the <body> element node.
To investigate this further, go through the following steps in your console:
1. Click on the dropdown icon just before "nodeList".
2. Select the text node by clicking on the dropdown icon before "test".
3. Check for the textContent option within the list options in the dropdown.
If you follow the above instructions, you will see that the test content of the first text node is
"/n ". This is a text node indicating a new line after the <body> element node, the <div> element
node, and the <a> element node.
Relationships Between Nodes in the DOM Tree
The nodes in the DOM tree have a hierarchical relationship with each other in the DOM tree.
They are defined by their position relative to each other in the DOM tree.
These are the node positions present in the DOM tree illustration above:
•
Root node: The root node is always at the apex of the DOM tree. In an HTML document, the
root node is always the <html> element.
•
•
•
•
•
Child node: A child node is a node embedded inside another node. In the illustration given
above, the <head> and the <body> elements are the children of the <html> element.
Descendant node: Any node positioned below another node in the hierarchical order is the
descendant of the nodes positioned above it. For example, although the <h1> element is not the
direct child of the <body> element, it is a descendant of the <body> and root <html> elements.
Parent node: Any node which has another node inside it is a parent node. For example,
the <body> element is the parent of the <div> and <a> elements in the above example. Note that
only element type nodes can be a parent node.
Sibling nodes: Nodes that are on the same level in hierarchical order in the DOM tree are sibling
nodes. For example, <div> and <a> elements in the above example are siblings.
Leaf nodes: The text inside of elements are leaf nodes. This is because they cannot have children
nodes of their own.
HTMLCollection vs nodeList
To manipulate the DOM tree, you need a way to select individual items or a collection of items in
it.
You can use a programming language like JavaScript to select an item or a collection of items in
the DOM tree by using a few methods provided by the DOM.
The methods getElementById() and querySelector() can select individual items. The
methods getElementsByClassName(), getElementsByTagName(), or querySelectorAll() can
select a collection of items.
In the DOM tree, we can either get an HTMLCollection or a NodeList based on the method used
to select a collection of items.
The getElementsByClassName() and getElementsByTagName() methods return
HTMLCollections, while querySelectorAll returns a nodeList.
HTMLCollection and nodeList share some similarities and differences. They're similar in the
following ways:
•
They are array-like objects.
•
They are collections of items.
•
They can be converted into an array by using the Array.from() method.
•
They both have a zero-based indexing.
•
They can both be iterated over with a for...loop.
•
They have a length property.
•
They do not have array methods available to them.
Below is a sample HTML document and JavaScript code to emphasize these similarities:
<!-- html documant -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
<ul>
<li>item one</li>
<li>item two</li>
<li>item three</li>
<li>item four</li>
</ul>
<script src="main.js"></script>
</body>
</html>
//javascript content
const listItemsHtmlCollection = document.getElementsByTagName("li")
console.log(listItemsHtmlCollection) // HTMLCollection(4) [li, li, li, li]
const listItemsNodeList = document.querySelectorAll("li")
console.log(listItemsNodeList) // NodeList(4) [li, li, li, li]
You can see from the above that while the getElementsByTagName returns an HTMLCollection
with items matching the <li> tag specified, the querySelectorAll returns a nodeList.
Now, let's use a for...loop to iterate on both collections:
for(let i = 0; i < listItemsHtmlCollection.length; i++) {
listItemsHtmlCollection[i].style.color = 'red'
}
for(let i = 0; i < listItemsHtmlCollection.length; i++) {
listItemsHtmlCollection[i].style.color = 'red'
}
In both instances the color of the text will be changed to red.
Now let's delete the for...loop iteration and use an array method on map to iterate over both
collections:
listItemsHtmlCollection.map( element => element.style.color = 'red' )
listItemsNodeList.map( element => element.style.color = 'red' )
To use the map array method successfully, you have to convert both items to an array with
the Array.from() method like this:
Array.from(listItemsHtmlCollection).map( element => element.style.color = 'red' )
Array.from(listItemsNodeList).map( element => element.style.color = 'red' )
There are two major ways in which HTMLCollection and a nodeList differ from one another.
They are:
•
•
A nodeList comes with some inbuilt methods and properties not available in an HTMLCollection.
The methods include the forEach() and the entries methods to iterate over a nodeList. The
properties include the keys property and the value property.
An HTMLCollection is always live, while a nodeList can either be live or static. A collection of
nodes is live if a change in the DOM tree updates the collection automatically. If a change in the
DOM tree does not affect the collection, then it is static. DOM changes can be the addition of a
new node or the removal of an existing node. DOM methods such
as getElementById() and getElementsByClassName() return HTMLCollections, which is always
live. The querySelectorAll() method returns a static nodeList.
DOM HTML Methods
The DOM level 1 core, Dom level 2 core, and Dom level 3 core introduced several methods that
allow web developers to manipulate the DOM tree. Some of these methods are the following:
The createElement() DOM method
The createElement() method creates
an element of the type specified as its argument.
//html document
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<ul>
<li>item one</li>
<li>item two</li>
<li>item three</li>
<li>item four</li>
</ul>
<script src="main.js"></script>
</body>
</html>
//javascript content
//select the parent element
const list = document.querySelector('ul')
//create a new element
const listItem = document.createElement('li')
//make the newly created element a child of the parent
list.appendChild('listItem')
console.log(list)
Now check the console for the list console logged. You'll see that there are five <li> elements
now within the <ul> parent element.
The createTextNode() DOM method
The createTextNode() method creates a text node with
add text to the <li> element we created above.
the string specified as its argument. Let's
//javascript content
const listText = document.createTextNode("item five")
listItem.appendChild(listText)
Now save your JavaScript file and reload your webpage.
The appendChild() DOM method
The appendChild() method adds
a node to the end of the list of children of a parent node.
If the specified child is an existing node in the document, appendChild() moves it from its
current position on the DOM tree to the new position. We used the method earlier to make our
newly created <li> element a child of the <ul> element.
The getElementById() DOM method
This method selects and returns the element whose ID is specified within it as a argument. If no
such element exists, the method returns null. Let's add an id attribute to our <ul> element in the
HTML document and give it a red border.
//html document
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<ul id="ulList">
<li>item one</li>
<li>item two</li>
<li>item three</li>
<li>item four</li>
</ul>
<script src="main.js"></script>
</body>
</html>
//javascript content
const ulList = document.getElementById("ulList")
ulList.style.border = '2px solid red'
The getElementsByClassName() DOM method
The getElementsByClassName() method selects
all elements with the specified class name and
returns them as an HTMLCollection in the order they appear on the DOM tree.
You can access individual elements in the HTMLCollection by their index number. Let's add a
class attribute to the first two <li> elements in our HTML document and change their text color
to red like this:
//html document
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<ul id="ulList">
<li class="itemOneAndTwo">item one</li>
<li class="itemOneAndTwo">item two</li>
<li>item three</li>
<li>item four</li>
</ul>
<script src="main.js"></script>
</body>
</html>
//javascript content
//select by elements one and two by their class name
const itemOneAndTwo = document.getElementsByClassName("itemOneAndTwo")
//change text color to red with use of index
itemOneAndTwo[0].style.color = 'red'
itemOneAndTwo[1].style.color = 'red'
The getElementsByTagName() DOM method
The getElementsByTagName() method returns an HTMLCollection of all the elements with the
tag name specified as its argument, in the order they appear on the DOM tree.
Let's select the <li> elements with the getElementsBytagName() method and change their font
style to italic.
//javascript content
const liTags = document.getElementsByTagName("li")
for(let i = 0; i < liTags.length; i++) {
liTags[i].style.fontStyle = 'italic'
}
The querySelector() DOM method
The querySelector() method accepts
any CSS string selector as an argument. It then uses the
specified selector to select the first within the document that matches that specified selector.
Let's change select our first two <li> elements with the querySelector() method and change
their text color back to black.
//javascript content
const querySelectItem = document.querySelector("itemOneAndTwo")
querySelectItem.style.color = 'black'
Notice that only the first list item has had its color changed to black.
The querySelectorAll() DOM method
The querySelectorAll() method, just
like the querySelector method, accepts any CSS string
selector as its argument. It then uses the specified CSS string selector to select all elements that
match that specified selector, put them in a nodeList, and return that nodeList.
Now, let's use it to change all the text in out list items to green.
//javascript content
const querySelectAllItems = document.querySelectorAll("li")
for(let i = 0; i < querySelectAllItems.length; i++) {
querySelectAllItems[i].style.color = 'green'
}
The setAttribute() DOM method
The setAttribute() method adds a new attribute name to an element. If an attribute with that
name is already present in the element, its value will change to that of the value set in the
argument.
The method accepts two arguments. The first argument is the name of the attribute you want to
create. The second argument is the value to set on the attribute, which is always a string.
Let's use it to give our third item a class attribute and change the text color to black.
//javascript content
const itemThree = querySelectAllItems[2]
itemThree.setAttribute("class", "attributeValue")
const attributeValue = document.querySelector('.attributeValue')
attributeValue.style.color = 'black'
The removeAttribute() DOM method
The removeAttribute() method removes
a specified attribute. It takes in the name of the
attribute to be removed as its argument. Let's remove the id attribute from the <ul> parent
element and use the removed id to removed its red border.
//javascript content
//remove attribute
ul.removeAttribute('id')
ul.style.border = 'none'
Now save your JavaScript file and reload your web page. Notice that the borders are still there. If
you check the console you will see an error message stating that ul is no longer defined.
The contains() DOM method
The contains() method returns
true if a node is a descendant of a node and returns false
otherwise.
<--HTML document-->
<body>
<h1>Heading</h1>
</body>
//javascript content
const body = document.querySelector('body')
const h1Element = document.querySelector('h1')
console.log( body.contains(h1Element) ) // true
The item() DOM method
The item() method returns
the item specified at the index specified as its argument when used on
a collection.
<--HTML document-->
<body>
<p>Paragraph</p>
<p>Paragraph</p>
</body>
//javascript content
const pElements = document.querySelectorAll("p")
console.log(pElements.item(0)) // <p></p>
The hasChildNodes() DOM method
The hasChildNodes method returns true if the element it is called on has child nodes within it and
returns false otherwise.
<--HTML document-->
<body>
<p>Paragraph</p>
<p>Paragraph</p>
</body>
//javascript content
const body = document.querySelector("body")
console.log(body.hasChildNodes()) // true
What are DOM Events?
To make our web page logically interactive by initiating automatic responses or incidents on the
web page, we need Events.
DOM events are:
actions or occurrences that happen in the system you are programming, which the system tells
you about so your code can react to them. (Source: MDN)
A common example of an event is when a user clicks a submit button in a form, which then
submits the data input by the user as a response to the click.
Another example is when a user clicks on a menu icon, which then triggers a drop-down
navigation or options.
You can use scripting languages such as JavaScript to register event handlers or listeners on
elements inside the DOM tree, which runs when the specified event fires.
An event handler is a:
block of code (usually a JavaScript function that you as a programmer create) that runs when the
event fires. When such a block of code is defined to run in response to an event, we say we
are registering an event handler. (Source: MDN)
Examples of events used on elements in the DOM tree include:
•
•
•
•
•
click: A click event is a mousedown or mouseup over an element on a webpage.
keypress: A keypress event occurs when keys on the keyboard are pressed.
mouseover: A mouseover event occurs when the pointing device is moved onto an element.
dblclick: A dblclick occurs when there is a double-click event over an element on a webpage.
submit: A submit event occurs when a form is submitted.
Conclusion
The DOM is the backbone of modern web dynamism. It represents every piece of a web
document as an object and provides programming languages with the necessary methods to
manipulate and modify each piece.
If you enjoyed this write-up, you should give me a shoutout.
References and Further Reading
1.
2.
3.
4.
5.
https://dom.spec.whatwg.org/
https://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core.html
https://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-html.html
https://www.w3.org/TR/DOM-Level-2-HTML/
https://www.w3.org/TR/DOM-Level-3-Core/core.html
How to Check if String is Empty in JavaScript
Learn multiple ways to easily check if a string is empty in JavaScript.
1. Comparing the String with an Empty String
To check if a string is empty in JavaScript, we can compare the string with an empty string ('') in
an if statement.
For example:
function checkIfEmpty(str) {
if (str === '') {
console.log('String is empty');
} else {
console.log('String is NOT empty');
}
}const str1 = 'not empty';
const str2 = ''; // emptycheckIfEmpty(str1); // outputs: String is NOT empty
checkIfEmpty(str2); // outputs: String is empty
To treat a string containing only whitespace as empty, call the trim() method on the string before
comparing it with an empty string.
function checkIfEmpty(str) {
if (str.trim() === '') {
console.log('String is empty');
} else {
console.log('String is NOT empty');
}
}const str1 = 'not empty';
const str2 = ''; // empty
const str3 = '
'; // contains only whitespacecheckIfEmpty(str1); // outputs: String is NOT
empty
checkIfEmpty(str2); // outputs: String is empty
checkIfEmpty(str3); // outputs: String is empty
The String trim() method removes all whitespace from the beginning and end of a string and
returns a new string, without modifying the original.
const str1 = ' bread ';
const str2 = '
milk tea
';console.log(str1.trim()); // 'bread'
console.log(str2.trim()); // 'milk tea'
Tip
Trimming a string when validating required fields in a form helps ensure that the user entered
actual data instead of just whitespace.
How to Check if a String is Empty, null, or undefined
Depending on your scenario, you might want to consider that the string could be a nullish value
(null or undefined). To check for this, use the string if statement directly, like this:
function checkIfEmpty(str) {
if (str) {
console.log('String is NOT empty');
} else {
console.log('String is empty');
}
}const str1 = 'not empty';
const str2 = ''; // empty
const str3 = null;
const str4 = undefined;checkIfEmpty(str1); // outputs: String is NOT empty
checkIfEmpty(str2); // outputs: String is empty
checkIfEmpty(str3); // outputs: String is empty
checkIfEmpty(str4); // outputs: String is empty
If the string is nullish or empty, it will be coerced to false in the if statement. Otherwise, it will be
coerced to true.
To remove all whitespace and also check for a nullish value, use the optional chaining operator (?.)
to call the trim() method on the string before using it in an if statement.
function checkIfEmpty(str) {
if (str?.trim()) {
console.log('String is NOT empty');
} else {
console.log('String is empty');
}
}const str1 = 'not empty';
const str2 = ''; // empty
const str3 = null;
const str4 = undefined;
const str5 = '
'; // contains only whitespacecheckIfEmpty(str1); // outputs: String is NOT
empty
checkIfEmpty(str2); // outputs: String is empty
checkIfEmpty(str3); // outputs: String is empty
checkIfEmpty(str4); // outputs: String is empty
checkIfEmpty(str5); // outputs: String is empty
The optional chaining operator lets us call the trim() method on a null or undefined string without
causing an error. Instead, it prevents the method call and returns undefined.
const str1 = null;
const str2 = undefined;console.log(str1?.trim()); // undefined
console.log(str2?.trim()); // undefined
2. Comparing the Length of the String with 0
Alternatively, we can access the length property of a string and compare its value with 0 to check if
the string is empty.
function checkIfEmpty(str) {
if (str.length === 0) {
console.log('String is empty');
} else {
console.log('String is NOT empty');
}
}const str1 = 'not empty';
const str2 = ''; // emptycheckIfEmpty(str1); // outputs: String is NOT empty
checkIfEmpty(str2); // outputs: String is empty
To check for strings containing only whitespace with this approach, we would also call
the trim() method before comparing the length of the trimmed string with 0.
function checkIfEmpty(str) {
if (str.trim().length === 0) {
console.log('String is empty');
} else {
console.log('String is NOT empty');
}
}const str1 = 'not empty';
const str2 = ''; // empty
const str3 = '
'; // contains only whitespacecheckIfEmpty(str1); // outputs: String is NOT
empty
checkIfEmpty(str2); // outputs: String is empty
checkIfEmpty(str3); // outputs: String is empty
Updated at: codingbeautydev.com
Every Crazy Thing JavaScript Does
A captivating guide to the subtle caveats and lesser-known parts of JavaScript.
Sign up and receive a free copy immediately.
7 Console Methods Used by Pros
Often while debugging, beginners use the console.log() method to print out values. But there are
a few other console methods that make your life much easier. Want to know what these
methods are? Let's dive in!
1. console.table()
Logging matrixes or even long arrays or objects is a headache using
the console.log() method. console.table() is a much more elegant way to do it.
// Matrix
console.table([
["apple", "banana", "cherry"],
["Rs 80/kg", "Rs 100/kg", "Rs 120/kg"],
["5 ⭐", "4 ⭐", "4.5 ⭐"],
]);// Maps
class Person {
constructor(firstName, lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
}const family = {};
family.mother = new Person("Jane", "Smith");
family.father = new Person("John", "Smith");
family.daughter = new Person("Emily", "Smith");console.table(family);
2. console.trace()
Are you having issues debugging a function? Left wondering how the execution
flows? console.trace() is your friend!
function outerFunction() {
function innerFunction() {
console.trace();
} innerFunction();
}outerFunction();
3. console.error() and console.warn()
Tired of boring logs? Spice things up with console.error() and console.warn().
console.error("This is an error message");
console.warn("This is a warning message");
console.log("This is a log message");
4. console.assert()
This is another brilliant tool for debugging! If the assertion fails, the console will print out
the trace.
function func() {
const a = -1;
console.assert(a === -1, "a is not equal to -1");
console.assert(a >= 0, "a is negative");
}func();
5. console.count() and console.countReset()
Yet another incredible debugging tool! console.count() will print out the number of times it is
executed.
function fibonacci(num) {
console.count("fibonacci");
if (num < 2) {
return num;
}
return fibonacci(num - 1) + fibonacci(num - 2);
}fibonacci(2);console.countReset("fibonacci");
console.log("COUNTER RESET");fibonacci(5);
6. console.time(), console.timeEnd(), and console.timeLog()
Need to check how long something takes? The timer methods are there to rescue you!
console.time("timeout-timer");setTimeout(() => {
console.timeEnd("timeout-timer");
}, 1000);setTimeout(() => {
console.timeLog("timeout-timer");
}, 500);
NOTE: The setTimeouts are not executed immediately, resulting in a small deviation from the
expected time.
7. console.clear()
After logging so much to the console, of course, you would need to clear it up for further use.
The console.clear() method is the way to go!
console.log("Some random text");
console.clear();
That’s all folks! Hope this helps you become a better, well-rounded developer!
Research says, writing down your goals on pen & paper makes you 21% to 39% more likely
to achieve them. Check out these notebooks and journals to make the journey of achieving
your dreams easier: https://www.amazon.com/Tapajyoti-Bose/e/B09VGDDHRR
Follow me for weekly new tidbits on the domain of tech!
Need a Top Rated Front-End Development Freelancer to chop away your development
woes? Contact me on Upwork
Want to see what I am working on? Check out my Personal Website and GitHub
Want to connect? Reach out to me on LinkedIn
Follow me on Instagram to check out what I am up to recently.
ES6 way to clone an array
Hello Folks
What’s up friends, this is SnowBit here. I am a young, passionate and self-taught developer and
have an intention to become a successful developer.
I hope you enjoy reading this article.
In the olden days, when ES6 was not introduced, we often use the slice() method to clone an
array. Now it's the time for ES6, you can use the spread operator to clone an array. It looks pretty
neat and right.
const ducks = [" ", " ", " ", " "]
// Old way
const ducksClone = ducks.slice()
// ES6 way
const ducksCloneES6 = [...ducks]
This is how you clone an array with ES6.
But your crazy mind would have wondered…
Why can’t I use = to clone an array?
This is because the array in JavaScript is only referenced values so when you put = and try to clone
an array will only copy the reference of the original array to a variable and not an array.
const ducks = [" ", " ", " "]
const pirateDucks = ducks
const cloneDucks = [...ducks]
console.log(ducks === cloneDucks)
// true -> same memory space
console.log(ducks === pirateDucks)
// false -> new memory space
Some problems arise when using = to clone array
In Javascript, arrays are mutable i.e their state can be modified. So, this might happen when
using =
const ducks = [" ", " ", " "]
const pirateDucks = ducks
pirateDucks.push(" ☠ ")
console.log(pirateDucks)
// [" ", " ", " ", " ☠ "]
console.log(ducks)
// [" ", " ", " ", " ☠ "] - Original duck array values got changed
Thank you for reading, have a nice day! Your appreciation is my motivation
•
Blog — Check out my blog
•
Follow me on Twitter — @codewithsnowbit
•
Subscribe to me on YouTube — Code With SnowBit
JavaScript Interview Questions that made me think — Do
I really know JavaScript?
Photo by Rahul Mishra on Unsplash
The best part of JavaScript which always amaze me is that there is always something new
happening around it. And no matter how much you know about it, you will always get to learn new
things about it.
I’ve collected these questions over a period of quite long time. In most of the questions I really had
no idea about what the output would be, until I tried them by myself.
So here I am documenting them so that other people can make use of it to learn new concepts:
.
.
.
Output:
21
{name: “Lydia”} // age won’t be included. Because property defined with defineProperty are
non enumerable by default.
.
.
.
Output:
false // delete operator only deletes a key in object
true // when we don't use any declaration before any variable, it will be treated as a global
variable, and will be added as deletable entity in window object.undefined
.
.
.
Output:
[ { name: "Noren Red"} ] Initially I thought it will log [ null ] because we have initialised
person with nullBut in reality, we are only setting new reference to person variable.
Previous reference will be used in members arrayIn Short, { name: "Noren Red"} lives in some
memory space whose address is X201and this is how referencing is workinglet person = X201
const members = [ X201 ]
person = null
.
.
.
Output
Silver Surfer Because when we return a property, the value of the property is equal to the
returned value, not the value set in the constructor function.
Output:
With the padStart method, we can add padding to the beginning of a string.
The value passed to this method is the total length of the string together with the padding.
The string "Silver Surfer" has a length of 13. name.padStart(14) inserts 1 space at the start
of the string, because 13 + 1 is 13. If the argument passed to the padStart method is smaller
than the length of the array, no padding will be added.
.
.
.
Output
777If we pass string and number combination to parseInt, what parseInt does is, it check at
which position wrong datatype is getting started, if value before the wrong datatype is a
valid number, it will return the valid number.
.
.
.
Output:
1 2 and undefined 3 and undefined 4
If we don't pass initial value, then by default x will be first value of array, and y will be
second value of array.
.
.
.
Output:
one - ["", " is ", " years old"]
two - Thor
three - 1000one - ["hey there, are you amazed"]
two - undefined
three - undefinedIf we use tagged template literals, the value of the first argument is
always an array of the string values.The remaining arguments get the values of the passed
expressions!
.
.
.
Output:
1
undefined
2
.
.
.
Output
function // Classes in JS are functions under the hood
kehey74, CC BY-SA 4.0, via Wikimedia Commons
Using the at() Method in JavaScript
Quickly Get the Last Item of an Array
Connect with us. Want to hear what’s new at The Pragmatic Bookshelf? Sign up for our
newsletter. You’ll be the first to know about author speaking engagements, books in beta, new
books in print, and promo codes that give you discounts of up to 40%.
A programmer’s daily tasks often involve making, modifying, or returning arrays. These simple yet
essential operations play an important role in JavaScript applications. That’s why ECMAScript
keeps adding cool new array methods such as flat() and flatMap() to make working with arrays
easier.
A recently introduced feature that hasn’t received the attention it deserves is the at() method —
mostly because it’s already possible to get the same result using the square bracket notation ([]).
Consider the following example:
const arr = ['first', 'second', 'last'];console.log(arr[0]);
console.log(arr.at(0));
// => first
// => first
As with [], at() returns the array item at the given index. But, when it comes to retrieving the last
item in an array, at() is simpler to use. Rather than having to read the length property and
subtracting one from it, we can pass -1. Compare:
const arr = ['first', 'second', 'last'];console.log(arr[arr.length-1]);
console.log(arr.at(-1));
// => last
// => last
When supplying at() with a negative number, it counts back from the end of the array, so if we
pass -2 it gets us the array’s second to last item. If there’s no item at the given index, the return
value will be undefined.
When you need to retrieve the last item in an array consider using at().
If you enjoyed this article, you may also like Faraz Kelhini’s book, Modern Asynchronous
JavaScript. Through March 31, 2022, you can use promo code fkajs_medium_35 to save 35%
on the ebook version. Promo codes are not valid on prior purchases.
Photo by Blake Connally on Unsplash
Debouncing and Throttling in Javascript
Debouncing and throttling are both used to enhance the website performance by limiting the
number of times the events are triggered. Debouncing and throttling are not provided by
JavaScript. They’re just concepts that can be implemented using the setTimeout web API.
What is Debouncing
Let’s take an example. You have opened an e-commerce website to search for laptop bags.
If debounce is not applied you can see in the below image the number of calls is made on every
keystroke.
After implementing debounce, we have significantly reduced the number of calls. Now calls are
made only when the user types again after the specified time. The function will be executed only
when the time taken by the user to type again is equal to the delay that we have provided.
Custom debounce function
What is throttling
Throttling is also used to rate-limit the function call. Throttling will fire the function call only once
in 1000ms(the limit which we have provided), no matter how many times the user fires the
function call.
Custom Throttling function
Conclusion
I hope after reading this article these two concepts by javascript are cleared.
Throttling and debouncing can be implemented to enhance the searching functionality, infinite
scroll, and resizing of the window.
In case I have missed something or you have some advice or suggestions do let me know in
the comment section.
You can also reach out to me
https://www.linkedin.com/in/akhatun/
https://github.com/amnahkhatun
Happy coding✌
Why do we need Streams in Node.js?
Photo by Snejina Nikolova on Unsplash
Streams are sequences of data made available over time. The difference with other types of data
like strings or arrays is that streams might not be available all at once, and they don’t have to fit in
memory.
Many of the built-in modules in Node implement the streaming interface, such as HTTP response,
HTTP request, fs, process.stdin, etc.
Let’s see how Stream solves our slow/blocking web server problem.
Assume we need to serve a big file using a Node web server.
mkdir blog-why-node-streams
cd blog-why-node-streams
echo "" > index.js
Update index.js with the below code,
const fs = require("fs");
const http = require("http");// code to generate random big sized file on fly
fs.stat("big.file", function (err, stat) {
if (err == null) {
console.log("File exists");
} else if (err.code === "ENOENT") {
const file = fs.createWriteStream("./big.file");
for (let i = 0; i <= 1e6; i++) {
file.write(
`Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur tempus id metus a
sodales. Maecenas faucibus bibendum mauris elementum ultrices. In hac habitasse platea
dictumst. Pellentesque consequat augue nec urna interdum, a sagittis arcu ornare. Duis
pulvinar odio vitae velit euismod, nec pretium nisi tempus. Lorem ipsum dolor sit amet,
consectetur adipiscing elit. Cras ante lorem, suscipit non lobortis venenatis, interdum a
dui. Donec rhoncus magna lectus, ut vestibulum eros rutrum gravida. Aenean sit amet fringilla
erat. In varius fermentum justo, in maximus sapien tempus non. Sed malesuada tempor erat eget
tristique. Pellentesque diam nulla, pharetra sed luctus nec, euismod non tortor.`
);
}
console.log("big.file created");
file.end();
} else {
console.log("Some other error: ", err.code);
}
});const server = http.createServer();server.on("request", (req, res) => {
fs.readFile("./big.file", (err, data) => {
if (err) throw err;
res.end(data);
});
});server.listen(8000, () => console.log("The server is running at localhost:8000"));
The above code does two things:
1. The first part of the code is to generate a huge (~600 MB) file. A utility code.
2. The second part is a simple web server endpoint serving the big.file file.
Let’s run the server.
> node index.jsThe server is running at localhost:8000
big.file created
After starting the node server, let’s see the memory usage using the Windows task manager. We
have ~5.8 MB of memory consumed by our server.
Node server memory consumption before serving the file
Now let’s curl the endpoint to download the file.
> curl localhost:8000Lorem ipsum dolor sit amet, consectetur adipiscing e......
............................
.......................
Now, look at the memory consumption for the server using task manager.
Node server memory consumption while serving the file
When we run the server, it starts out with a normal amount of memory, ~5.8 MB. Then we
connected to the server. Note what happened to the memory consumed. The memory
consumption jumped to ~684 MB.
How does it work?
We basically put the whole big.file content in memory before we wrote it out to the response object.
This is very inefficient.
Solution,
The HTTP response object (res in the code above) is also a writable stream. This means that if we
have a readable stream that represents the content of big.file, we can simply pipe those two
together and get nearly the same result without consuming ~ 684 MB of memory.
Node’s fs module can give us a readable stream for any file using the createReadStream method. We
can pipe that to the response object.
So, replace the request handler code with the below code snippet and measure the memory
consumption.
server.on("request", (req, res) => {
const src = fs.createReadStream("./big.file");
src.pipe(res);
});
Let’s run our server again,
> node index.jsThe server is running at localhost:8000
big.file created
Now, let’s curl the endpoint,
> curl localhost:8000
> curl localhost:8000Lorem ipsum dolor sit amet, consectetur adipiscing e......
............................
.......................
Now, look at the memory consumption for the server in the task manager.
Node server memory consumption while serving the file with streaming chunks
When we ran the server, it started out with a normal amount of memory, ~ 5.8 MB. Then we
connected to the server (curl). Note what happened to the memory consumed. The memory
consumption is just ~8 MB.
Now, what’s changed, and how is it working?
When a client asks for that big file, we stream it one chunk at a time, which means we
don’t buffer it in memory at all. The memory usage grew by about ~8 MB and that’s it.
These scenarios need not be for just an HTTP server; they may be applicable to cases such as file
content manipulation, big file creation, uploading files from client to server, or sending big audio
or video file to a client, etc.
Error Handling in Node.js Like a Pro
All you need to know to get started.
Photo by ThisisEngineering RAEng on Unsplash
Handling errors are one of the most important aspects of any production-grade application.
Anyone can code for the success cases. Only true professionals take care of the error cases.
Today we will learn just that. Let’s dive in.
First, we have to understand that not all errors are the same. Let’s see how many types of errors
can occur in an application.
•
User Generated Error
•
Hardware failure
•
Runtime Error
•
Database Error
We will see how we can easily handle these different types of errors.
This article is part of a series where I am building a ExpressJSBoilerplate from Scratch. You
can check that here
Get a basic express application
Run the following command to get a basic express application built with typescript.
git clone https://github.com/Mohammad-Faisal/express-typescript-skeleton.git
Handle not found URL errors
How do you detect if a hit URL is not active in your express application? You have an URL
like /users, but someone is hitting /user. We need to inform them that the URL they are trying to
access does not exist.
That’s easy to do in ExpressJS. After you define all the routes, add the following code to catch all
unmatched routes and send back a proper error response.
app.use("*", (req: Request, res: Response) => {
const err = Error(`Requested path ${req.path} not found`);
res.status(404).send({
success: false,
message: "Requested path ${req.path} not found",
stack: err.stack,
});
});
Here we are using “*” as a wildcard to catch all routes that didn’t go through our application.
Handle all errors with a special middleware
Now we have a special middleware in Express that handles all the errors for us. We have to include
it at the end of all the routes and pass down all the errors from the top level so that this
middleware can handle them for us.
The most important thing to do is keep this middleware after all other middleware and route
definitions because otherwise, some errors will slip away.
Let’s add it to our index file.
app.use((err: Error, req: Request, res: Response, next: NextFunction) => {
const statusCode = 500;
res.status(statusCode).send({
success: false,
message: err.message,
stack: err.stack,
});
});
Have a look at the middleware signature. Unline other middleware, This special middleware has
an extra parameter named err, which is of the Error type. This comes as the first parameter.
And modify our previous code to pass down the error like the following.
app.use("*", (req: Request, res: Response, next: NextFunction) => {
const err = Error(`Requested path ${req.path} not found`);
next(err);
});
Now, if we hit a random URL, something like, http://localhost:3001/posta, then we will get a
proper error response with the stack.
{
"success": false,
"message": "Requested path ${req.path} not found",
"stack": "Error: Requested path / not found\n
at
/Users/mohammadfaisal/Documents/learning/express-typescript-skeleton/src/index.ts:23:15\n"
}
Custom error object
Let’s have a closer look at the default error object provided by Node.js.
interface Error {
name: string;
message: string;
stack?: string;
}
So when you are throwing an error like the following.
throw new Error("Some message");
Then you are only getting the name and the optional stack properties with it. This stack provides
us with info on where exactly the error was produced. We don't want to include it in production.
We will see how to do that later.
But we may want to add some more information to the error object itself.
Also, we may want to differentiate between various error objects.
Let’s design a basic Custom error class for our application.
export class ApiError extends Error {
statusCode: number;
constructor(statusCode: number, message: string) {
super(message);
this.statusCode = statusCode;
Error.captureStackTrace(this, this.constructor);
}
}
Notice the following line.
Error.captureStackTrace(this, this.constructor);
It helps to capture the stack trace of the error from anywhere in the application.
In this simple class, we can append the statusCode as well. Let's modify our previous code like the
following.
app.use("*", (req: Request, res: Response, next: NextFunction) => {
const err = new ApiError(404, `Requested path ${req.path} not found`);
next(err);
});
And take advantage of the new statusCode property in the error handler middleware as well
app.use((err: ApiError, req: Request, res: Response, next: NextFunction) => {
const statusCode = err.statusCode || 500; // <- Look here res.status(statusCode).send({
success: false,
message: err.message,
stack: err.stack,
});
});
Having a custom-defined Error class makes your API predictable for end users to use. Most
newbies miss this part.
Let’s handle application errors
Now let’s throw a custom error from inside our routes as well.
app.get("/protected", async (req: Request, res: Response, next: NextFunction) => {
try {
throw new ApiError(401, "You are not authorized to access this!"); // <- fake error
} catch (err) {
next(err);
}
});
This is an artificially created situation where we need to throw an error. The real life, we may have
many situations where we need to use this kind of try/catch block to catch errors.
If we hit the following URL http://localhost:3001/protected, we will get the following response.
{
"success": false,
"message": "You are not authorized to access this!",
"stack": "Some details"
}
So our error response is working correctly!
Let’s improve on this!
So we now can handle our custom errors from anywhere in the application. But it requires a try
catch block everywhere and requires calling the next function with the error object.
This is not ideal. It will make our code look bad in no time.
Let’s create a custom wrapper function that will capture all the errors and call the next function
from a central place.
Let’s create a wrapper utility for this purpose!
import { Request, Response, NextFunction } from "express";export const asyncWrapper = (fn:
any) => (req: Request, res: Response, next: NextFunction) => {
Promise.resolve(fn(req, res, next)).catch((err) => next(err));
};
And use it inside our router.
import { asyncWrapper } from "./utils/asyncWrapper";app.get(
"/protected",
asyncWrapper(async (req: Request, res: Response) => {
throw new ApiError(401, "You are not authorized to access this!");
})
);
Run the code and see that we have the same results. This helps us to get rid of all try/catch blocks
and call the next function everywhere!
Example of a custom error
We can fine-tune our errors to our needs. Let’s create a new error class for the not found routes.
export class NotFoundError extends ApiError {
constructor(path: string) {
super(404, `The requested path ${path} not found!`);
}
}
And simplify our bad route handler.
app.use((req: Request, res: Response, next: NextFunction) => next(new
NotFoundError(req.path)));
How clean is that?
Now let’s install a small little package to avoid writing the status codes ourselves.
yarn add http-status-codes
And add the status code in a meaningful way.
export class NotFoundError extends ApiError {
constructor(path: string) {
super(StatusCodes.NOT_FOUND, `The requested path ${path} not found!`);
}
}
And inside our route like this.
app.get(
"/protected",
asyncWrapper(async (req: Request, res: Response) => {
throw new ApiError(StatusCodes.UNAUTHORIZED, "You are not authorized to access this!");
})
);
It just makes our code a bit better.
Handle programmer errors.
The best way to deal with programmer errors is to restart gracefully. Place the following line of
code at the end of your application. It will be invoked in case something is not caught in the error
middleware.
process.on("uncaughtException", (err: Error) => {
console.log(err.name, err.message);
console.log("UNCAUGHT EXCEPTION!
Shutting down...");
});
process.exit(1);
Handle unhandled promise rejections.
We can log the reason for the promise rejection. These errors never make it to our express error
handler. For Example, if we want to access a database with the wrong password.
process.on("unhandledRejection", (reason: Error, promise: Promise<any>) => {
console.log(reason.name, reason.message);
console.log("UNHANDLED REJECTION!
Shutting down...");
process.exit(1);
throw reason;
});
Further improvement
Let’s create a new ErrorHandler class to handle the errors in a central place.
import { Request, Response, NextFunction } from "express";
import { ApiError } from "./ApiError";export default class ErrorHandler {
static handle = () => {
return async (err: ApiError, req: Request, res: Response, next: NextFunction) => {
const statusCode = err.statusCode || 500;
res.status(statusCode).send({
success: false,
message: err.message,
rawErrors: err.rawErrors ?? [],
stack: err.stack,
});
};
};
}
This is just a simple error handler middleware. You can add your custom logic here. And use it
inside our index file.
app.use(ErrorHandler.handle());
That’s how we can separate the concerns by respecting the single responsibility principle of
SOLID.
I hope you learned something new today. Have a wonderful rest of your day!
Want to Connect?You can reach out to me via LinkedIN or my Personal Website
Is it time to ditch Svelte, React, and VUE?
Almost every modern web application built these days starts with an enormous clusterf*ck of
JavaScript on the front-end which literally replaces the entire browser viewport with a JSrendered virtual DOM and consumes JSON via a REST API which is built as a separate (but tightly
coupled) application. Sounds kinda crazy, right? Spoiler alert: That’s because it is totally f*cking
crazy!
If you’re building a Single Page Application (SPA) like maybe the next Figma or Trello, then one of
those tools might fit the bill perfectly. But if you’re building a Multi Page Application (MPA) like a
typical e-commerce website or even something like Gmail, I’m here to tell you that using a SPA
framework is likely adding far more complexity than it’s worth.
The trouble with SPA architecture
Using the server only as a “dumb” API means we can no longer easily rely on it to maintain our
application state. So we’ve moved all that state management to the client, inspiring a whole new
category of frameworks like Redux and MobX. And since we can no longer use the server for basic
routing, new libraries like React Router and Page.js were created to simulate the natural routing
functionality we used to get for free.
Authentication used to be trivially easy to implement with server-side sessions. With SPA
architecture, we typically use JSON Web Tokens which are far more difficult to implement (and far
easier to implement badly). Even basic form submission can no longer rely on the browser’s
standard implementation of HTML to submit form fields based on their name attributes. We're now
required to bind those values to a JS object and manage and submit that object "manually".
In other words, all this stuff we used to get for free, now requires quite a lot of extra work. But is it
worth it?
How did we get here?
In the olden days, the web was simple. Your browser sent an HTTP request, the server sent a new
document, and your browser dutifully rendered it to the viewport blowing away whatever was
there before. This was a bit clunky, though. It meant that if you wanted to update just one little
piece of the page, you had to re-render the entire thing. Then JQuery came along which made it
relatively simple to update only parts of the page using AJAX without a full-page refresh and to
build web applications which felt far more interactive and responsive — more “app-like”. But it
involved a lot of imperative JavaScript and was hard to maintain. If you wanted to make
something moderately complex, it didn’t take long before you had an unmaintainable rat’s nest of
JQuery.
Then along came Angular, followed by React and friends with a radical new approach: What if we
re-think the whole concept of a “front-end” not as a DOM sprinkled with JavaScript — but rather
as a JavaScript application which ultimately renders a DOM. Let’s turn it upside down! And it
worked brilliantly if what you wanted to build was a Single Page App. Sure you lost a lot
of the simplicity of a basic client/server architecture with HTML on the wire. But it freed you to
build a truly app-like frontend experience. This new approach was exciting — almost intoxicating.
And before long, every greenfield project looked like a good candidate for SPA.
But user expectations for a modern, reactive website have also increased dramatically over the past
5 or 10 years. So building a “web 1.0” style application with full page reloads just won’t cut it
anymore.
Modern UI without SPA
So how can we build a modern MPA website without using a SPA-frontend / REST-backend
architecture, without writing 80,000 lines of crufty JQuery, and without a janky full-page refresh
on every click like it was built circa 1999?
There’s a new crop of libraries designed to provide modern interactivity while working with the
grain of HTML and HTTP — both of which start with HT for Hypertext. This is key. The web was
designed with the idea of Hypertext going up and down the wire. Not JSON. New libraries like
Hotwire, HTMX, and Unpoly allow you to swap out chunks of your DOM in a declarative way by
adding HTML attibutes or tags to your markup — without writing any JavaScript yourself. For
example, an “Add to Cart” button could send a request to the server which modifies the serverside state of the cart items in your server-side session, then send back two chunks of DOM which
replace only the #cart-sidebar and #cart-icon-badge on the page. This can be done quite elegantly
and with beautiful CSS animations too.
When we send HTML down the wire as God (aka, Tim Berners-Lee) intended, it turns out there’s a
ton of stupid shit we no longer need. Things like client-side state management — the DOM is the
client-side state. Client-side routers? Don’t be ridiculous. JSON Web Tokens? Server sessions are
tried and true — and so much easier to implement. Our database queries become very easy too
since we’re writing all our routes on the server-side where we already have secure, direct access to
the database.
I wrote a simple ExpressJS-based framework to implement this style of architecture which you can
check out here: https://www.sanejs.dev
Ruby on Rails Shines in 2022 (no, really!)
Like most modern web developers, I’ve long shunned Ruby on Rails as a legacy framework
designed to build a style of monolithic web application which is no longer even relevant. But here’s
the thing: If we’re using something like Hotwire or HTMX on the frontend, we can use anything
we want for the backend. Since we’re working with the grain of HTML, ideally, we want the very
best system for creating server-rendered templates. There really aren’t that many full-featured,
batteries-included frameworks out there. The big ones are Rails, Django, and Laravel. There are a
few others up and coming such as Phoenix based on Elixer and Buffalo based on Go. But Rails has
a huge community, is very polished, and is honestly just a joy to work with.
But crucially, the latest Rails 7.0 released last December includes the incredible new Hotwire
library for frontend interactivity. Hotwire can be used with or without Rails — but it’s designed to
pair perfectly with Rails development and is baked-in by default. So believe it or not, in 2022, Rails
may now be the perfect full-stack framework for building post-jamstack era MPA web applications
with modern interactivity that works with the grain of HTML rather than replacing it wholesale
with the clusterf*ck of JS we’ve come to expect on the front-end plus a whole ‘nuther app for the
backend API.
Wrapping it all up
If the ultimate goal is to build modern MPA websites in a way that is fast, organized, and
maintainable, then it’s worth seriously considering whether SPA/Jamstack architecture is really
the right tool for the job. With the arrival of modern DOM-swapping interactivity libraries like
Hotwire, HTMX, and Unpoly, we finally have a real, practical alternative to SPA which allows us to
create modern, elegant interfaces that work with the grain of HTML meaning we don’t have to
reinvent the wheel for basic things like application state management and form submission. So if
we’re going back to server-rendered templates, then maybe it’s time to take another look at the
reigning all-time champion of web frameworks, Ruby on Rails. Especially now that the brand new
7.0 release comes with Hotwire baked-in, Rails just might be the very best solution in 2022 for
building modern Multi Page Applications.
Kafka with Node.Js
Photo by Erlend Ekseth on Unsplash
Kafka is one of the very efficient and popular event/message steaming platforms for building
microservices-based software applications, Kafka can work as the backbone of your microservices
so that different microservices can communicate with each other in an asynchronous fashion.
Integration with Node.JS — We are going to create a simple node.js application to consume
data from Wikimedia event stream API and put it inside a Kafka Topic using a producer and then
read the data from Kafka and store it inside ElasticSearch. below is the HLD of the sample
application
HLD of project
To work with Kafka we will be using the NPM module KafkaJS and to listen to the event stream we
will use eventsource module so let's get started
First, we will create a node project and install npm modules
npm i eventsource kafkajs
Our Project is ready now let's start with listening to the Wikimedia stream
import {
Kafka }
from
'kafkajs';
var EventSource = require('eventsource');
var es = new EventSource('https://stream.wikimedia.org/v2/stream/recentchange');
es.on('message', async (data: any, err: any) => {
const payload = JSON.parse(data.data);
console.log('Received Data: ', payload);
});
the above code will listen to the Wikimedia stream and then when a message is received it will log
in to the console, next step is to send the event inside Kafka so let's write our producer
import {
Kafka,
Producer }
from
'kafkajs';
export const getProducer = async () => {
const kafka = new Kafka({
clientId: 'producer-client',
brokers: ['localhost:9092'],
});
const producer: Producer = kafka.producer();
await producer.connect();
return producer;
};
We can use the above producer to send a message to Kafka using the following code, we will use it
inside our main file
await producer.send({
topic: 'wikimedia.recentchanges',
messages: [{ value: JSON.stringify(payload) }],
});
With producer setup being done now its time to build our consumer which will consume the
message publisher on ‘wikimedia.recentchanges’ topic from Kafka
import { ConsumerSubscribeTopic, Kafka } from 'kafkajs';
export const getConsumer = async () => {
const kafka = new Kafka({
clientId: 'consumer-client',
brokers: ['localhost:9092'],
});
const consumer = kafka.consumer({ groupId: 'my-group' });
const subscription: ConsumerSubscribeTopic = {
topic: 'wikimedia.recentchanges',
fromBeginning: false,
};
await consumer.connect();
await consumer.subscribe(subscription);
return consumer;
};
Now our producer and consumer are ready let's create an index file to run both producer and
consumer
import {
EachMessagePayload,
Message } from
'kafkajs';
import { getConsumer } from './consumer';
import { getProducer } from './producer';
var EventSource = require('eventsource');
var es = new EventSource('https://stream.wikimedia.org/v2/stream/recentchange');
const start = async () => {
const producer = await getProducer();
es.on('message', async (data: any, err: any) => {
const payload = JSON.parse(data.data);
console.log('Received Data: ', payload);
//publish the message to Kafka
await producer.send({
topic: 'wikimedia.recentchanges',
messages: [{ value: JSON.stringify(payload) }],
});
});
// Start of Consumer
const consumer = await getConsumer();
await consumer.run({
eachBatchAutoResolve: false,
eachMessage: async (messagePayload: EachMessagePayload) => {
const { topic, partition, message } = messagePayload;
const prefix = `${topic}[${partition} | ${message.offset}] / ${message.timestamp}`;
console.log(message);
},
});
};
start();
if you run our main file you will see that we are successfully able to send and consume messages to
and from Kafka, now let’s save the message inside ElasticSearch.
To work with elastic search we will be using the npm module @elastic/elasticsearch let's change
our main file to save Wikimedia changes to elastic search
import {
EachMessagePayload
} from 'kafkajs';
const { Client } = require('@elastic/elasticsearch');
import { getConsumer } from './consumer';
import { getProducer } from './producer';
var EventSource = require('eventsource');
var es = new EventSource('https://stream.wikimedia.org/v2/stream/recentchange');
const start = async () => {
const producer = await getProducer();
es.on('message', async (data: any, err: any) => {
const payload = JSON.parse(data.data);
console.log('Received Data: ', payload);
//publish the message to Kafka
await producer.send({
topic: 'wikimedia.recentchanges',
messages: [{ value: JSON.stringify(payload) }],
});
});
//Create the elastic search client
const elsticClient = new Client({
node: 'http://localhost:9200',
});
// Start of Consumer
const consumer = await getConsumer();
await consumer.run({
eachBatchAutoResolve: false,
eachMessage: async (messagePayload: EachMessagePayload) => {
const { topic, partition, message } = messagePayload;
const prefix = `${topic}[${partition} | ${message.offset}] / ${message.timestamp}`;
console.log(message);
// Publish the message to elastic
await elsticClient.index({
index: 'wikimedia_recentchanges',
document: message,
});
},
});
};
start();
Now if you run the main file and open Kibana you will see the documents are getting indexed
inside ElasticSearch index ‘wikimedia_recentchanges’
The complete source code of the project is available on Github.
Thanks for reading! I hope you find this article useful.
Node JS Event Loop And Custom Event
Node JS executes in a single process and single thread only, but the execution performance is very
high because of event-driven design and callback function usage. This article will introduce the
Node JS event loop mechanism and tell you how to create custom events in Node JS by example.
1. Node JS Event Loop Introduction
1. When you execute an asynchronous method provided by the Node JS framework, you should
provide a callback function to the method. You can continue to execute other js codes without
having to wait for the method to complete.
2. The callback function is an observer of the event queue. When the task is complete, the Node
JS framework will add the event back to the event queue, then the callback function which just
observes the event in the queue will be invoked to process the result.
3. An example of this design pattern is Node JS implemented HTTP web server. The HTTP web
server starts and listens for requests. When a request arrives, it will create an event in the queue
and then waiting for the next request.
4. When the previous request process is complete, the callback function will be invoked to send the
response back to the client.
5. This pattern is very high performance because web servers just waiting for requests and do not
do other things like IO operation which will cost time and resources.
2. Create And Listen Custom Event In Node JS
1. If you find Node JS built-in event is not enough for you, you can create a custom event and
bind the custom event process function with the event follow the below steps.
2. Include Node JS events built-in module.
var events = require('events');
3. Create an EventEmitter object use the above module
var event_emitter = new events.EventEmitter();
4. Create a JavaScript function as a callback function that will be triggered when the custom event
happen.
var data_receive_handler = function(){
console.log('Data received.');
}
5. Register custom events with the callback function. The callback function will observe the custom
event.
event_emitter.on('receive_data', data_receive_handler);
6. Trigger custom event then the callback function will be executed.
event_emitter.emit('receive_data');
3. Node JS Custom Event Example
1. custom-event.js
// Include node js prebuilt events module.
var events = require('events');
// Create event emitter object.
var event_emitter = new events.EventEmitter();
// Create connect event process function.
var connect_handler = function connected() {
console.log('Connect success.');
// Trigger receive_data event
event_emitter.emit('receive_data');
}
// Bind custom event connect with connect_handler process function.
event_emitter.on('connect', connect_handler);
// Create receive data event process function.
var data_receive_handler = function(){
console.log('Data received.');
}
// Bind custom event receive_data with data_receive_handler function.
event_emitter.on('receive_data', data_receive_handler);
// Trigger connect event.
event_emitter.emit('connect');
console.log("Code exit");
2. Run the below command to execute the above javascript source code in Node Js
$ node custom-event.js
3. Below is the above example output
Connect success.Data received.Code exit
So this was all in “Node JS Event Loop And Custom Event” ,I will be coming with more such
articles in the future.
Till then follow me for more !
2