Enterprise Automation with Python Automate Excel, Web, Documents, Emails, and Various Workloads with Easy-to-code Python Scripts Ambuj Agrawal www.bpbonline.com i ii Copyright © 2022 BPB Online All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor BPB Online or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. BPB Online has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, BPB Online cannot guarantee the accuracy of this information. Group Product Manager: Marianne Conor Publishing Product Manager: Eva Brawn Senior Editor: Connell Content Development Editor: Melissa Monroe Technical Editor: Anne Stokes Copy Editor: Joe Austin Language Support Editor: Justin Baldwin Project Coordinator: Tyler Horan Proofreader: Khloe Styles Indexer: V. Krishnamurthy Production Designer: Malcolm D'Souza Marketing Coordinator: Kristen Kramer First published: August 2022 Published by BPB Online WeWork, 119 Marylebone Road London NW1 5PU UK | UAE | INDIA | SINGAPORE ISBN 978-93-55511-379 www.bpbonline.com Dedicated to My beloved Parents: Anil Agrawal & Saroj Agrawal iii iv About the Author l Ambuj Agrawal is an industry expert in Artificial Intelligence and Enterprise Automation. He has received numerous innovation awards from Citibank, Imperial College London, Ministry of Justice UK, Bristol University, among others. He is also one of the youngest recipients of the "Exceptional Talent Visa in Digital Technology" by the UK Government for expertise in Compiler Design and Machine Learning. He has been one of the youngest Speakers at the Money2020 Europe, Fin.Techsummit Europe, Future of Work Summit London and Automation Summit Paris on the topic “Automation and Future of Work”. v About the Reviewer v Akanksha Sinha is an Architect in Intelligent Process Automation team with 8 years of Automation experience out of her 15 years of journey in various progressive roles of Software Development in Cognizant. She is extensively involved in digital transformation of client's landscapes through automations in domains of Digital Marketing and Technology. Her area of work revolves in exploring automation potential, identifying automation use cases,solutioning and developing a suite of bots along with her team in close collaboration of multiple client stakeholders and eventually providing digital experience to her clients through a suite of automation solutions. She has worked with clients like Google, Twitter and Hartford Life. Her skills are in Javascript, Google Apps Script, Google Cloud, web development, Unix, NLP, Python, Chatbots, Open Source RPA. Open Source RPAs and NLP are her areas of interest where she has worked on several prototypes. She was one of the founding members of the Tools & Automation team in Cognizant for Operation Teams working for Tech giants. She is Google Cloud certified associate and Kore.AI certified Virtual Assistant developer. She has a B.E in E&E from RTM Nagpur University, and is working with Cognizant. vi Acknowledgement m First and foremost, I would like to thank my parents who continuously encouraged me to write this book — I could have never completed this book without their support. I would also like to thank my family and friends who provided me with the continued support during my writing of this book. I am also grateful to the team at BPB Publications, who provided me the opportunity to publish this book and for providing valuable feedback throughout the process of writing this book. vii Preface This book takes the reader through different examples and code samples to automate repetitive work tasks. This book also gives solutions to common automation requirements and repetitive tasks faced during the day to day work environment. After reading this book you will be able to create automations for business processes using Python. You will also be able to identify the most common business process for automation. This book will equip you with the knowledge of creating, reading, modifying and extract data from Excel documents using Python programs. You will also be able to extract data from websites, PDF documents and send and read messages using Gmail, Outlook and WhatsApp. This book will help readers to create automations to automate their boring work and increase the efficiency of their organizations by 500%. This book is divided into 11 chapters. The details are listed below. In Chapter 1, you will be introduced to the installation steps and setting up the development environment for Python. We will also cover the installation of Python packages and libraries required for building automations. In Chapter 2, you will be introduced to the installation steps and setting up the development environment for Python. We will also cover the installation of Python packages and libraries required for building automations. In Chapter 3, we will discuss the mindset needed to be successful in implementation of automations within your organizations. We will go through the process of identifying and prioritizing automation opportunities. We will also discuss the ways to share the developed automations with the wider organization once they are created. In Chapter 4, we will discuss ways to automate Excel workflows including creating, writing, and updating the Excel documents. We viii will also discuss the data manipulation techniques with Excel and CSV documents. In Chapter 5, we will go through automation for websites and webbased tasks. We will look at how to download data from websites and automate data extraction from websites by parsing HTML documents. We will also look at the Selenium framework to automate web actions such as mouse click and keyboard actions on different websites. In Chapter 6, we will look at various file-based automations for different file types in Python. We will discuss some of the Python libraries that are used to automate different file types. We will also look at ways to extract data from PDF documents and Word documents type file structure. In Chapter 7, we would learn to automate email-based tasks using Gmail, Outlook and other SMTP clients. We will also look at Text message and WhatsApp automation using the Twilio API. In Chapter 8, we would learn to automate Graphical User Interface (GUI) by controlling the Keyboard and Mouse Actions. We will be using the Python library PyAutoGUI which works with Windows, Mac and Linux and provides automations for GUI elements within the application. In Chapter 9, we will look at computer Image fundamentals and the Pillow Python library for manipulating images. We would also look at the Tesseract library which can be used to extract text within images and scanned documents. In Chapter 10, we will look at scheduling automations using dates and timers. We would also look at external applications that can allow us to run automations based on certain events such as receiving a new email or during the start of an application. In Chapter 11, we will look at methods to extend your Python scripting knowledge and develop complex end to end process automations based on your requirements. We will learn how to work with external libraries and use external code to build these automations. We would also look at creating Python web services and using Machine Learning for automation. ix Code Bundle and Coloured Images Please follow the link to download the Code Bundle and the Coloured Images of the book: https://rebrand.ly/de9f96 The code bundle for the book is also hosted on GitHub at https://github.com/bpbpublications/Enterprise-Automation-withPython. In case there's an update to the code, it will be updated on the existing GitHub repository. We have code bundles from our rich catalogue of books and videos available at https://github.com/bpbpublications. Check them out! Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : errata@bpbonline.com Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family. Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at : business@bpbonline.com for more details. At www.bpbonline.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks. x Piracy If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at business@bpbonline.com with a link to the material. If you are interested in becoming an author If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit www.bpbonline.com. We have worked with thousands of developers and tech professionals, just like you, to help them share their insights with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea. Reviews Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions. We at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about BPB, please visit www.bpbonline. com. xi Table of Contents 1. Setting Up the Automation Environment.................................... 1 Introduction.................................................................................. 1 Structure........................................................................................ 1 Objectives...................................................................................... 1 Installing and getting started with Mu for Python 3.............. 2 Start Mu......................................................................................... 2 Installing third party packages with Mu.................................. 4 Conclusion.................................................................................... 5 Further reading............................................................................ 5 Questions...................................................................................... 6 2. Fundamentals of Python.................................................................. 7 Introduction.................................................................................. 7 Structure........................................................................................ 7 Objectives...................................................................................... 8 Introduction to Python................................................................ 8 Decision statements....................................................................11 if statement............................................................................ 11 if-else...................................................................................... 12 if-elif-else................................................................................ 12 Loops/repetition........................................................................ 13 The for loop............................................................................ 13 while loops.............................................................................. 14 The break statement............................................................... 15 The continue statement.......................................................... 16 Data structures........................................................................... 16 Lists........................................................................................ 16 Tuples..................................................................................... 18 xii Dictionaries............................................................................ 18 Sets......................................................................................... 20 Functions..................................................................................... 20 Libraries, modules, or packages.............................................. 21 Conclusion.................................................................................. 22 Further reading.......................................................................... 23 Questions.................................................................................... 23 3. Automation Mindset – Python as a Tool for Automation....... 25 Introduction................................................................................ 25 Structure...................................................................................... 25 Objectives.................................................................................... 25 Mindset for automation............................................................ 26 Common processes for automation........................................ 26 Identifying business processes................................................ 28 Conclusion.................................................................................. 30 Further reading.......................................................................... 31 Questions.................................................................................... 31 4. Automating Excel-Based Tasks..................................................... 33 Introduction................................................................................ 33 Structure...................................................................................... 33 Objectives.................................................................................... 34 Installing the library to read/write Excel.............................. 34 Creating Excel documents........................................................ 36 Reading Excel documents........................................................ 37 Updating a workbook............................................................... 40 A sample of Excel-based automation...................................... 41 CSV file automations................................................................. 43 Conclusion.................................................................................. 45 Further reading.......................................................................... 45 Questions.................................................................................... 46 xiii 5. Automating Web-Based Tasks...................................................... 47 Introduction................................................................................ 47 Structure...................................................................................... 47 Objectives.................................................................................... 48 Downloading files from the Internet...................................... 48 Introduction to HTML, CSS, and JavaScript.......................... 52 HTML.......................................................................................52 CSS............................................................................................55 JavaScript..................................................................................55 Extracting data from websites................................................. 56 Controlling the browser with Selenium................................. 61 Conclusion.................................................................................. 69 Further reading.......................................................................... 69 Questions.................................................................................... 70 6. Automating File-Based Tasks....................................................... 71 Introduction................................................................................ 71 Structure...................................................................................... 71 Objectives.................................................................................... 72 Reading and writing files......................................................... 72 PDF documents automation.................................................... 74 Word documents automation.................................................. 80 Convert a PDF to a Word document....................................... 83 Conclusion.................................................................................. 85 Further reading.......................................................................... 85 Questions.................................................................................... 85 7. Automating Email, Messenger Applications, and Messages........................................................................................... 87 Introduction................................................................................ 87 Structure...................................................................................... 87 Objectives.................................................................................... 87 xiv Simple Mail Transfer Protocol................................................. 88 Sending emails using Gmail.................................................... 89 Outlook email automation....................................................... 93 Text and WhatsApp message automation............................. 94 Conclusion.................................................................................. 99 Further reading.......................................................................... 99 Questions.................................................................................. 100 8. GUI – Keyboard and Mouse Automation............................... 101 Introduction.............................................................................. 101 Structure.................................................................................... 101 Objectives.................................................................................. 102 Introduction to the PyAutoGUI module.............................. 102 Controlling mouse actions...................................................... 104 Controlling keyboard actions................................................. 107 Automation using screenshots...............................................110 Conclusion.................................................................................112 Further reading.........................................................................113 Questions...................................................................................113 9. Image Based Automations........................................................... 115 Introduction...............................................................................115 Structure.....................................................................................115 Objectives...................................................................................115 Computer image fundamentals..............................................116 Pillow for image manipulation.............................................117 Extracting text from images using OCR............................... 120 Conclusion................................................................................ 125 Further reading........................................................................ 125 Questions.................................................................................. 125 xv 10. Creating Time and Event - Based Automations...................... 127 Introduction.............................................................................. 127 Structure.................................................................................... 127 Objectives.................................................................................. 127 Scheduling automation........................................................... 128 Writing timer programs.................................................................132 Launching programs from Python.................................................135 Using external tools for triggers.....................................................136 Conclusion................................................................................ 137 Further reading........................................................................ 137 Questions.................................................................................. 138 11. Writing Complex Automations................................................... 139 Introduction.............................................................................. 139 Structure.................................................................................... 139 Objectives.................................................................................. 140 Creating APIs with Python.................................................... 140 Combining multiple automation scripts.............................. 147 Finding solutions online......................................................... 148 Using machine learning for automation.............................. 150 Conclusion................................................................................ 153 Further reading........................................................................ 153 Questions.................................................................................. 154 Index......................................................................................... 155-159 xvi Setting Up the Automation Environment 1 Chapter 1 Setting Up the Automation Environment Introduction In this chapter, you will be introduced to the installation steps and setting up the development environment for Python. We will also cover the installation of Python packages and libraries required for building automations. Structure In this chapter, we will cover the following topics: • Installing and getting started with Mu for Python 3 • Installing third party packages with Mu Objectives After studying this chapter, you will be able to set the automation environment on your machine. You will also get an understanding of Python development environments and be able to run Python on your machine. 2 Enterprise Automation with Python Installing and getting started with Mu for Python 3 The code with Mu is a simple Python editor for beginner programmers. Download Mu installer from https://codewith.mu/ en/download. Find the installer you just downloaded (it’s probably in your Downloads folder). Double click on the installer to run it. If you get any warning while installing, accept those warnings and run the installer. Once the installation has completed successfully, click on Finish to close the installer. Start Mu You can start Mu by clicking on the icon in the Start menu or by typing Mu in the Search box. The first run will take a bit of time, and it will install and load all the required modules. Once you have started Mu, the code editor will look as shown in the following figure: Figure 1.1: Mu code editor The button bar in Mu contains buttons for creating and running the Python code along with the help instructions: Figure 1.2: Mu code editor toolbar