ODU Computer Science CS411 Lab 1 EmVi Project Outline and Description Keith Walsh <kwalsh@cs.odu.edu> 5-9-2013 Keith Walsh CS410 Fall 2013 Final Paper Contents 1) Introduction .................................................................................................................................................. 2 a) Societal Problem/Problem Statement ..................................................................................................... 2 b) Needed Solution Characteristics/Solution Statement ............................................................................. 3 c) EmVi – Email Viewer (Email Content Management System) ................................................................... 3 2) Product Description ...................................................................................................................................... 3 a) EmVi Product Description ........................................................................................................................ 3 b) Solution Summary .................................................................................................................................... 4 c) Goals/Objectives ...................................................................................................................................... 4 3) Key Product Features/Capabilities ............................................................................................................... 4 a) Content Management .............................................................................................................................. 4 b) Upload to CDN .......................................................................................................................................... 4 c) Approval System....................................................................................................................................... 4 d) Email Preview ........................................................................................................................................... 5 e) Streamlining ............................................................................................................................................. 5 4) Major Components (Hardware/Software) ................................................................................................... 5 a) Real World Product MFCD........................................................................................................................ 5 b) Hardware .................................................................................................................................................. 6 c) Third Party Software ................................................................................................................................ 6 d) Web Server Software ............................................................................................................................... 7 5) Identification of Case Study.......................................................................................................................... 7 a) Keith Walsh, Development Lead, Email Marketing Systems .................................................................... 7 b) EmVi is being developed for users of Email Content Management Systems........................................... 7 c) Faculty Mentor & Domain Experts ........................................................................................................... 7 6) Product Prototype Description ..................................................................................................................... 7 a) The EmVi tool prototype is basically a Email Content Management System that will ............................ 7 b) Prototype Functional Goals/Objectives ................................................................................................... 8 7) Prototype Architecture (Hardware/Software) ............................................................................................. 9 a) Prototype MFCD ....................................................................................................................................... 9 b) Hardware .................................................................................................................................................. 9 1 Keith Walsh CS410 Fall 2013 Final Paper c) Third Party Software ................................................................................................................................ 9 d) Web Server Software ................................................................................ Error! Bookmark not defined. 8) Prototype Features/Capabilities ................................................................................................................. 10 a) Major Features ....................................................................................................................................... 10 b) User Role Interfaces ............................................................................................................................... 10 c) Registration ............................................................................................................................................ 10 d) User Login ............................................................................................................................................... 10 e) Email Previewing .................................................................................................................................... 11 f) Testing .................................................................................................................................................... 11 g) Risk Mitigation ........................................................................................................................................ 11 9) Prototype Developmental Challenges ........................................................................................................ 11 a) Risks ........................................................................................................................................................ 11 b) Completing development of integral and creative aspects of the tool..... Error! Bookmark not defined. 10) Glossary .................................................................................................................................................. 12 11) References .............................................................................................................................................. 12 1) Introduction a) Societal Problem/Problem Statement Our team has defined the problem statement for our project as follows: The process of creating marketing email content and testing across all emails clients is time consuming and complicated. The underlying premise is that marketing email has a significant business role to play and consumes a great deal of time in people’s lives, whether professionally or personally. Email accounts are abundant worldwide. Free email services such as Gmail, Hotmail and Yahoo mail have greatly simplified the process of creating email accounts, and marketplace projections show that the number of active email accounts worldwide will continue to increase In terms of marketing email, research shows that consumers are most comfortable receiving offers in their email boxes as opposed to text messaging, social Web sites such as Facebook, direct mail or telephone. Consumers tend to check their email often—at least once a week is the norm. 2 Keith Walsh CS410 Fall 2013 Final Paper Companies that use email for marketing purposes have reported a return on investment (ROI) as high as 4,000% ($40 returned for every $1 invested). This return on investment has driven a great deal of spending on email marketing. In 2012, more than one billion dollars were spent by US companies on email marketing. Most expect to continue to grow their email marketing budgets in the coming years. However, despite the efforts of many companies in the email marketplace, the process of creating, storing and testing email content is inefficient. b) Solution Characteristics/Solution Statement To address the problems faced by email marketers, our team will create an online solution which we have preliminarily dubbed “EmVi,” short for Email Viewer. Our solution proposal is as follows: Provide a stand-alone email marketing content management system that streamlines the development and testing process so designers can perform end-to-end testing independently. c) EmVi – Email Viewer (Email Content Management System) This solution will allow the people who are creating email content to store, retrieve, review and test email content all in one place. The goal is to provide a simple and automated way to release email that functions well and looks consistent across all email clients running in all different browsers on all different operating systems. 2) Product Description a) EmVi Product Description EmVi will be a stand-alone, web-based system. The primary feature will be email content storage and a workflow/approval system. It will also contain two key integration points with third-party products: a means of easily adding images to a content distribution network (CDN) and a way to view content across the major email clients. CDN integration will include two of the major cloud services—Amazon Web Services (AWS) and Microsoft Azure; content viewing will be made possible via an application programming interface (API) Proxy which will allow integration with products like Litmus and Email on Acid. 3 Keith Walsh CS410 Fall 2013 Final Paper b) Solution Summary The intended users of EmVi will be Web designers and developers who are involved in creating HTML content for email campaigns. It will allow them to manage content up to the point where they can say it is ready for release to customers. The largest benefit the product will provide to organizations is reducing the time it takes to view and approve email content previews. Most email marketing systems include a preview feature, but it is intended to be used only a final check at the very end of the process. When last-minute changes are made to content, they often result in delays or mistakes. c) Goals/Objectives The goals of the product are to automate the following: Version control Approval process Image uploading to CDNs Test email generation Preview of test emails on all major email clients 3) Key Product Features/Capabilities a) Content Management The primary feature of EmVi is content management. HTML and text content are produced outside of the tool and uploaded into it. Once in the tool, all associated files can be tagged with metadata such as launch date, campaign name and keywords; all of those files are then searchable by those attributes. b) Upload to CDN The nature of mass email is that it causes spikes in http requests to the Web servers hosting images. As such, it is an industry best practice to ship images to a distributed CDN. CDNs have two qualities that make them ideal for this job: 1) high availability, meaning the Web servers are reliable to very close to 100% of the time, and 2) sensors that allow them to serve images from data centers close to the user who makes the request. The two CDNs we will build support for are Amazon CloudFront CDN and Windows Azure CDN. c) Approval System 4 Keith Walsh CS410 Fall 2013 Final Paper As part of the content management system, EmVi will include a workflow component, where reviewers are assigned to each campaign, and that campaign cannot progress until that reviewer signs off. d) Email Preview One of the most significant features of EmVi is its ability to integrate with email preview tools. The two industry leaders in this area are Litmus and Email On Acid. These products require subscription fees and provide their own Web interfaces. Our system will allow users to access them in this very straightforward way, but it will also provide easy integration through an API proxy. e) Streamlined Process Together, these features add up to a streamlined content review process. Email designers can perform endto-end testing on their content and ensure that it looks good in all email clients before handing it off for deployment. 4) Major Components (Hardware/Software) a) Real World Product MFCD The functional diagram for EmVi shows how it allows Web designers to complete the email content task. First, it allows designers to upload content into the tool. The design will accept content with relative URLs in image tags to and update them to the URLs where the images are stored on the CDN. During the campaign process, the designer will be able to upload images, which will be stored within the tool. Part of the process will be to ship the images to the CDN with date/time parameters for go live and retirement. When the end date passes, the images are removed by the tool. EmVi also contains two types of test engines: the ability to send content to live inboxes and an integration point with automated testing products via an API proxy. 5 Keith Walsh CS410 Fall 2013 Final Paper Figure 4.1: Major Functional Component Diagram b) Hardware EmVi will be a Web-based application. The hardware necessary is the following: Web server File system Database We will deploy EmVi on a virtual machine on a Computer Science server running Linux. c) Third Party Applications/Software CDN support (subscription-based): Windows Azure and Amazon CloudFront Email rendering products (subscription-based): Litmus and Email On Acid Database Engine: MySQL Web Application Framework: Play (A Java-based framework for developing Web pages and applications) 6 Keith Walsh CS410 Fall 2013 Final Paper d) Web Server Software Apache Simple Mail Transfer Protocol (SMTP) Public API for all EmVi core functionality 5) Identification of Case Study a) Keith Walsh, Development Lead, Email Marketing Systems The proposal for this product was made by Keith Walsh, who leads a software development team focusing on email marketing at Microsoft. The team uses an enterprise level CRM system call Epiphany Marketing to handle outbound email. His team has identified a gap in the marketplace between creative agencies, who create email content, and list management/email deployment tools. His team has developed an in-house product to manage a larger approval process, but the steps that include uploading images and testing content remain largely manual and require a good deal of coordination between team. b) EmVi is being developed for users of Email Content Management Systems This product is not intended to replace a functioning content manage system (CMS) if that system is used to create HTML and text content. Rather, it takes completed files and readies them for email, which has different requirements from Web pages. c) Faculty Mentor & Domain Experts Faculty Mentor: Steven Zeil, Associate Professor, Old Dominion University (ODU) Systems and Process Integration: Dana Rambo, Program Manager, Microsoft Email Marketing Designer: Sara Johnson, President, A Touch of Tech Programming Expert: Ryan Ward, Network Administrator, SimIS 6) Product Prototype Description a) Email Content Repository and Approval System at Heart 7 Keith Walsh CS410 Fall 2013 Final Paper The major purpose of EmVi is to store email content and make it accessible for marketers. We expect most of our development effort to focus on the following four primary things: 1. 2. 3. 4. A clean, intuitive user interface (UI). Security and user roles. Document and image storage and retrieval. Workflow process that permits multiple users to interact with content. The goal is produce a tool that streamlines and stores the content approval process, reduces errors in the testing and development process, and decreases the time it takes to go from creative development to fully finished product. b) Prototype Functional Goals/Objectives Create a Web interface that allows user to register and administer their own accounts. Create various user roles for users and customized interfaces for each. Trigger emails when a person registers, when their registration is approved by an administrator and when action in the tool is expected of them. Allow users to upload HTML, text and image files. Display email layouts inside the browser. Allow users to send multipart emails to their own specified addresses using the uploaded HTML and text files. Push image files to a CDN and modify image URLs accordingly. Support API integration with email testing products. 8 Keith Walsh CS410 Fall 2013 Final Paper 7) Prototype Architecture (Hardware/Software) a) Prototype MFCD Figure 7.1: MFCD for Prototype b) Hardware The hardware running the prototype is undetermined at this point. It will be a virtual machine running in the ODU Computer Science lab. The operating system will be a flavor of Linux. We will run a development instance, and pre-production instance and a production instance of the software at all times. For the purpose of the class, all three will run on the same virtual server. c) Third Party Software All software for this product will be open source. We intend to use the following: Eclipse for Java code development Git for code storage and version control Play Framework 2.1.1 as our starting point for code development Apache for our Web server MySQL for our database 9 Keith Walsh CS410 Fall 2013 Final Paper Additionally, we with integrate via public APIs with the following globally distributed CDNs: Amazon CloudFront Windows Azure For content rendering, we will provide an API proxy which we’ll test against the two leading products on the market today—Litmus and Email On Acid—but due to subscription costs will not support once the product goes live. 8) Prototype Features/Capabilities a) Major Features The major features included in EmVi are the following: User authentication Content storage and security Workflow and audit trail Multipart email sending b) User Role Interfaces We will set up user roles according to two conditions: System and Campaign. The system will have two roles: Administrator: Can approve/modify/delete users; can perform any task on any campaign in the system; can re-assign campaign owners. User: Can create/view/modify/delete campaigns according to specific campaign role. Campaigns will each have the following two roles: Owner: Can create campaigns; can view/modify/delete campaigns they have created; can assign other owners and approvers to the campaign. Approver: Can view campaigns; can approve/reject campaigns to which they’ve been assigned. c) Registration A confirmation email will be sent to a user after they register. A second email will be sent after the system administrator approves them as either an Admin or User. d) User Login 10 Keith Walsh CS410 Fall 2013 Final Paper The login page will be open to the public. It will prompt for a user name and password. e) Email Previews The email preview interface will allow a user to enter an email address or choose from a list of stored email addresses. It will send a copy of the content in multipart MIME format to each of the addresses. f) Testing Process The campaign owner and approver will be responsible for viewing the email across multiple platforms to ensure consistency. There will be no automated process for viewing layout. Litmus and Email On Acid provide a single page view of an email across many email clients, making the testing process quicker. EmVi, however, integrates with these products only to the extent that their APIs allow. 9) Prototype Developmental Challenges a) Risks Program Latency/Scaling: The product will be expected to contain all image and content files for email campaigns. As such, it must store and retrieve a lot of data. Version Control: Users must be able to revise content. Since multiple users can work on a given campaign, version control must be considered. Security: Since corporate strategy may be embedded in marketing campaigns, unauthorized users must be prevented from accessing the system. System Maintenance: As an open source project, EmVi code will ultimately be released to the community. Developers will be expected to maintain the system without support from the project builders. Associated Costs: Hardware and support may be a barrier to entry for some users who would like to use EmVi. b) Risk Mitigation Program Latency/Scaling: Create algorithms (multiple levels of testing) that will improve processing time of data. Version Control: Incorporate a check-in/check-out process for locking campaigns as users modify them. Retain change history for all content. Security: Ensure access control is incorporated on all externally facing elements. System Maintenance: Documentation. Tons of it. Associated Costs: Keep the system requirements slim. The system will not require that external APIs by employed so that those associated costs can be avoided. 11 Keith Walsh CS410 Fall 2013 Final Paper 10) Glossary Glossary of Terms Access Control- security features that control who can access resources in the operating system, selective restriction. Administrator (Admin) - has the ability to create and remove new contributors as well as restrict their access. Analytics- is the process of transforming data into meaningful patterns to help in the decision making process. API-Application Programming Interface Approval Chain- the series of steps necessary for an email to be authorized and distributed. Approval Tracking- is specific to a workflow, the workflow action to perform when a user sets an approval type. Authentication- the process of identifying an individual, usually based on a username and password. Azure- a Content Distribution Network (CDN), a Microsoft product that allows you to build, deploy, and manage applications globally. Campaign Filter- allows campaigns to display based on criteria that are chosen by the user. Campaign Search- searching for the content of a campaign (content browsing). Content Distribution Network (CDN) - or delivery network is a large system of servers that allows for faster and more efficient delivery of content to end-users. Contributor- has the ability to read, write, and edit content. Customize-changing or altering to fit current needs. Database-a collection of information organized in a manner which allows for efficient retrieval. 12 Keith Walsh CS410 Fall 2013 Final Paper Dynamic messaging- the use of variable content to fill particular sections of an email message. Some examples are using first name personalization or product name insertions within the body of a message. Drupal- a Content Management System (CMS) that allows for easy organization, management, and publishing of content, with an endless variety of customization. ECMS- Email Content Management System Email Campaign- a single instance of an email sent to a list of email addresses. Campaigns may include multiple sends and multiple messages. Email Client-a computer program used to access and manage a user’s email. Email header- the data that appears in the header of an email message, usually consisting of to and from email addresses, email subject and IP-level tracking information. Email Marketing- usually done by a company to directly market a commercial message for promotional or notification purposes to a group of consumers through the use of email. Email Message- a single email received to an email address within a campaign. A message contains a multipart, alternative message which includes an HTML and text file. ESP- Email Service Provider Exact Target- a provider of data driven marketing solutions for email content creation, list management, etc. This company is capable of integration with CDNs. HTML Email- a subset of HTML that is not well defined and can sometimes have differing results depending on the email client in which it is viewed in. Some email clients do not support HTML Email at all. Integrative-combining or unifying. Litmus- a company that allows for the rendering and testing of email across various email clients. Also, email analytics, spam filter tests, and page tests can be performed. Metadata- is descriptive data about campaigns and images that can be used to search for content. Multipart, Alternative- an email that includes both an HTML and text version. The email client determines which version to display. Open Source-is computer software made available publicly and free of charge. Outsourcing-the contracting of internal business processes to a third party organization. 13 Keith Walsh CS410 Fall 2013 Final Paper Permissions-or rights, are characteristics given by users or network administrators that prevent or allow access to files on a computer network. RACI Chart-Responsible, Accountable, Consulted, Informed. This chart displays the various roles and responsibilities required in completing tasks for a project or business process. Simple Mail Transfer Protocol (SMTP) - an internet protocol for sending and receiving email messages. Version Control- a system to record changes that are made to a campaign. This grants the ability to restore the campaign to a previous version if necessary. Web Application- is an application that is accessed over the internet usually through a web browser. This allows the application to be used on multiple platforms. Web Server-the hardware or software that helps deliver web content and can be accessed through the internet. Workflow- a series of connected steps to complete a process. 11) References 14