Chapter 1 The Database Environment and Development Process Jason C. H. Chen, Ph.D. Professor of MIS School of Business Administration Gonzaga University Spokane, WA 99258 chen@jepson.gonzaga.edu Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-1 Importance of Information • Information (and energy) are at the core of everything around us. Our entire existence (including businesses) is a process of gathering, analyzing, understanding, and acting on the information. • Modern organizations are said to be drowning in data but starving for information. Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-2 CAREER EARNING POTENTIAL • Employment for Computer Systems Analysts is expected to grow much faster than average. Jobs in this area are expected to increase by 25% between 2012 and 2020. With a Bachelor's degree -- the median salary per year for this occupation is $79,680. It is expected that 127,000 new jobs will be created in this area by 2020. • Source: U.S. Bureau of Labor Statistics. Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-3 Objectives • • • • • • • • • • Define terms Name limitations of conventional file processing Explain advantages of databases Identify costs and risks of databases List components of database environment Identify categories of database applications Describe database system development life cycle Explain prototyping and agile development approaches Explain roles of individuals Explain the three-schema architecture for databases Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-4 Definitions • Data: Meaningful facts, text, graphics, images, sound, video segments. • Database: An organized collection of logically related data. • Information: Data processed to be useful in decision making. • Metadata: Data that describes data (Data about the data). Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-5 Business and Modeling Environment Examples: Model Meta-Data Core/Essence Data Debt, Revenue Business Reality Furniture Store Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems Invoice TM 1-6 Business and Modeling Environment Examples: Model Meta-Data Core/Essence Data Debt, Revenue Business Reality Furniture Store Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems Invoice TM 1-7 Types of Data Processing • Two types of data processing – File-based data processing • e.g., applications developed by Java – Data-based data processing • e.g, applications developed by Oracle or MS/Access) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-8 Figure 1-2: Old file processing systems at Pine Valley Furniture Duplicate Data Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems What is the main problem in the company’s processing systems? TM 1-9 Disadvantages of File Processing • Program-Data Dependence – All programs maintain metadata for each file they use • Data Redundancy (Duplication of data) – Different systems/programs have separate copies of the same data • Limited Data Sharing – No centralized control of data • Lengthy Development Times – Programmers must design their own file formats • Excessive Program Maintenance – 80% of of information systems budget Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-10 Problems with Data Dependency Each application programmer must maintain their own data Each application program needs to include code for the metadata of each file Each application program must have its own processing routines for reading, inserting, updating and deleting data Lack of coordination and central control Non-standard file formats Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-11 Problems with Data Redundancy (conti.) • Waste of space to have duplicate data • Causes more maintenance headaches • The biggest Problem: – When data changes in one file, could cause inconsistencies – Compromises data integrity Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-12 SOLUTION: The DATABASE Approach • Central repository of shared data • Data is managed by a controlling agent • Stored in a standardized, convenient form Requires a Database Management System (DBMS) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-13 Database Management System A software system that is used to create, maintain, and provide controlled access to user databases Application #1 (Order Filing) Application #2 (Invoicing Sys) Application #3 (Payroll Sys.) DBMS Database containing centralized shared data DBMS manages data resources like an operating system manages hardware resources Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-14 Database Management System • A DBMS is a data storage and retrieval system which permits data to be stored nonredundantly while making it appear to the user as if the data is well-integrated. • In short, a DBMS is a software package that manages a data base. Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-15 Advantages of Database Approach • Program-Data Independence – Metadata stored in DBMS, so applications don’t need to worry about data formats (you will know this when you learn Oracle) – Data queries/updates managed by DBMS so programs don’t need to process data access routines – Results in: increased application development and maintenance productivity • Minimal Data Redundancy – Leads to increased data integrity/consistency Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-16 Advantages of Database Approach • • • • Program-data independence Planned data redundancy Improved data consistency Improved Data Sharing – Different users get different views of the data • Improved productivity of application development • Enforcement of Standards – All data access is done in the same way • Improved Data Quality – Constraints, data validation rules • Improved data accessibility and responsiveness – Use of standard data query language (SQL) • Reduced program maintenance • Improved decision support • Security, Backup/Recovery, Concurrency (not in Table 1-3) – Disaster recovery is easier Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-17 Costs and Risks of the Database Approach • New specialized personnel • Up-front costs: – Installation Management Cost and Complexity – Conversion Costs • Ongoing Costs – Requires New, Specialized Personnel – Need for Explicit Backup and Recovery • Organizational Conflict – Old habits die hard • Other hidden costs Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-18 Why Do We Still Learn File Processing Systems? • File processing systems are still widely used today, especially for backing up database systems. • Understanding the problems and limitations inherent in file processing systems can help us avoid these same problems when designing database. Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-19 Elements of the Database Approach • Data models – Graphical system capturing nature and relationship of data – Enterprise Data Model–high-level entities and relationships for the organization – Project Data Model–more detailed view, matching data structure in database or data warehouse • Entities – Noun form describing a person, place, object, event, or concept – Composed of attributes • Relationships – Between entities – Usually one-to-many (1:M) or many-to-many (M:N) • Relational Databases – Database technology involving tables (relations) representing entities and primary/foreign keys representing relationships Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-20 A Data Model on Customer and Order Segment of an Enterprise Data Model CUSTOMER ORDER M ORDER? Q1. One CUSTOMER normally places ___ 1 CUSTOMER? Q2. One ORDER normally is placed by __ How about the relationship between ORDER and PRODUCT? (see next slide) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-21 A Data Model on Customer and Order Segment of an Enterprise Data Model CUSTOMER ORDER M ORDER? Q1. One CUSTOMER normally places ___ 1 CUSTOMER? Q2. One ORDER normally is placed by __ How about the relationship between ORDER and PRODUCT? (see next slide) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-22 Figure 1-3(a): Comparison of enterprise and project level data models ORDER ORDER_NUMBER Customer_ID Order_Date Q3. One PRODUCT normally is contained in M __ ORDER? Q4. One ORDER normally contains __ PRODUCT? M Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-23 Figure 1-3(a): Comparison of enterprise and project level data models ORDER ORDER_NUMBER Customer_ID Order_Date Q3. One PRODUCT normally is contained in M __ ORDER? Q4. One ORDER normally contains __ PRODUCT? M Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-24 Figure 1-3: Comparison of enterprise and project level data models (a) Segment of an Enterprise Data Model Q: what are two major differences between (a) & (b)? (b) Segment of a Project-Level Data Model Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-25 FIGURE 1-15: Project data model for Home Office product line marketing support system Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-26 Figure 1-3 Segment from Enterprise Data Model Figure 3 Enterprise data model is a graphical model that shows the high-level entities for the organization and the relationship among these entities. (E/R Diagram) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-27 Figure 1-3 Segment from enterprise data model Figure 3 One customer may place many orders, but each order is placed by a single customer One-to-many relationship Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-28 Figure 1-3 Segment from enterprise data model One order has many order lines; each order line is associated with a single order One-to-many relationship Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-29 Figure 1-3 Segment from enterprise data model Figure 3 One product can be in many order lines, each order line refers to a single product One-to-many relationship Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-30 Figure 1-3 Segment from enterprise data model Figure 3 Therefore, one order involves many products and one product is involved in many orders Many-to-many relationship Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-31 Figure 1-4 Enterprise data model for Figure 1-3 segments Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-32 Figure 1-5 Components of the Database Environment Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-33 Components of the Database Environment • CASE Tools–computer-aided software engineering • Repository–centralized storehouse of metadata • Database Management System (DBMS) –software for managing the database • Database–storehouse of the data • Application Programs–software using the data • User Interface–text and graphical displays to users • Data/Database Administrators–personnel responsible for maintaining the database • System Developers–personnel responsible for designing databases and software • End Users–people who use the applications and databases Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-34 34 Evolution of Database Technologies • • • • • • • Flat files - 1960s - 1980s Hierarchical – 1970s - 1990s Network – 1970s - 1990s Relational – 1980s - present Object-oriented – 1990s - present Object-relational – 1990s - present Data warehousing – 1980s - present Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-35 Figure 1-10a Evolution of Database Technologies Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-36 Figure 1-10b Database architecture Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-37 Figure 1-10b Database architecture Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-38 The Range of Database Applications • Personal databases • Two-tier Client/Server databases • Multitier/N Client/Server (or called web-enabled) databases • Enterprise applications – Enterprise resource planning (ERP) systems – Data warehousing implementations (Local Area Network) (Web-enabled Database ) (WAN) (Wide Area Network) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-39 Figure 1-11 Two-tier database with local area network Chapter 1 Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-40 Figure 1-12 Three-tiered client/server database architecture Chapter 1 Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-41 Muti/N-tier: Web-Enabled Databases • Web applications requiring databases – – – – – Customer relationship management (CRM) Business-to-consumer (B2C) Electronic data interchange (EDI) Private intranets XML-defined Web services Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems 42 TM 1-42 Enterprise Applications • Enterprise Resource Planning (ERP) – Integrate all enterprise functions (manufacturing, finance, sales, marketing, inventory, accounting, human resources) • Data Warehousing implementation – Integrated decision support system derived from various operational databases Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-43 Break ! (Ch. 1) Exercise #1 (p.44) Homework: 1. Complete chapter1 quiz by Sunday evening. 2. HW#12 (a) only (homework assignment - high-level (no attributes) - draw by Visio clearly - turn in a hardcopy next class Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-44 Be Prepared for ... Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-45 Discuss HW • HW#12 (a) • Volunteer? Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-46 Enterprise Data Model Enterprise data model is a graphical model that shows the high-level entities for the organization and the relationship among these entities. (E/R Diagram) • Enterprise data modeling is the first step in database development, in which the scope and general contents of organizational databases are specified. • Descriptions of entity types Q: Is ER/M a top• Relationships between entities down or bottom-up approach? • Business rules Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-47 Database Design • Systems Development Life Cycle (SDLC) • Entity-relationship model (E-R model) • Normalization Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-48 Systems Development Life Cycle Understand the Business Problem or Opportunity Develop an Information System Solution Systems Investigation (Definition) Product: Feasibility Study Systems Analysis Product: Functional Requirements Systems Design Product: System Specifications Implement the Information System Solution Systems Implementation Product: Operational System Systems Maintenance Product: Improved System Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-49 Steps in the Database Development Process • Enterprise Modeling • Conceptual Data Modeling – Cuts across Project Initiation and Planning & Analysis phases of SDLC • • • • Logical Database Design (E/R) Physical Database Design and Creation Database Implementation Database Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-50 Life Cycle Phases of DA and DBA (Ch.11) Database Planning Database Analysis Database Design Database Implementation Operations and Maintenance Growth and Change Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-51 Two Approaches to Database and IS Development • SDLC – System Development Life Cycle – Detailed, well-planned (and structured) development process – Time-consuming, but comprehensive – Long development cycle • Prototyping – – – – Rapid application development (RAD) Cursory attempt at conceptual data modeling Define database during development of initial prototype Repeat implementation and maintenance activities with new prototype versions – Tool: Oracle Designer Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-52 Database Design • Systems Development Life Cycle (SDLC) • Entity-relationship model (E-R model) • Normalization Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-53 Systems Development Life Cycle Understand the Business Problem or Opportunity Develop an Information System Solution Systems Investigation (Definition) Product: Feasibility Study Systems Analysis Product: Functional Requirements Systems Design Product: System Specifications Implement the Information System Solution Systems Implementation Product: Operational System Systems Maintenance Product: Improved System Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-54 Prototyping • Prototpying is one of the most popular rapid application development (RAD) methods. • It is an iterative process of system development in which requirements are converted to a working system that is A prototype is a small, continually revised through but working system that close work between analysts contains only those and users. important (not complete) features. Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-55 Systems Development Life Cycle (see also Figure 1-7) Project Identification and Selection Project Initiation and Planning Analysis Logical Design Physical Design Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-56 Systems Development Life Cycle (see also Figure 1.7) Project Identification and Selection Purpose --preliminary understanding Deliverable –request for project Project Initiation and Planning Analysis Logical Design Physical Design Database activity – enterprise modeling Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-57 Systems Development Life Cycle (see also Figure 1.7) Project Identification and Selection Purpose – state business situation and solution Deliverable – request for analysis Project Initiation and Planning Analysis Logical Design Physical Design Database activity – conceptual data modeling Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-58 Systems Development Life Cycle (see also Figure 1.7) Project Identification and Selection Purpose –thorough analysis Deliverable – functional system specifications Project Initiation and Planning Analysis Logical Design Physical Design Database activity – conceptual data modeling Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-59 Systems Development Life Cycle (see also Figure 1.7) Project Identification and Selection Project Initiation and Planning Purpose –information requirements structure Deliverable – detailed design specifications Analysis Logical Design Physical Design Database activity – logical database design Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-60 Systems Development Life Cycle (see also Figure 1.7) Purpose –develop technology specs Deliverable – program/data structures, technology purchases, organization redesigns Project Identification and Selection Project Initiation and Planning Analysis Logical Design Physical Design Database activity – physical database design Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-61 Systems Development Life Cycle (see also Figure 1.7) Purpose –programming, testing, training, installation, documenting Deliverable – operational programs, documentation, training materials Project Identification and Selection Project Initiation and Planning Analysis Logical Design Physical Design Database activity – database implementation Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-62 Systems Development Life Cycle (see also Figure 1.7) Project Identification and Selection Purpose –monitor, repair, enhance Deliverable – periodic audits Project Initiation and Planning Analysis Logical Design Physical Design Database activity – database maintenance Implementation Maintenance Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-63 Figure 1-7: Database development activities during the systems development life cycle (SDLC) Planning (Enterprise modeling) Project Identification and Selection Project Initiation and Planning Conceptual data modeling Analysis Upper CASE tool (Front-end) Logical Design Integrate database views and perform normalization Physical Design Implementation Lower CASE tool (Back-end) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems Growth and Change Maintenance TM 1-64 Figure 1-8The prototyping methodology and database development process Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-65 Figure 1-8 The prototyping methodology and database development process (cont.) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-66 Figure 1-8 The prototyping methodology and database development process (cont.) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-67 Figure 1-8 The prototyping methodology and database development process (cont.) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-68 68 Figure 1-8 The prototyping methodology and database development process (cont.) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-69 Managing Projects • Project–a planned undertaking of related activities to reach an objective that has a beginning and an end • Initiated and planned in planning stage of SDLC • Executed during analysis, design, and implementation • Closed at the end of implementation Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-70 70 Managing Projects: People Involved • Project is a planned undertaking of related activities to reach an objective that has a beginning and an end • People involved: – Business analysts – work with management and users to analyze business – Systems analysts - business situation and IS needs – Database analysts and modelers - requirements and design for the database component of the IS – Users - assessment of their information needs and monitor the developed system meet their needs – Programmers – design and write computer programs – Database architects – establish standards – Database and data administrators - ensure database consistency, integrity and provide consulting, training etc.. – Project managers – oversees assigned projects – Other technical experts - network administrators, testers, technical writers Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems 71 TM 1-71 The Technology Level of Models • Conceptual models focus on the underlying content of an information system with no assumptions about technology • Logical models assume a general class of technology (H/S W independent) – a relational database • Internal models assume specific technologies – for example, an Oracle database engine Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-72 Database Schema • External Schema (during the analysis and logical design phases) – User Views – Subsets of Conceptual Schema – Can be determined from business-function/data entity matrices – DBA determines schema for different users – This is part of people-management in databases • Conceptual Schema – ER models (during the analysis phase)– covered in chapters 2 and 3 • Internal Schema – Logical structures–covered in Chapter 4 – Physical structures–covered in Chapter 5 Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-73 Figure 1-9 Three-schema architecture Different people have different views of the database…these are the external schema The internal schema is the underlying design and implementation` Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems 74 TM 1-74 Figure 1-9: Three-schema database architecture External Schema Ch. 4 Ch. 2,3,4 Meta-data/ Repository/ D.D. E/R, OO … Relations Ch. 5 Database Internal Schema Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems N TM 1-75 Levels of database schemas • Different schemas are presented to different users External View 1 External View 2 External View 3 External level logical to external mappings Logical Schema Logical level internal to logical mapping disk Internal Schema Internal level N Exercise/Homework Homework (1) #17 ; p.45; three views with E/R - Statement View - Deposit View - Conceptual View (an integrated view the above two views) Hint: account/customer is one of common entity (Draw by Visio/Word, turn in hardcopy, due next class) (2) Online Quiz#1 (due date, midnight Sunday) Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-77 FIGURE 1-15 (a): Preliminary data model for Home Office product line marketing support system Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-78 Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-79 Copyright © Addison Wesley Longman, Inc. & Dr. Chen, Business Database Systems TM 1-80