Neos - Overview
NEOS
NEOS information system design and development is the specialized company for development of complex
information systems with medium-sized and large customers in Croatia, broader region, and Europe.
Neos was founded in 2002 and since the beginning we were members of Oracle Partner Network as Oracle Certified
Partner. After OPN reorganization in 2009, we were acknowledged as Oracle Gold Partner. At the beginning of 2011,
we have achieved our first specialized status – Specialized Oracle Business Intelligence Foundation – as one of only
30 companies in the world at the time.
In the second half of 2011 we have achieved our second specialization – Specialized Oracle Application
Development Framework – as the second company in the world.
Those achievements are proof of our orientation towards business excellence, technological excellence, and
compliance to the proven best practices in IT industry. Besides Oracle, we have partnerships with other industry
leaders such as IBM, Microsoft, and Novell.
In recent years Neos has been experiencing constant growth in aspects of our business – number of employees,
income, and profit, as well as in number of our clients and users of our services and products. They have recognized
us as a competent and reliable partner in improvement of their own businesses.
Our employees are highly educated professionals with extensive experience in project management, development
and deployment of highly available IT systems in various areas of business with special emphasis on
telecommunications and financial industries.
Line of business
Neos excels in fields in our line of business – DW/BI & EPM Solutions and Custom development solutions based on
ADF, Java and related technologies. On top of both line of business we are also providing consulting services
regarding business and technology optimization.
NEOS has long-term experience in development of BI projects and establishing of systems and solutions in the field
of Data Warehouse, Business Intelligence and Enterprise Performance Management. Since the founding of the
company we are oriented towards superb results in our field of work, which in large part consists of BI & DW
solutions. This is confirmed by our Oracle BI Foundation Specialization, which was achieved as addition to Oracle
Gold Partner status. Our product – NEOS BI Framework – is additional guarantee for successful BI projects.
Custom application development is the second area of business we are dedicated to, which is also confirmed by
Specialized Oracle Application Development Framework we have achieved as the second company worldwide! NEOS
ADF KickStart is the product that helps our clients in rapid application development or conversion of legacy systems.
Besides excellence in Oracle ADF, we are experts in Java/JEE, SOA, and related technologies.
1
Neos - Overview
Clients
The profile of our clients includes large companies from Financial, Telco and other industries who have recognized
the need for specialized solutions in fields related to our line of business. Here are just few of our main references
in different industries:
Financial Industry





Privredna Banka Zagreb, Croatia; member of Intesa SanPaolo Group, Second Largest Croatian Bank
Zagrebačka Banka, Zagreb, Croatia; member of Unicredit Group, Largest Croatian Bank
Erste & Steiermärkische Bank, Zagreb, Croatia; member of Erste Group), Third Largest Croatian Bank
Isbank Turkey, Istanbul, Turkey; Largest Turkish Bank
AZFond Pension Funds, Zagreb, Croatia; Largest Croatian Pension fund
Telco Industry



THT - Hrvatski Telekom, Zagreb, Croatia; member of Deutche Telekom Group
Makedonski Telekom, Skopje, FYR Macedonia; member of Deutche Telekom Group
Vipnet, Zagreb, Croatia; member of Telekom Austria Group
Other Industries


Hrvatska elektroprivreda, Zagreb, Croatia; Croatian National Electricity Company
Croatia Osiguranje, Zagreb, Croatia; Croatian Largest Insurance Company
Besides working on our projects and products, we are successful as business and technology consultants. Our experts
are competent not only in development technologies, but also in diverse business fields. They will grasp your
business demands and will be able to transfer them into solutions.
We are competent in the fields of highly specialized solutions in the financial and telecommunications industry that
are completely IT dependent and require high-quality solutions due to sensibility and quantity of data they deal with
daily. Also we are having large hands-on experience in fields of material accounting, manufacturing, retail, wholesale,
distribution, and logistics, and other fields we encounter in commerce and production companies.
Our solutions in the above-mentioned business areas are produced on the basis of world-proven technologies and
methodologies.
The management and operations of our company applies all applicable legal regulations of Republic of Croatia. We
have established a quality management system, and are using the latest IT technologies and best practices in the
areas of our business.
All our professional staff is certified in areas in which they work on (Oracle Certified Professional/Associate, Oracle
Specialist etc.).
Competencies and experience
Neos is having over 10 years of experience based on number of domestic and international DW/BI/EPM projects
providing consulting services related to different areas ranging from Business consulting, High-level System
Architecture design to Technical Implementation of individual modules.
2
Neos - Overview
Competency areas

Business Intelligence and Data Warehouse Systems

Profitability Analysis, Risk Management & Performance Management Solutions

Financial and Telco Industries specific business knowledge

Specific business solutions based on Oracle Middleware (WebLogic, Java development stack)

Consulting services and education related to our competency areas
Technology experience

Oracle RDBMS, Oracle Warehouse Builder (OWB), Oracle Data Integrator (ODI), Oracle Business Intelligence
Enterprise Edition; Hyperion Essbase, Informatica

Oracle Financial Services Analytic Applications Suite

Oracle Middleware, Oracle WebLogic application server

Java/JEE (EJB 3, JPA, JSF, JAX-WS, JMS), jBoss RichFaces, Java Sprig/Hibernate, Flex, Oracle jDeveloper and
other technologies related to Java/JEE, SOA, ESB

Experience with diverse specific platforms (EXADATA, EXALOGIC)
3
Business Intelligence Solutions
NEOS Business Intelligence solutions
Business Intelligence constitutes a separate group of application and technology solutions for collection, storage,
cleansing, unifying and analysis enabling client’s independent access to information with a better decision-making
goal. Data Warehouse is architectural and technological prerequisite serving as foundation for BI applications.
Business intelligence systems as a rule provide: decision-making support, ad-hoc analyses and reporting, online
analytical processing (OLAP), statistical analyses, forecasts etc.
Services
The main services that we provide in this business segment can be divided into the following groups:
Consulting and Analysis – this field refers to services that can be relevant in different project stages, including:







Assessment of readiness for BI & DW System Implementation;
Business Requirements Analysis and Documenting;
BI & DW System Planning and Implementation;
Selection of Tools/Technologies;
Existing DW & BI environment assessment (Health Check);
Data Governance;
Data Quality Analysis;
Design and implementation – represents the most extensive field in which following activities can be briefly defined:





Organisation of the entire BI & DW system lifecycle;
Project initiation – Definition of Phases and Deliverables;
Logical and Physical System Design;
Implementation of all modules (Data collection – ETL/ELT, Data model, Configuration of SW&HW
environment);
Integration, Testing and Documentation.
Training – is an important part of a BI & DW project and special attention should be devoted to it. The following
combination is recommended:


Predefined standard courses (e.g. Oracle University – often featuring Neos consultants as lecturers),
Tailored user workshops focusing on the specific areas of interest.
Project methodology
Although we believe that each DW & BI project is a case for itself, we certainly recommend that the project is
organized pursuant to the methodology based on years of experience on similar projects. Thus we can define the
basic elements of any DW & BI project:
Analysis – its aim is to set the project bases regarding demands, deliverables, stages, duration etc. The main goals
of this stage are:





definition of user needs,
definition of system architecture,
identification of the main subject areas of interest,
setting project stages according to priorities,
definition of clear and measurable deliverables.
Logical architecture and standards – this stage refers to setting the project framework in terms of logical system
architecture and the development stage. This stage is crucial as it reflects on all further stages of system
development, and includes:


data flow definition according to logical system segments (DSA, ODS, EM, TGT etc.),
documentation and development standards (names, reusable algorithms, documentation bases etc.).
4
Business Intelligence Solutions
Physical architecture and technology – this stage refers to definition of HW & SW platform in accordance with
current requirements and expected system development.
Iterative development of specific subject areas – this stage (or stages) is focused on deliverables according to the
predefined priorities. General recommendation is that these stages last for up to 3-6 months and that they should
lead to results that are evident to the end users.
Training – is held in form of interactive workshops with the end users, where future users get familiar with the
system through specific practical problems.
Neos DW framework – refers to a set of development and documentation standards and reusable modules, with
upgrade for standard tools such as e.g. Oracle Data Integrated. The DW framework is used during the entire project
and its aim is to increase productivity and documentation and upgradeability of the system. The main system
components are:





reusable modules and algorithms (increased productivity and decreased implementation risk),
mechanisms for recording of process performance status (enable process restarting/traceability, loading
consistency in case of error),
monitoring and system governance modules (process performance statistics, dependencies, failure
removal),
standardized notification and alarming mechanisms,
module development and documentation standards (document templates).
NEOS BI & DW Expertise
NEOS is providing consultancy & implementation services related to DW, BI, EPM and related systems in different
industries – including financial industry and telecom industry – which are our largest customers.
We are providing this kind of services for more than 10 years having number of successfully finished projects behind
us. In this chapter we will explain few key concepts based on our experience and common best practices related to
DW/BI system implementation.
DW Architecture
From logical aspect, DW systems consists of several segments, which must not be always present. Segments
containing data are:



DSA – Data Staging Area
EM or ODS – Enterprise Model or Operational data Store
TGT – Target Area
Each of those segments are important and have their distinctive role in data preparation and analysis. Other
potentially present segments:


WH_DQ – Data Quality module repository
WH_EXT – External Modules – external reporting, data mining source, etc.
The whole system is based on the module that is implemented as integral part of the Neos DW/BI Framework:

WH_SYS – Metadata Module for Neos DW/BI Framework
Whole system can be surrounded by external modules, which are not strictly part of the DW system, but are parts
of the system as a whole:



Source systems – data from external sources which are fed to the DW system
BI Analytical Reporting System – BI system used for analytical reporting, which consists of its own
modules, and provides analytical reporting functionalities for BI/DW system users
External reporting systems – other external systems, which use DW as source system for their needs. They
can use data stored in Target area, often in Enterprise Model area, or separate areas prepared specially
for their needs.
5
Business Intelligence Solutions
Schema on following picture shows above-mentioned logical DW system components.
NEOS as DW/BI implementation partner
Neos as implementation partner can be engaged in different roles based on customer requirements and system
specifics. Neos consultants involvement is ranging to purely consultancy services related to Business/technical
aspects to detail technical implementation of different components.
In this chapter we are providing you with general overview of project methodology defining project phases on high
level based on our internal DW/BI project methodology.
Main project is divided in phases based on each delivery (Subject Area), and each phase can be divided in sub phases
from implementation point of view as described below.
Project kick-off
Purpose of this phase is to:



Achieve mutual understanding and agreement project goals
Introduction about general project phases, project team structure and expectation on Client
resources in each project phase.
Present project organization and to assign roles.
Reporting requirements analysis
This is initial phase related to detail analysis of reporting requirements defined by business users.
Idea is to analyze all existing and to-be reporting specifications, to identify required data on lowest level of
granularity and to define KPIs.
6
Business Intelligence Solutions
Methods
-
Gathering information about data reporting requirements and required
dataset
-
Analysis of existing reports and specifications provided by business users
-
Meetings and interviews with targeted business area representatives
Goals
-
Understanding and documenting all reporting requirements
To identify required basic data set on lowest level of granularity
Definition of list of KPIs and calculation algorithms
Expected results
-
List of required subject areas and reporting requirements
List of KPIs and calculation algorithms
Definition of required dashboards for each user profile
Source system data analysis
Main goal of this phase is to analyze data availability, quality and amount of required data in source IT systems (this
has already been done in previous project phase – but in this case focus will be on data quality and availability from
technical point of view).
Methods
-
Goals
-
Expected results
-
-
Gathering information about data availability, quality, data owners etc.
Analysis of existing documentation about source systems
Meetings and interviews with IT staff for each source system containing
data needed for future DW
Understanding and identifying data sources for each subject area and
data flows
Mapping data from source system to each specific attribute in data
model (transformations, data quality checks, etc.)
Data acquisition approach definition
Detailed mapping for each attribute in target data model having
following information
- Source system table/column/procedure
- Needed transformations
- Data quality rate
- Expected data domain
- Additional comments
Data availability analysis identifying gap between required and existing
(available) data in source systems
Infrastructure setup
In this phase all required infrastructure SW should be installed, configured and prepared for development, testing
and production platform.
Methods
-
Installation of required SW modules
Configuration of installed modules based on estimated system scale
Goals
-
To install and setup all required SW components
To create ODI and OBI repositories
Setup of development, test and production infrastructure
Expected results
-
Fully functional development, test and production environment
Data model design/customization and implementation
This phase should result in logical and physical data model based on business/reporting and technical requirements.
7
Business Intelligence Solutions
Methods
-
Goals
-
Expected results
-
-
Data modeling based on reporting/business requirements and technical
demands.
Data model physical implementation will be developed based on
available infrastructure resources in order to comply with best practices
on maintenance practices, scalability and performance gains
Logical data model design and physical implementation of full data
model in all DWH stages
Each entity/attribute identified in analysis phase to be revised by
availability and technical point of view
Each data model entity to be defined on all physical aspects regarding to
Oracle database available options
To define specific requirements on database parameterization and
configuration
Each data object to be fully defined and described in OWB repository
(tables, columns, comments, tablespaces, physical parameters, indexes,
partitions etc.)
Detailed SQL scripts for initial object creation
Partition schema defined for each candidate table (if partition option
used on DB)
Materialized view or aggregate table definitions based on expected
common queries
ETL/ELT Procedures implementation
This phase should result in fully functional and documented procedures for Extraction, Transformation and Load
data from source system to DWH.
These procedures are to be developed based on detailed mapping definition developed in previous phase.
Methods
-
Goals
-
Expected results
-
Implementation of ETL procedures in Oracle Data Integrator or other
preferred ETL tool
Procedures will be developed using DW Best practices based on
experience and standards
Implementation of ETL procedures for each subject area
Initial testing and optimization of all developed procedures
Implementation of scheduling and logging mechanism based on source
system lifecycle limitations and requirements
Implementation of data quality checks and alerts based on business
definitions
Fully functional, documented and tested set of ETL modules - ODI
interfaces, packages and scheduling definitions
Scripts for creating initial versions of ETL objects
Scripts for initial load prepared
All defined data quality checks implemented
Design and implementation of Cubes and aggregate tables
In this phase goal is to design and implement all aggregate tables and OLAP cubes in order to have pre-aggregated
and pre-calculated measures on higher level of hierarchies and different dimensions combinations.
Methods
-
Goals
-
Design of aggregate tables and olap cubes based on reporting
requirements and query performance requirements
Design and implementation of cube loading procedures
Definition of required aggregate tables and olap cubes in order to speedup ad-hoc analysis and predefined reports execution
Creation of loading procedures based on reporting requirements and
technical limitations
8
Business Intelligence Solutions
Expected results
-
Designed and implemented aggregate tables and olap cubes
Designed and implemented loading procedures
Reporting metamodel design and implementation (OBI)
This phase includes design and implementation of OBI repository including definition of physical, business and
presentation layer. Each fact table should have measures, dimensions and aggregate tables/cubes defined, and each
dimension should have attributes and hierarchies designed and implemented.
Methods
-
OBI repository creation and setup
Import/design of physical layer
Definition of business layer and mapping data sources form physical
layer
Defining of presentation layer based on user defined labels and
hierarchies
To enable data analysis and report creation by end users using defined
OBI object
To describe full physical data model in reporting metamodel
-
OBI repository created and configured
Physical, Business and Presentation layer created
-
Goals
Expected results
-
Design and implementation of basic set of reports and dashboards
In this phase, al required predefined reports and dashboards are created using OBI metamodel defined in previous
phase.
Methods
-
Reports creation using OBI Answers and Dashboards
Pixel perfect reports creation using BI Publisher
Reports testing from logical and performance point of view
Goals
-
Implementation of predefined set of reports and dashboards
Implementation of predefined pixel-perfect reports
All predefined reports implemented and tested
Fully functional reporting system
Expected results
Security policy implementation (horizontal and vertical data access policies)
This phase includes users and roles definition – and definition of security roles and policies on data access
implementation. This includes horizontal and vertical data access privileges.
Methods
-
User profiles and security policies analysis
Implementation of horizontal and vertical data access policies
Testing and optimization of implemented rules
Goals
-
Implementing different user roles based on business definitions
Limiting data access based on user profiles
Users and roles created and tested
All security policies implemented
Expected results
Exposing Rollout - final testing, optimization and knowledge transfer
This phase goal is to perform potentially needed optimization and fine tuning of already implemented modules in
previous phases.
Also, this phase will result in final documentation regarding daily maintenance and operation of the system.
This phase is generally described as period in which system is fully functional and monitored by Client IT and Service
provider in order to ensure that there is no specific problems in system functioning.
9
Business Intelligence Solutions
Methods
Monitoring daily system activities using logs and other monitoring tools.
Identifying potential performance problems and tuning.
Detailed documentation of daily processes and procedures.
Goals
-
Expected results
-
Ensure that system is fully functional comparing to initially refined
requirements
Knowledge transfer on Client IT staff in charge to daily maintenance
To identify possible performance and data quality problems and fix
them.
Detail documentation about daily activities and procedures
All modules functioning in optimal way
Client IT staff been able to maintain system with minimal engagement
NEOS as DW/BI education partner
Besides DW/BI modules implementation, Neos is author and owner of NEOS DW/BI Framework specific hands-on
training related to DW architecture, development and best practices, which was held to Oracle partners and
customers. Neos experts are also Oracle University instructors on standard Oracle technology tools related to DW
and ETL.
Training/Education is in form of:

Presentations lead by instructor with examples demonstrated on demo environment;

Practices executed by participants on local environment or demo server;

Questions answered on specific topics of interest.
In DW & BI area, training is intended for implementators, as well as business and technical users of Neos DW/BI
Framework and related Neos solutions, giving overview on basic architecture, modules and functionalities with detail
technical insight and recommendations based on implementation experiences and best practices.
Target audiences are technical consultants/implementators with basic DW/BI knowledge about related business
concepts.
Training is performed by consultants having hands-on experience on implementation of OFSAA modules.
Brief description of the training schedule and topics that are covered during training:

DW/BI concepts

Neos approach in DW/BI projects

ETL/ELT concepts

Neos DW/BI Framework Modules and Concepts

Data Quality in DW projects

Administration and Monitoring
10
Neos – Contact Us
Neos DW/BI Framework
Neos DW/BI Framework is product based on over more than 10 years of experience in projects of DW/BI system
design and implementation. Main goal is to leverage pre-built modules and best practices in all project phases
guaranteeing successful implementation and future flexibility and system maintainability.
Key components are: Design & Implementation Standards, Predefined Industry Specific Data models and
Dashboards/Reports, ETL Tool Interface Adapters, Unified ETL Repository, Execution and Maintenance Agents,
Administration Console.
Best Pract ices
Years Of Experience
All in One Package
Neos DW/BI Framework is developed using Oracle technology stack, providing data model and DW platform that is
highly customizable and can be easily expanded to suit company's needs as business expands and evolves.
Based on best practices and long-term experience, Neos DW/BI Framework has large number of pre-built
components, enabling rapid DW&BI project start and development, without compromising quality.
Key facts
NEOS DW/BI Framework is ExaStack Ready – Exadata and Exalytics fully compatible.
Data Models






Industry Specific Ready Data Models
Naming Conventions
Object Templates
Physical Parameters (Tables, Tablespaces, Sequences..)
Standards for partitioning, compression
Database Configuration Parameters
ETL



Best Practice Reusable Algorithms
Standardized ETL Module Parameters
Predefined Modules
Reports



Reporting Tool Configurations
Predefined Industry Specific Dashboards & Reports
Metamodel Best Practices
11
Neos – Contact Us
NEOS DW/BI Framework components
ETL Tool Integration


ETL Tool Repository Integration
o Extension to Standard ETL Tools Repository
o Supported Tools - OWB, ODI ...
o Reading Dependencies, Object Definitions Directly From Repository
o Additional Definition & Control Parameters
o Integration with non-Oracle tools and technology
o Benefits
Full Integration with ETL Repository
o Additional Functionalities
o Easier Maintenance
o Easier Development
Unified repository


Repository
o Process Execution Control Parameters
o Loaded from ETL repository and Manual Definitions
o Supports different types of execution and monitoring controls
o Integrated with ETL Tool and Dev Interfaces
Benefits
o Automatically Updated
o Ensures ETL Process Execution Consistency
o Supports Multiple ETL Tools
o Additional Logging and Monitoring Parameters
Standards and Dev Interfaces



Standards
o Architecture & Modeling Standards
o ETL Tool Specific Development Standards
o Best Practice Configurations
o Flexible Modeling Techniques
ETL
o Predefined pre/post ETL Module Procedures
o Best Practice Templates
Benefits
o Solutions Based on Best Practices
o Reusability and Improved Efficiency
o Developers can Focus on ETL Specific Business Requirements - not on System tasks
o Automatically Ensured Process Execution Consistency
o No need for additional parameterization of ETL process Prerequisites
Agents



Plan & Execute
o RWYC Principle Based Execution
o Dynamic Execution Plans
o Based on Prerequisites Automatically Identified from ETL Repository
o Configurable (Priorities, Execution Times.. )
Clean & Utils
o Automatic Cleaning of Processed Data (based on ETL execution status)
o Automatic and ETL aware Statistics Calculation
o Configurable on Lowest Level
Logs & Stats
o Additional Process Execution Statistics
o Detail logs of execution times, type of operations
o Base for Performance Issues Analysis
12
Neos – Contact Us
Data Quality
Business Entities & Rules



Defined By Business Users
Rules based on Business Terms
Horizontal & Vertical Rules
Configs/Mappings




Mappings from Business Terms to Physical Data Dictionary
Support for multiple Databases - Reusability
Errors/Actions Configurations
Checks Executed on Remote Databases
Logs & Stats



Summary Info on Central Repository
Detail info on Local Database
Row Level Info on Errors/Warnings
Integration/Execution


Scheduled Data Validation
ETL Initiated Rules Check
Administration & Monitoring Console
Monitor





Active Processes
o What is Executing now and for how long?
Execution Prerequisites
o Which processes should already be started and
why they did not start?
Execution Status
o Which processes have been finished and in which status (Error, Success)
Execution Dependency
o Which process is dependent of which process/prerequisites?
Error resolving
o What is Error cause and where?
Administration





Process Parameterization
o Execution periods, Priority, Starting Time
Start/Stop
o Manually Start/Stop processes
Loaded Data Overview
o Insight in loaded periods
Resolving problems
o Easily identify cause of problems
Process Control
o Run/Hold/Stop
Statistics




Process Execution Times
o Trend, Top N, Identify Problematic Executions
Starting times
o Overview of Usual Starting Time
Activity Types
o Insight in execution time on Select/Update/Insert
Detail Execution Times on ETL process parts
13
Neos – Contact Us


Number of Records
o Number of loaded/processed records, Trends
Top N Analysis
o Identify top n most demanding processes
Technology Expertise
Following is the list of technologies used in DW/BI system implementation on which we are having long time
experience.
General Overview
Sources

Any structured or non-structured data source (any database, flat files…)
Target and Repository

Oracle Database 11g - Enterprise Edition recommended
ETL/ELT



Oracle Data Integrator 11g
Oracle Warehouse Builder 11g
Informatica
OLAP

Oracle Essbase 11g
Analytics & Reporting



Oracle Business Intelligence Suite enterprise Edition (OBI EE)
SAP Business Objects
Compatible with other reporting tools
Oracle Technology Stack
Here we are going to describe recommended Oracle Tools/Technology stack used for DW/BI system implementation
based on Oracle platform.
Database: Oracle Database Enterprise Edition (With Partition Option)
Oracle Database 11g is a comprehensive database platform for data warehousing and business intelligence that
combines industry-leading scalability and performance, deeply-integrated analytics, and embedded integration and
data-quality -- all in a single platform running on a reliable, low-cost grid infrastructure.
Oracle Database 11g provides best-of-breed functionality for data warehouses and data marts, with robust
partitioning functionality, proven scalability to 100's of TBs, and innovative query-processing optimizations. Oracle
Database 11g also provides a uniquely integrated platform for analytics; by embedding OLAP, Data Mining, and
statistical capabilities directly into the database, Oracle delivers all of the functionality of standalone analytic engines
with the enterprise scalability, security, and reliability of the Oracle Database.
ETL: Oracle Data Integrator
Oracle Data Integrator (ODI) is a best-of-breed data integration platform focused on fast bulk data movement and
handling complex data transformations.
Oracle Data Integrator is fully integrated with the Oracle technology stack, including Oracle Database, Exadata
Database Machine, Oracle WebLogic Server, Business Intelligence, and Oracle Applications. ODI is the strategic data
integration platform for Oracle.
Oracle Data Integrator is built on several components all working together around a centralized metadata repository.
These components – graphical modules, runtime agents and web-based interfaces – in conjunction with other
14
Neos – Contact Us
advanced features make ODI a lightweight, legacy-free, state of the art data integration platform. This technical brief
describes the Oracle Data Integrator architecture in detail.
Architecture Overview
The ODI architecture is organized around a modular repository, which is accessed in client-server mode by
components such as the ODI Studio and execution Agents that are written entirely in Java. The architecture also
includes a web-based application, the ODI Console, which enables users to access information through a Web
interface and an extension for Oracle Fusion Middleware Control Console.
ODI Studio provides four graphical Navigators for managing ODI artifacts: Designer, Operator, Topology and Security.
ODI Studio offers an easy to use graphical user interface and can be installed on various platforms such as Microsoft
Windows, Linux and Mac OS.
ODI Studio Navigators connect to the repository
The ODI Studio Navigators are as follows:
o
o
o
o
Designer defines declarative rules for data transformation and data integrity. All project development takes
place in this module; this is where database and application metadata are imported and defined. The
Designer module uses metadata and rules to generate data integration scenarios or load plans for
production. This is the core module for developers and metadata administrators.
Operator manages and monitors data integration processes in production. It is designed for operators and
shows execution logs with error counts, the number of rows processed, execution statistics, the actual code
that is executed, and so on. At design time, developers can also use the Operator module for debugging
purposes.
Topology defines the physical and logical architecture of the infrastructure. The infrastructure or projects
administrators register servers, database schemas and catalogs, and agents in the master repository
through this module.
Security manages user profiles and their privileges. Security can also assign access authorization to objects
and features. Security administrators generally use this module.
All modules store their information in the centralized repository.
Runtime Components: The Agents
At runtime, the Agent coordinates the execution of the ODI scenarios. It retrieves the code stored in the ODI
repository, connects to the various source and target systems and orchestrates the overall data integration process.
There are two types of ODI Agents:
o
The Standalone Agent can be installed on the source or target systems and requires a Java Virtual Machine.
15
Neos – Contact Us
o
The Java EE Agent is deployed on Oracle WebLogic Server and can benefit from the application server layer
features such as clustering for High Availability requirements.
With the Extract-Load Transform (E-LT) architecture, the Agent rarely performs any transformation. It simply
retrieves code from the ODI repository and then requests database servers, operating systems, or scripting engines
to execute that code. When the execution is completed, the Agent updates the execution logs in the repository and
then reports error messages and execution statistics. Users can review the execution logs from the Operator
Navigator, the ODI Console Web interface or Oracle Fusion Middleware Control Console.
It is important to understand that although the Agent can act as a transformation engine, it is rarely used for that
purpose. Agents are installed at tactical locations in the information system to coordinate the integration processes
and leverage existing systems. They are multithreaded, load-balanced, lightweight components in this distributed
integration architecture.
Figure 2 - Standalone Runtime Components
Repositories
The Repository consists of a Master Repository and typically several Work Repositories. These repositories are sets
of tables stored in relational database management systems such as Oracle, Microsoft SQL Server, IBM DB2 and
others. All objects that the ODI modules configure, develop, or use are stored in one of these repositories, and are
accessed in client-server mode by the various components of the architecture.
The Master Repository contains the security information (user profiles and privileges), the topology information
(definitions of technologies and servers), and the source code for all versions of all ODI objects ever versioned. The
information contained in the Master Repository is maintained with Topology and Security Navigators in ODI Studio
as well as with ODI Console.
16
Neos – Contact Us
Figure 3 - Master repository and Work repositories
Project objects are stored in a Work Repository. Several Work Repositories can coexist in the same installation. This
is useful for maintaining separate environments or to reflect a particular versioning lifecycle—for example,
development, quality assurance, and production environments.
A work repository stores information for
o
o
o
Models (i.e. metadata)—including datastores, columns, data integrity constraints, cross references, data
lineage and impact analysis
Projects—including interfaces, packages, procedures, folders, knowledge modules, and variables
Runtime information—including scenarios, load plans, scheduling information, and logs
Users manage the content of a work repository with the Designer and Operator Navigators in ODI Studio. The Agent
at runtime also accesses work repositories. When a Work Repository is used only to store execution information
(typically for production purposes), it is called an execution repository. An execution repository is accessed at
runtime with the Operator Navigator, ODI Console and by the Agents. It is important to remember that each Work
Repository is always attached to one and only one master repository.
ODI Console is a Java Enterprise Edition (Java EE) application that provides Web access to repositories. It allows users
to browse Design-Time objects, including projects, models, and execution logs. Through its comprehensive Web
interface, users can see flow maps, trace the source of all data, and even drill down to the field level to understand
the transformations used to build the data. In addition, end users can launch and monitor scenarios execution
through ODI Console.
ODI Console can be installed on Oracle WebLogic Server and also provides administrators with the ability to view
and edit Topology objects such as Data Servers, Physical and Logical Schemas as well as to manage their repositories.
Oracle Data Integrator offers an extension for the Oracle Enterprise Manager Fusion Middleware Control Console
which allows end users to monitor their ODI components along with other Fusion Middleware modules from a single
administration console.
17
Neos – Contact Us
Figure 4 - ODI Console and Enterprise Manager Fusion Middleware Control Console
Other Features
Oracle Data Integrator also includes the following components and features:
o
o
o
o
o
o
Knowledge Modules make it possible to quickly and easily integrate technologies, databases, and
applications. They exist for a large range of platforms, including Oracle, Exadata, Big Data, Oracle
Applications, Oracle GoldenGate, Teradata, Netezza, SAP ERP, SAP BW and XML.
The Changed Data Capture (CDC) feature tracks changes in source systems and reduces the volume of
processed data by extracting only the changed data. ODI can use native database features to capture
changes or Oracle GoldenGate.
The integration with Oracle Business Intelligence Enterprise Edition provides business users with report-tosource data lineage.
Advanced Web Services capabilities offer best-in-class integration with Service-Oriented Architecture (SOA)
environments
The Load Balancing feature enables large volumes of data to be processed by automatically balancing the
workload between several Agents.
Advanced Version Management provides an interface to manage, safeguard, and replicate revisions of units
of work, even in the largest development environments.
Conclusion
Oracle Data Integrator is a lightweight, legacy-free, state-of-the-art data integration platform. With its superior
performance and flexible architecture, Oracle Data Integrator can be used in various types of projects such as Data
Warehousing, SOA, Business Intelligence or Application Integration.
Reporting & Analysis: Oracle Business Intelligence Enterprise Edition
Generally speaking there is no specific limitations on customer selection of front end user tool for Reports viewing,
ad-hoc queries, dashboards etc.
Data model should be modeled by design principles which are not subject to specific front end tools in order to
support future scalability.
18
Neos – Contact Us
In this case, we have Database and ETL tool on Oracle platform and therefore logical choice for front end tool (client
tool) would be Oracle BI EE set of tools.
Besides large set of functionalities and possibilities, one of main advantages is architecture synergy since all products
are from same vendor and can be easily integrated and maintained
Here is a broad overview of Oracle BI components, from an end-user perspective.
Oracle BI EE consists of following components:
Oracle BI Presentation Services. Oracle BI Presentation Services consists of the following interfaces:
-
Oracle Business Intelligence Answers
-
Oracle Business Intelligence Delivers,
-
Oracle Business Intelligence Interactive Dashboards
-
MS Office Plugins
Oracle Business Intelligence Answers: Oracle BI Answers provides answers to business questions. This interface
allows users with the appropriate permissions to build and modify reports, also called requests, that let end users
explore and interact with information, and present and visualize information using charts, pivot tables, and reports.
Your organization may also have purchased prebuilt reports specific to your industry.
The results of an Oracle BI Answers request can be formatted, saved, organized, and shared with others. A report
can be configured to refresh results in real-time.
Reports created with Oracle BI Answers can be saved in the Oracle BI Presentation Catalog and integrated into any
Oracle BI home page or dashboard. Results can be enhanced through options such as charting, result layout,
calculation, and drilldown features.
Oracle Business Intelligence Interactive Dashboards: Interactive Dashboards provide points of access for analytics
information. When an end user accesses Oracle BI, the user’s default dashboard is typically the first page that
appears. Dashboards are typically used to display reports that contain content specific to the needs of individual
users or groups.
19
Neos – Contact Us
Historical and current data sources can be merged into a single dashboard. Users with the appropriate permissions
can place results from Oracle BI Answers into dashboards for use by end users.
Oracle Business Intelligence Delivers: Oracle BI Delivers is the interface used to create Oracle Business Intelligence
Alerts based on analytics results. Specific results can be detected within reports and the appropriate people notified
immediately through Web, wireless, and mobile communications channels. Oracle BI Delivers uses intelligence bots
called iBots to detect specific results. iBots are software-based agents, driven by schedules or events that can access,
filter, and perform analytics on data based on specified criteria. Users with the appropriate permissions can use
Oracle BI Delivers to set up the conditions to trigger an alert. For example, if a user should be notified when a
particular event occurs, such as customer account going critical, an alert can be created that will notify the user
through a number of delivery options, such as email or cell phone.
20
Neos – Contact Us
NEOS
is helping its clients reach new levels in business excellence.
Be one of them.
21