Aster Development Environment
User Guide
Release Number 5.10
Product ID: B700-6030-510K
May 2013
The product or products described in this book are licensed products of Teradata Corporation or its affiliates.
Teradata, Active Data Warehousing, Active Enterprise Intelligence, Applications-Within, Aprimo, Aprimo Marketing Studio, Aster, BYNET,
Claraview, DecisionCast, Gridscale, MyCommerce, Raising Intelligence, Smarter. Faster. Wins., SQL-MapReduce, Teradata Decision Experts,
"Teradata Labs" logo, "Teradata Raising Intelligence" logo, Teradata ServiceConnect, Teradata Source Experts, "Teradata The Best Decision Possible"
logo, The Best Decision Possible, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the
United States and other countries.
Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.
AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.
Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation
in the United States and/or other countries.
Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access,
Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum
Support are servicemarks of Axeda Corporation.
Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.
GoldenGate is a trademark of Oracle.
Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.
Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other
countries.
Intel, Pentium, and XEON are registered trademarks of Intel Corporation.
IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation.
Linux is a registered trademark of Linus Torvalds.
LSI is a registered trademark of LSI Corporation.
Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States
and other countries.
NetVault is a trademark or registered trademark of Quest Software, Inc. in the United States and/or other countries.
Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.
Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates.
QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.
Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license.
SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.
SPARC is a registered trademark of SPARC International, Inc.
Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and
other countries.
Unicode is a registered trademark of Unicode, Inc. in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other product and company names mentioned herein may be the trademarks of their respective owners.
THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN "AS-IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION
MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL,
OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
The information contained in this document may contain references or cross-references to features, functions, products, or services that are not
announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions,
products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or
services available in your country.
Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated
without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time
without notice.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document.
Please email: teradata-books@lists.teradata.com.
Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata
Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform,
create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata
Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including
developing, manufacturing, or marketing products or services incorporating Feedback.
Copyright © 2000-2013 by Teradata Corporation. All Rights Reserved.
Table of Contents
Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Chapter 2: Installing Aster Development Environment . . . . . . . 7
Installation Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Verify Java Development Kit (JDK) version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Verify Eclipse IDE version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Add Text File Associations to Support UTF-8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Installing ADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Updating ADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Removing ADE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Troubleshooting Linux ADE Installation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Chapter 3: Using Aster Development Environment . . . . . . . . . . . 15
Creating and Deploying an ADE Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Creating an Aster Database Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Configuring a Database Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Adding Third-Party JARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Creating a SQL-MapReduce Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Creating the SQL-MapReduce Function Stub Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Defining the SQL-MR Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Creating an Input Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Creating Output Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Adding SQL-MapReduce Function Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Testing SQL-MapReduce Functions Locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Creating a Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Creating Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Running the Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Creating Queries to Invoke SQL-MapReduce Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Creating a Sample Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Creating a Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Compiling, Deploying, and Running. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Aster Development Environment User Guide
3
Managing Functions and Files on Aster Database Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Exporting SQL-MapReduce Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Making SQL-MapReduce Functions Available to Any Aster Database Cluster . . . . . . . . 36
Viewing Aster Database Cluster Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Uploading Functions and Files to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Downloading, Removing, or Moving Files on Aster Database . . . . . . . . . . . . . . . . . . . . . . 39
Downloading a File or Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Removing a File or Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Moving a File or Function From One Cluster to Another . . . . . . . . . . . . . . . . . . . . . . 40
Managing Database Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4
Aster Development Environment User Guide
CHAPTER 1
Introduction
Aster Development Environment (ADE) is a development environment that lets you write,
test, and deploy your SQL-MapReduce functions on Aster Database. Teradata Aster provides
ADE as a plug-in that you can use with Eclipse IDE for Java Developers.
This document describes how to install and use ADE and guides you through the process of
creating and deploying SQL-MapReduce functions.
For more information about SQL-MapReduce functions, see “SQL-MapReduce Java API” in
the Aster Database User Guide.
Tip: If the ISO does not appear in the Browse list, make sure the.iso file has been copied to the VM host
machine.
Aster Development Environment User Guide
5
Introduction
6
Aster Development Environment User Guide
CHAPTER 2
Installing Aster Development
Environment
•
“Installation Prerequisites” on page 7
•
“Installing ADE” on page 9
•
“Updating ADE” on page 12
•
“Removing ADE” on page 13
•
“Troubleshooting Linux ADE Installation Issues” on page 13
Note: The Eclipse procedures in this section are for the Windows operating system. Both
Eclipse and Aster Development Environment run on Windows, MAC OS X, and Linux,
although the specific steps for each operating system are not listed here.
Installation Prerequisites
Before installing the ADE the following Eclipse prerequisites must be met.
Verify Java Development Kit (JDK) version
Before installing the ADE, install or verify that the version of your Java Development Kit
(JDK) is 32-bit, and not 64-bit. Ensure it is installed and running on your Windows XP,
Windows 7, MAC OS X, or Linux system.
Verify Eclipse IDE version
Before installing the ADE, install or verify the correct version number of the Eclipse IDE for
Java Developers.
•
Ensure that version 3.5.1 or higher of Eclipse IDE for Java Developers is installed and running
on your Windows XP, Windows 7, MAC OS X, or Linux system.
Add Text File Associations to Support UTF-8
If your Java text files contain Unicode characters for languages such as Chinese or Japanese,
you must configure support for UTF-8. Once UTF-8 encoding is set, the Java files can be saved
as UTF-8 in the ADE and the Unicode characters will display correctly in both the window
Aster Development Environment User Guide
7
Installing Aster Development Environment
Installation Prerequisites
console and the corresponding output file. In order to support the UTF-8 character set, two
new text file associations must be added and configured in Eclipse:
•
•
*.in (for the default input file)
*.out (for the default output file)
These files allow a developer to create a TestRunner test file in Eclipse in order to test code
locally as you work on it, without an Aster Database connection. See “Testing SQLMapReduce Functions Locally” on page 26 for more information.
To add text file associations to support UTF-8:
8
1
Launch Eclipse and navigate to Window > Preferences.
2
In the Preferences window navigate to General > Content Types > Text > and click the Add
button in the File associations section.
3
In the Add Content Type Association dialog box enter “*.in” and click OK.
4
In the Default encoding dialog box enter “UTF-8” and click Update.
5
Repeat the previous two steps to add “*.out” content type and its “UTF-8” default encoding.
Aster Development Environment User Guide
Installing Aster Development Environment
Installing ADE
The File associations section of this window displays the two added files and they may be
edited or removed as necessary. This screen also displays three additional content types (*.sql,
*.test, and *.txt) that are added by default. Theses default types are locked and cannot be
removed or edited.
Installing ADE
The ADE installation process described here is for an installation on Windows. ADE runs on
Windows XP, Windows 7, MAC OS X, and Linux.
To install ADE:
1
Obtain the Aster Database ADE plugin package (ADE-plugin-only.zip).
2
Extract the contents of ADE-plugin-only.zip to a folder on your system.
For example, if you extract the contents of the ZIP file to C:\ on Windows, the extraction
tool creates the C:\ADE-plugin-only folder and stores the ADE installer in it.
3
(For Linux Only) Set the GDK_NATIVE_WINDOWS environment variable.
export GDK_NATIVE_WINDOWS=1
4
Launch Eclipse and choose Help > Install New Software.
5
In the Install window, click Add.
6
In the Add Site dialog box, click Local.
7
In the Browse for Folder dialog box, select the ADE-plugin-only folder you created in Step 2
and click OK.
8
(Optional) In the Name field, enter a name of the site where the plugin is added and click
OK.
An Aster Data entry appears in the list of available software.
If not, from the Work with drop-down menu, choose --All Available Sites-- or the name of the
site you created in this step. The Aster Data entry appears in the list.
9
Expand the Aster Data node and check the Aster Development Environment check box and
click Next.
Aster Development Environment User Guide
9
Installing Aster Development Environment
Installing ADE
10 In the Install Details screen, select Aster Development Environment from the list.
11 Click Finish to start the plugin installation process.
12 When prompted to continue the installation process, click OK.
This warning message appears because the ADE plugin contains unsigned content.
13 If the “Do you trust these certificates?” dialog box appears, check the check box to accept
the Eclipse.org Foundation certificate and click OK.
14 Click Yes to restart Eclipse.
15 After Eclipse restarts, choose Other from the Perspective Chooser drop-down menu.
10
Aster Development Environment User Guide
Installing Aster Development Environment
Installing ADE
16 In the Open Perspective window, choose Aster SQL-MapReduce and click OK to activate the
Aster Development Perspective and finish the ADE installation.
17 To confirm that ADE is installed, open the File > New menu.
The following items are visible in Eclipse:
•
Aster SQL-MapReduc Project
•
Aster SQL-MapReduce Function
•
Aster SQL File
•
Aster SQL-MapReduce Test Configuration
Go to “Using Aster Development Environment” on page 15 for instructions on how to create a
SQL-MapReduce function.
Aster Development Environment User Guide
11
Installing Aster Development Environment
Updating ADE
Updating ADE
To update ADE:
12
1
Launch Eclipse.
2
Choose Help > Install New Software.
3
In the Install window, click the What is already installed? link.
4
In the Installation Details window, click Aster Development Environment.
5
Click Update.
Aster Development Environment User Guide
Installing Aster Development Environment
Removing ADE
Removing ADE
To remove ADE from Eclipse:
1
Launch Eclipse.
2
Choose Window > Close All Perspectives.
You cannot remove ADE if the Aster Development perspective is not closed.
3
Choose Help > Install New Software.
4
In the Install window, click the What is already installed?.
5
In the Installation Details window, click Aster Development Environment.
6
Click Uninstall.
7
In the Uninstall window, click Finish.
Troubleshooting Linux ADE Installation Issues
•
If the GDK_NATIVE_WINDOWS environment variable was not set in Step 3 in the
Installing ADE section, there may be buttons in ADE that do not show up or do not
respond when clicked or selected. This known issue affects the ADE on Ubuntu 9.10.
To resolve this issue:
a
Set the GDK_NATIVE_WINDOWS variable in the command-line:
export GDK_NATIVE_WINDOWS=1
b
Run Eclipse or run ADE directly.
To run ADE directly, use this CLI command:
./AsterDevelopmentEnvironment
•
If errors occur when launching ADE, check GTK permissions on Ubuntu.
To resolve this issue, run this command:
gksudo ./AsterDevelopmentEnvironment
Aster Development Environment User Guide
13
Installing Aster Development Environment
Troubleshooting Linux ADE Installation Issues
14
Aster Development Environment User Guide
CHAPTER 3
Using Aster Development
Environment
•
“Creating and Deploying an ADE Project” on page 15
•
“Creating an Aster Database Project” on page 15
•
“Creating a SQL-MapReduce Function” on page 19
•
“Testing SQL-MapReduce Functions Locally” on page 26
•
“Creating Queries to Invoke SQL-MapReduce Functions” on page 31
•
“Compiling, Deploying, and Running” on page 34
•
“Managing Functions and Files on Aster Database Clusters” on page 35
•
“Managing Database Connections” on page 40
Creating and Deploying an ADE Project
An ADE project serves two purposes:
•
It is the container that holds your SQL-MapReduce code, related libraries, and tests.
•
It provides the connection detail that allows your code to connect to Aster Database.
Creating an Aster Database Project
To create an ADE project:
1
Launch Eclipse.
2
Choose File > New > Project.
The New Project wizard window appears.
Aster Development Environment User Guide
15
Using Aster Development Environment
Creating an Aster Database Project
16
3
In the Select a wizard screen, select Aster SQL-MapReduce Project and click Next.
4
In the Create an Aster SQL-MapReduce Project screen:
a
In the Project name field, enter the name of the project (only use letters, numerals, and
hyphens).
b
If you have not configured an Aster Database connection, or if you need to configure a
new one, click the Configure Aster Database link and follow the steps in “Configuring a
Database Connection” on page 18 before proceeding to the next step.
c
To set this project to always use a particular database connection, click the Use a specific
Aster Database radio button and choose the database from the drop-down menu and
click Next.
d
Alternatively, to use the currently selected database connection in your Eclipse project,
click the Use a workspace-wide Active Aster Database radio button and click Next.
e
In the Java Settings screen, click Finish.
Aster Development Environment User Guide
Using Aster Development Environment
Creating an Aster Database Project
f
If prompted to confirm whether to open the Aster SQL-MapReduce perspective, click
Yes.
The new project appears in the Package panel.
g
5
To display that the Aster Database SQL-MapReduce libraries, expand the project.
(Optional) To add third-party project libraries or other prebuilt code to your project,
follow the steps in “Adding Third-Party JARs” on page 19.
You are now ready to create a SQL-MapReduce function.
Aster Development Environment User Guide
17
Using Aster Development Environment
Creating an Aster Database Project
Configuring a Database Connection
The Preferences (Filtered) window is where the new Aster Database connection is added and
configured. The Configure Aster Database link in the Create an Aster SQL-MapReduce Project
screen, lets you add and configure new Aster Database connection.
To add and configure a new Aster Database connection:
18
1
In the Preferences (Filtered) window, click Add.
2
In the Add New Aster Database Connection window, complete the following fields:
•
Name: The name of the database connection.
•
Hostname: The hostname or IP address of the Aster Database queen that hosts that
database.
•
Database: The database name.
•
Username and Password: The username and password for accessing the database.
3
To ensure that the ADE project can connect to the database, click Check connection.
4
To add the connection, click OK.
Aster Development Environment User Guide
Using Aster Development Environment
Creating a SQL-MapReduce Function
By default, the connection that you add becomes the active connection. If not, or if you
want to make another connection active, select it and click Set as Active.
5
To close the Preferences (Filtered) window click OK.
Adding Third-Party JARs
To add third-party libraries or other prebuilt code to your project:
1
Right-click the name of your project in the Package panel, and choose Build Path > Add
External Archives.
2
Select the jars you want to include and click Open.
The jar appears in your project under Referenced Libraries.
3
Right-click your project and choose Build Path > Configure Build Path.
4
In the Properties window, click Java Build Path in the navigation tree.
5
In the Order and Export panel, use the Up/Down buttons to change the order of the library
in the list.
6
Select the library by clicking its check box.
7
Click OK.
Creating a SQL-MapReduce Function
To create a SQL-MapReduce (SQL-MR) function, perform these steps:
•
“Creating the SQL-MapReduce Function Stub Code” on page 19
•
“Adding SQL-MapReduce Function Implementation” on page 24
Creating the SQL-MapReduce Function Stub Code
To create the SQL-MapReduce function stub code, perform these steps:
•
“Defining the SQL-MR Function” on page 20
•
“Creating an Input Schema” on page 21
•
“Creating Output Columns” on page 23
Aster Development Environment User Guide
19
Using Aster Development Environment
Creating a SQL-MapReduce Function
Defining the SQL-MR Function
To define a SQL-MR function:
1
In Eclipse’s Package panel, select your ADE project.
2
Choose File > New > Aster SQL-MapReduce Function.
The New Aster Data Function wizard appears.
3
In the SQL-MapReduce Function screen:
a
(Optional) in the Package field, enter the name of the package or click Browse, select a
package and click OK.
b
In the Name field, enter the name of the function.
c
Select the interfaces that this function implements by checking the check boxes that
apply. In this example, select Single Input RowFunction.
For information about row, partition, and multiple input functions, see “Introduction
to SQL-MapReduce” in the Aster Database User Guide.
The use of the default package is discouraged when Single Input is selected.
d
20
Click Next.
Aster Development Environment User Guide
Using Aster Development Environment
Creating a SQL-MapReduce Function
4
In the Argument clauses screen, set up the function to accept arguments from the query
that calls the function.
These are the parameter settings that you provide so that the user can specify how the
function should operate in the context of the user’s query.
To add an argument clause:
a
To an argument clause row to the list, click Add.
b
Define the argument clause:
In Name cell, enter the name of the column.
In the Type cell, choose the type of the argument from the drop-down menu.
In the Optional cell, check the check box if the argument clause is optional.
In the List cell, check the check box to provide a list of comma-separated acceptable
values for the argument, and enter them in the adjacent cell.
c
Click Next.
An introduction screen appears describing the either single- or multiple-input functions.
In this example, Introduction Page screen describes what a single-input function is.
5
In the Introduction Page screen click Next.
6
In the Input Schema screen, create an input schema, as described in Creating an Input
Schema.
Creating an Input Schema
Depending on your interface selection, you may need to create either a single-input schema or
a multiple-input one.
•
“Creating a Single-Input Function Schema” on page 21
•
“Creating a Multiple-Input Function” on page 23
Creating a Single-Input Function Schema
To create a single-input function schema:
Aster Development Environment User Guide
21
Using Aster Development Environment
Creating a SQL-MapReduce Function
1
In the Input Schema for single-input function screen, specify the types of columns your
single input row function takes as input. Do one of the following:
•
To accept all columns, click Accept all input columns (you can add restrictions in your
SQL-MapReduce Java code later).
•
To specify certain columns only, choose Accept columns with exactly these types, click Add,
and choose a column type from the drop-down menu. In this example, choose integer.
To ensure that the your chosen column types match those of a particular table or view that
you plan to use as input to the function, click Query Aster Database. This button connects to
the database and opens a window that lets you choose the table column whose type you
want to add to the list.
22
2
Click Next.
3
Proceed to “Creating Output Columns” on page 23.
Aster Development Environment User Guide
Using Aster Development Environment
Creating a SQL-MapReduce Function
Creating a Multiple-Input Function
To create a multiple-input function schema:
1
In the Input Schema for multiple-input function window, specify the number of inputs your
function accepts:
•
To specify any number of inputs, click Any number of Inputs.
•
To specify any number of inputs, click A range of Inputs, and enter the minimum and
maximum (optional) number of inputs.
•
To specify an exact number of inputs, click Exactly ____ Inputs, and enter the number of
inputs. Also, click Add, and specify the input name and input kind your function
accepts.
There are rules that govern the number and kinds of input allowed. See the chapter on
SQL-MR in the Aster Database User Guide for more information on these rules and the
different input kinds.
2
Click Next.
3
Proceed to “Creating Output Columns” on page 23.
Creating Output Columns
To define the output column names and types:
1
In the Output Columns screens, choose the output column names and types:
•
To return all the input columns, choose Exactly the input column names and types.
Aster Development Environment User Guide
23
Using Aster Development Environment
Creating a SQL-MapReduce Function
•
To return all the input columns in addition to the columns listed in the list, choose The
input columns with the below columns added. Click Add to add the additional columns.
•
To return only the columns specified in the list, choose Exactly these names and types. If
you choose this option, the output columns need not match the names or types of the
input columns. Click Add to add the output columns.
In this example, choose Exactly these names and types, click Add, set the Name cell to
cuberootresult, and choose numeric from the Type drop-down menu.
2
Click Finish.
ADE generates stub code for the function.
3
In Eclipse, choose File > Save.
Adding SQL-MapReduce Function Implementation
In the Java stub code that ADE generates, the constructor specifies mapping between input
and output columns of the function.
In most cases you can use the constructor as-is, however, you need to add your
implementation of the SQL-MapReduce function to one of these important methods:
•
operateOnSomeRows()—The row SQL-MapReduce function.
•
operateOnPartition()—The partition SQL-MapReduce function.
•
operateOnMultipleInputs()—The multiple-input SQL-MapReduce function.
In this example, you implement a row function.
To add your implementation to one of these functions:
1
24
(Optional) Before adding your implementation, create a TestRunner test file in Eclipse that
lets you test your code locally (no Aster Database needed) as you work on it. For
instructions on how to create the test file, see “Testing SQL-MapReduce Functions
Aster Development Environment User Guide
Using Aster Development Environment
Creating a SQL-MapReduce Function
Locally” on page 26.
2
In Eclipse, open the Java class generated by ADE.
In this example, the class is called CubeRoot.java.
3
In the SQL-MapReduce function, in this example operateOnSomeRows(), add your
implementation to the while(inputIterator.advanceToNextRow()) loop:
a
Delete the exception code:
throw new ClientVisibleException(
“operateOnSomeRows() needs to be implemented!”);
These lines serve as a hint that the implementation has not yet started.
b
In the place of deleted exception code, provide your SQL-MapReduce function
implementation.
For example, this code implements a cube-root row function:
public void operateOnSomeRows(RowIterator inputIterator,RowEmitter
outputEmitter) {
//
Aster Development Environment User Guide
25
Using Aster Development Environment
Testing SQL-MapReduce Functions Locally
// This method will be called once for each set of rows.
// while (inputIterator.advanceToNextRow()) {
//
// Construct output rows here using calls like
outputEmitter.addInt(),
// outputEmitter.addString(), or outputEmitter.addFromRow().
// Once finished building the row, call outputEmitter.emitRow().
//
int value = inputIterator.getIntAt(0);
double root = Math.pow(value, 1.0/3);
outputEmitter.addDouble(root);
outputEmitter.emitRow();
}
}
4
To test your code locally, go to “Testing SQL-MapReduce Functions Locally” on page 26.
5
To create a SQL query that invokes your function, go to “Creating Queries to Invoke SQLMapReduce Functions” on page 31.
For more information about implementing row and partition functions, see the
“SQL-MapReduce Java API” section in the Aster Database User Guide.
Testing SQL-MapReduce Functions Locally
ADE includes a feature called TestRunner that lets you create a test file in Eclipse to test your
code locally, without an Aster Database connection.
To test your SQL-MapReduce function, perform these steps:
•
“Creating a Test” on page 26
•
“Creating Input Data” on page 28
•
“Running the Test” on page 29
Creating a Test
To create your test in TestRunner:
1
Choose File > New > Aster SQL-MapReduce Test Configuration.
The New Test Configuration wizard appears.
26
Aster Development Environment User Guide
Using Aster Development Environment
Testing SQL-MapReduce Functions Locally
2
In the Aster SQL-MapReduce Test Configuration screen of the wizard, configure the test
settings. For this example, keep the default values.
This is a description of the fields in this screen:
•
Folder—The folder where the test file is saved.
•
Filename—The name of the test file containing the test data and settings.
•
Function name—The name of the Java class that implements your SQL-MapReduce
function (in this example, CubeRoot).
•
Which type of SQL-MR function will the SQL invoke?—The type of SQL-MR function
you are testing, either Single Input Function or Multiple Input Function.
•
Argument Classes—If your function takes arguments, click Add and add a name/value
pair for each argument your test passes to your SQL-MR function.
3
Click Next.
4
In the Input screen, define the source and schema of the input data the test uses.
a
Click Use File.
Aster Development Environment User Guide
27
Using Aster Development Environment
Testing SQL-MapReduce Functions Locally
5
b
Click comma-separated values (CSV).
c
To add an input, click Add and specify the input column name and type. In this
example, add an integer input.
d
Enter or browse to the Input File to specify the input file to be used in the test.
NOTE: If an input file “*.in” exists before running the TestRunner configuration UI
wizard, the ADE will not generate a new input file. Instead, the ADE will use the
existing “*.in” input file. Rename your files, if necessary, before running the TestRunner.
e
Click Next.
In the Define Partitioning and Ordering screen, define how the input data should be
partitioned and reordered. Then, click Next.
For this example, no need to make any changes to this screen.
6
In the Output screen, specify the format and destination of the test output.
You can direct the output to a file or to the Eclipse console. For this example, send the
results to the console by clicking Use the console.
7
Click Finish.
ADE generates the following:
•
Test file (its name ends with .test)
•
If specified, the output data file (its name ends with .out)
•
If specified, the input data file (its name ends with .in)
These files appear in the Eclipse Package panel.
8
Proceed to “Creating Input Data” on page 28.
Creating Input Data
If your test uses an input data file, edit it in Eclipse:
1
In the Package panel, select the input data file.
For this example, select CubeRoot.in.
28
Aster Development Environment User Guide
Using Aster Development Environment
Testing SQL-MapReduce Functions Locally
2
To open the file for editing, double-click it.
3
Enter the test input values.
4
Click Save.
5
Proceed to “Running the Test” on page 29.
Running the Test
To test your SQL-MapReduce function, run the test you created in Eclipse:
1
Double-click the test in the Package panel.
In this example, double-click CubeRoot.test.
2
Choose Run > Run As > Locally in TestRunner.
3
If prompted to save sources, click OK.
Aster Development Environment User Guide
29
Using Aster Development Environment
Testing SQL-MapReduce Functions Locally
The results of the test appear in the Console panel.
4
When testing your function locally with a “.test” file, you can use the debugging tools in
Eclipse.
For example:
a
Right-click in the margin of the Java class that implements your SQL-MR function and
choose Toggle Breakpoint to set a breakpoint.
b
Debug the test file by choosing the test file from the Debug drop-down menu in the
tool bar.
Running with breakpoints launches the Debug perspective in Eclipse, where you can
step through your code to pinpoint errors.
30
Aster Development Environment User Guide
Using Aster Development Environment
Creating Queries to Invoke SQL-MapReduce Functions
Creating Queries to Invoke SQL-MapReduce
Functions
You can invoke SQL-MapReduce functions in SQL SELECT queries. ADE helps you write the
basic query that invokes your function. This section describes how you can create a sample
query that invokes a sample SQL-MapReduce function.
•
“Creating a Sample Table” on page 31
•
“Creating a Query” on page 32
For more information about invoking SQL-MapReduce functions in SQL SELECT queries,
see the “SQL-MapReduce Query Syntax” section of the Aster Database User Guide.
Creating a Sample Table
For testing purposes, it is convenient to create a table with sample input data for the query.
To create a sample table in Eclipse:
1
Select Aster SQL-MapReduce perspective.
2
Choose File > New > Aster SQL File.
Aster Development Environment User Guide
31
Using Aster Development Environment
Creating Queries to Invoke SQL-MapReduce Functions
3
Click Browse and select the folder where the query is stored.
4
In the New SQL File window, in the File name field, enter example_table.sql.
5
Click Finish.
The new SQL file appears in the Package panel.
6
Delete the provided code and replace it with this code, which creates a table and adds data
to it:
-- Set up example table
CREATE FACT TABLE to_root (
INPUT int, PARTITION KEY (input)
);
-- Place test input data in table
INSERT INTO to_root VALUES
(10000), (999), (12), (3), (2432), (1000), (8);
7
Click Save.
Creating a Query
To create a query in Eclipse:
32
1
Select File > New > Aster SQL File to generate the stub SQL query.
2
Click Browse and select the folder where the query is stored.
Aster Development Environment User Guide
Using Aster Development Environment
Creating Queries to Invoke SQL-MapReduce Functions
3
In the File name field, enter get_cube_root.sql.
4
Click Finish.
The get_cube_root.sql file appears in the Package panel.
5
6
Edit the query to match your function:
a
In the SELECT clause, specify the table columns to be retrieved, or * for all columns.
b
Replace functionname with a name for your function.
c
Replace the table-or-query for each ON clause with the name of your input table.
d
If this function contains partitioned inputs, edit the PARTITION BY clause to partition
the input data. If needed, edit the ORDER BY clause to sort the rows within the
partitions.
e
If this function is a multiple-input function that contains an input that uses data as it is
already stored, use PARTITION BY ANY.
f
If this function is a multiple -input function that contains dimensional inputs, use the
DIMENSION clause to replicate the input data.
g
If your SQL-MapReduce function takes arguments, replace clausename (arg) with
the name of your first name/value argument pair, and add more argument pairs as
needed, separated by commas.
For this example, replace the code provided by ADE with the following:
-- Example query using cuberoot function
SELECT *
FROM cuberoot(
ON to_root
);
7
Click Save.
8
Go to “Compiling, Deploying, and Running” on page 34.
Aster Development Environment User Guide
33
Using Aster Development Environment
Compiling, Deploying, and Running
Compiling, Deploying, and Running
After writing an SQL-MR function and an SQL query to invoke the function, compile and
deploy the files on Aster Database.
To compile and deploy the files on Aster Database:
1
In the Eclipse, in the Package panel, select the SQL file that creates the sample input table.
2
Right-click the file name and choose Run As > Target Aster Database.
3
If prompted to save resources, click OK.
ADE automatically builds and deploys the SQL-MapReduce functions in the current
project, then executes the SQL file on Aster Database.
The output appears in the Console panel.
4
In the Package panel, select the SQL query file that calls your SQL-MapReduce function.
5
Right-click the file and choose Run As: Target Aster Database.
ADE automatically builds and deploys the SQL-MapReduce functions in the current
project, then executes the SQL file on Aster Database.
The output appears in the Console panel.
If the query fails, edit the SQL or your Java code in Eclipse, save it, and invoke the Run As
command again. Each time you do this, your SQL-MapReduce functions are uploaded to
Aster Database, replacing the old versions there.
After you have run the function in this manner, it remains available in the database where you
installed it. Within a database, SQL-MapReduce functions reside in the public schema by
default, so any user of the database can make use of it. If you also need it in other databases,
you must deploy it separately to each database.
34
Aster Development Environment User Guide
Using Aster Development Environment
Managing Functions and Files on Aster Database Clusters
Managing Functions and Files on Aster
Database Clusters
You can use ADE to manage the functions and files deployed on Aster Database clusters that
are accessible from your ADE project. This is useful for all sorts of administrative purposes.
For example, you can make executable files available for use on other clusters, see what has
been installed on a given cluster, uninstall files from clusters, and move files from one cluster
to another.
•
“Exporting SQL-MapReduce Functions” on page 35
•
“Viewing Aster Database Cluster Information” on page 37
•
“Uploading Functions and Files to Aster Database” on page 38
•
“Downloading, Removing, or Moving Files on Aster Database” on page 39
Exporting SQL-MapReduce Functions
Once you have built your SQL-MapReduce function (as described in “Compiling, Deploying,
and Running” on page 34) you can save the compiled SQL-MapReduce function as a jar that
you can deploy on any Aster Database cluster.
To export SQL-MapReduce Functions:
1
In the Eclipse, in the Package panel, expand your project and Deployment nodes.
Your compiled project appears under Deployment. The file ends with .zip.
2
Right-click the zip file and choose Export.
3
Select Java > JAR file and click Next.
4
Click Browse, select the export destination, specify the name of the JAR file, and click Save.
5
Click Finish.
Aster Development Environment User Guide
35
Using Aster Development Environment
Managing Functions and Files on Aster Database Clusters
Making SQL-MapReduce Functions Available to Any Aster Database Cluster
To make the JAR file containing SQL-MR functions available for uploading to any Aster
Database cluster:
36
1
Import the file into your ADE project in Eclipse by choosing File > Import.
2
In the Select screen of the Import wizard, expand the General node and double-click File
System. Click Next.
3
Click Browse for the From directory field, select the folder containing the file to import, and
click OK.
4
Check the check box of the files to import.
5
Click Browse for the Into folder field, then select your project.
Aster Development Environment User Guide
Using Aster Development Environment
Managing Functions and Files on Aster Database Clusters
The file appears in the Package panel.
To install the JAR file on an Aster Database cluster, see “Uploading Functions and Files to
Aster Database”.
Viewing Aster Database Cluster Information
To see installed files and other information about the Aster Database clusters that are
configured as part of your ADE project:
1
In Eclipse, in the Package panel, click the name of your project, then choose Project >
Properties.
2
In the Properties window, expand the Aster Development Environment node, select Aster
Database, then click the Configure Aster Database link.
3
In the Preferences (Filtered) window, select an Aster Database, then click Info.
Aster Development Environment User Guide
37
Using Aster Development Environment
Managing Functions and Files on Aster Database Clusters
The Aster Database Information window appears. This window displays the files that are
currently installed on the cluster.
Uploading Functions and Files to Aster Database
You can use ADE to install SQL-MapReduce functions and other files onto your Aster
Database clusters. You can install:
•
SQL-MapReduce functions—Compiled Java and C executables that can be called by name
in the FROM clause of a SELECT statement.
•
Files—Installed files are typically used to provide settings to SQL-MapReduce functions.
Installed files can only be used in your SQL-MapReduce functions. Installed files cannot
be directly invoked or accessed via SQL.
To install a files or a function:
38
1
Display the Aster Database Information window as described in “Viewing Aster Database
Cluster Information”.
2
Click Upload File.
3
In the Select file window, in the Select a resource to open field, type part of the filename or a
regular expression. Matching files are displayed below.
Aster Development Environment User Guide
Using Aster Development Environment
Managing Functions and Files on Aster Database Clusters
4
Select the file you want to upload, and click OK.
Downloading, Removing, or Moving Files on Aster Database
Perhaps you only need to install files on one Aster Database. In that case, the procedure in the
previous section, “Uploading Functions and Files to Aster Database”, might be all you need.
However, if you have several clusters, you might need to move files around from one to
another. And it is likely that you might need to uninstall a file from a cluster at some point.
You can download, remove, and move files through the Aster Database Information window,
as described in this section.
•
“Downloading a File or Function” on page 39
•
“Removing a File or Function” on page 40
•
“Moving a File or Function From One Cluster to Another” on page 40
Downloading a File or Function
To download a file or function from a cluster:
1
Display the Aster Database Information window as described in Viewing Aster Database
Cluster Information (page 37).
2
Choose a schema and file, then click Download File.
3
Select the download folder.
The file will be added to your current ADE project.
Aster Development Environment User Guide
39
Using Aster Development Environment
Managing Database Connections
Removing a File or Function
To remove a file or function from a cluster:
1
Display the Aster Database Information window as described in Viewing Aster Database
Cluster Information (page 37).
2
Choose a schema and file, then click Remove File.
3
Click OK.
ADE removes the file from the cluster. It might still exist in one or more other locations,
such as on another cluster or in your current ADE project.
Moving a File or Function From One Cluster to Another
To move a file or function from one cluster to another:
1
Display the Aster Database Information window as described in Viewing Aster Database
Cluster Information (page 37).
2
Choose a schema and the file to move, then click Download File.
The file is added to your current ADE project.
3
Click OK to dismiss the Aster Database Information window.
The Preferences (Filtered) window appears again.
4
Select the cluster to which you want to move the file, then click Info.
5
Upload the file to its new home, as described in “Uploading Functions and Files to Aster
Database” on page 38.
Managing Database Connections
To edit an existing database connection:
1
Open your ADE project in Eclipse.
2
Click on your project’s name in the Package panel.
3
Choose Project > Properties.
4
Click Aster Development Environment > Aster Database.
5
Click the Configure Aster Database link.
The Preferences (Filtered) window appears.
40
6
Select the Aster Database you want to work with, and click Edit.
7
In the Update Aster Database Connection window, edit the settings.
8
Click OK.
Aster Development Environment User Guide
Index
A
add new nCluster connection 18
C
connection
edit db connection in ADE project 40
new db connection in ADE project 18
D
database
edit connections in ADE projects 40
new connections in ADE projects 18
E
Eclipse prerequisite before ADE Installation, 7
edit nCluster connection 40
G
GDK environment variable 13
get latest version 12
L
launching ADE
on Ubuntu 9.10 13
S
Support UTF-8 character set 8
T
troubleshooting
launching ADE on Ubuntu 9.10 13
U
Ubuntu 9.10
launching ADE on 13
updating the ADE plugin 12
V
version 3.5.1 or higher of Eclipse IDE for Java Developers 7
Aster Development Environment User Guide
41
42
Aster Development Environment User Guide