Digipede Technologies
Many Legs Make Light Work™
High Throughput Computing Week
Introduction to the Digipede Network™
Agenda
• Introduction to the Digipede Network
• System Overview
• Components
• Concepts in the Digipede Network
• Job templates, jobs, and tasks
• FileDefs, parameters, and other settings
• Using the Digipede Network with no programming
• Designing jobs using wizards in Digipede Workbench
• Using designers
http://www.digipede.net
What is the Digipede Network?
A Windows-based:
• grid computing solution;
• middle-tier, distributed computing solution;
• tool to improve application performance, scalability,
through-put, and reliability.
The Digipede Network is a set of tools that make it
easy to distribute an application’s execution onto a
network.
http://www.digipede.net
Digipede Network
Installation overview
`
`
`
eb
W
S
v
er
D
as
at
s
et
s
Re
e
ic
ts
ul
s
Digipede
Agents
Digipede
Workbench
e
Master
W
Application(s)
b
r
Se
ce
vi
s
Digipede
Server
Excel
User
Experience
http://www.digipede.net
External
Resources
Digipede Network
Components
• Infrastructure
• Digipede Server
• Digipede Agent
• Administration
• Digipede Control
• Job Submission
• Digipede Workbench
• Digipede Framework SDK
• PowerShell or JobSubmit.exe
http://www.digipede.net
Digipede Network
Installation: Overview
Infrastructure
• The Digipede Server
• A Digipede Agent on each compute node
Job creation/submission tools
• Digipede Workbench
• Digipede Framework SDK
http://www.digipede.net
Concepts to Know
Job Templates, Jobs, and Tasks
• Job Templates
• Defines how to work on a class of jobs
• Describes files to move and cache
• How to invoke the job
• Jobs
• Run against a job template
• May contain data or files that apply just to that job
• Task
• Part of a job
• Executed on one machine – atomic unit of work
• May contain data or files that apply just to that task
Basic Concepts
• Job
Template
• Job
• Task
• Parameter
• File
Definition
• Job Defaults
Digipede Network
File Definitions
• File Definitions describe the files that will be
moved as part of a job or task
• Relevance
•
•
•
•
JobTemplate
JobPlaceholder -> Input
InputPlaceholder -> Input
ResultPlaceholder -> Result
• File Transfer
• Streamed (Digipede API apps only)
• Hosted
• Location
• Transfer Types
Digipede Network
Parameters
• Parameters are name-value pairs that can be
used to pass information with a job template, job,
or task
• Relevance
• JobTemplate
• JobPlaceholder - Input
• InputPlaceholder - Input
• Substitution Variables
• $(tmpfile) - A unique temporary file name
• $(JobID) – The Job ID of the Job
• $(string) – The variable is replaced by the value of the
parameter or file name of the file indicated by string.
Digipede Network
Job Defaults
Minimum CPU
Minimum RAM
Reassignment
Interval
Maximum
Assignments
Maximum
Failures
Digipede Network
Job Template
• File Definition
• Job Defaults
• Application Control
• Command line
• .NET
• COM
• Version
Digipede Network
Job
• File Definitions
• Parameter
• Settings
• Collection of tasks
Digipede Network
Task
• File Definitions
• Result Files
• Parameters
• Command line (can be combination of hardcoded
text, parameters, and file names)
Digipede Network
Digipede Workbench: Wizards
Job Template
Wizard
Digipede Network
Digipede Workbench: Wizards
Job
Wizard
Digipede Network
Digipede Workbench: Designers
Job Template
Designer
Digipede Network
Digipede Workbench: Job Tracking
Digipede Network
Digipede Control: Monitoring Pages
Job
Page
Hello, World!
• Stage the files
• Manually deploy to each node
• Or put in an accessible (via copy or HTTP) location
• Create a Job Template (and Job) in Digipede
Workbench
• Define “common file” (HelloWorld.exe)
• Create tasks (use a “Range” parameter)
• Tell it the command line
• In advanced options, make sure you select “Save Standard
Output”
http://www.digipede.net
Install and Start Workbench
1. Open a browser and navigate to
\\HTC-Server\DigipedeControl, and log in. Use your
PC name as a login and password (e.g., LABPC04).
Then, click on “Digipede Workbench” and follow
installation instructions.
During the install, the server name is HTC-Server
and the login/password is your PC name.
2. Start Workbench (Start Menu -> Programs ->
Digipede -> Workbench)
http://www.digipede.net
The Job Wizard
3. Start Job Wizard
5. Name your job
4. Existing? No
6. Move files via Share
http://www.digipede.net
Define Files and Parameters
7. One common file
10.One parameter
8. Browse and select
11.Define range parameter
9. No Job or Task files
http://www.digipede.net
Finish and Submit
12.No result files or
Digipede API
13.Define command line
16.Select Advanced
Options, Save Standard
Output
14.Add TaskID to the
command line
HelloWorld.exe $(TaskID)
17.Finish and run!
15.No notifications
http://www.digipede.net