PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
FasTrak Issue “Bulk Insert” Method – PR50007
Design Document
Team: Transaction Services
106740250
Page 1 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Table of Contents
1
Document Scope
4
2
Requirements Addressed
4
3
2.1
Assumptions
2.1.1 API Interface for Web Server HTTP Requests
2.1.2 File System Access
2.1.3 Naming Conventions
2.1.4 TeamScript Performance
4
4
4
5
5
2.2
Submission of Issues
2.2.1 Issue Creation Using the Bulk Insert Method
2.2.2 Process Flow Diagram
2.2.3 File Attachment Details
5
5
10
11
2.3
Issue Data Source
2.3.1 Issue Data Format
2.3.2 Issue Data Limits
2.3.3 Issue Data Storage in FasTrak
11
11
11
11
2.4
TeamScripting
2.4.1 Bulk Data Validation TeamScript
2.4.2 Bulk Data Validation TeamScript Pseudocode
2.4.3 Bulk Insert Process TeamScript
2.4.4 Bulk Insert Process TeamScript Pseudocode
2.4.5 Error Handling
12
12
12
13
14
15
2.5
16
Results Status Report
Alternatives
16
3.1
Emulation of Web Server HTTP Requests
3.1.1 Assumptions
3.1.2 Benefits
3.1.3 Drawbacks
16
17
17
17
3.2
Use of FasTrak API for Issue Create
3.2.1 Assumptions
3.2.2 Benefits
3.2.3 Drawbacks
17
17
17
17
3.3
Stand-alone Custom Executable
3.3.1 Assumptions
3.3.2 Benefits
3.3.3 Drawbacks
18
18
18
18
Team: Transaction Services
106740250
Page 2 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
3.4
TeamScript Only
3.4.1 Assumptions
3.4.2 Benefits
3.4.3 Drawbacks
18
18
18
19
4
Data Model Changes
19
5
Documentation Updates
19
6
Testing
19
Appendix A – Issue Data Fields
Team: Transaction Services
106740250
20
Page 3 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
1 DOCUMENT SCOPE
This document is a detailed description of the resolution for the browser-driven,
automated creation of multiple FasTrak Issues, using a method to be known as a “Bulk
Insert.” These Issues will be created as a group using data provided by a user, who will
then trigger the process with a simple browser-action in the new FasTrak system.
2
REQUIREMENTS ADDRESSED
Based on original specifications found in the “PR-50007 FasTrak Enhancements”
document, the “Bulk Insert” method will work in tandem with other methods of Issue
creation. Bulk Insert is to provide the ability for a user to have multiple D2D- or DEVor LPA-type (“Load Profile Assignment”) Issues be created in the FasTrak system from a
web browser with just a few button clicks.
After Bulk Insert, there will be two other methods of Issue creation in the FasTrak
system. The first involves the manual creation of one Issue at a time from a web browser
(“GUI”). The second allows the creation, or “upload,” and maintenance of Issue data
through an external, API-driven web interface. As the latter describes a quite similar
objective and process, it would be to the advantage of Bulk Insert to share in that
functionality. The recommended design which follows describes a method of this kind.
2.1 Assumptions
The Bulk Insert method detailed here will have dependencies on code packages being
developed external to the method itself. The development time of and protocols specified
by these packages could impact the development and delivery of the Bulk Insert package.
2.1.1 API Interface for Web Server HTTP Requests
The API Interface for Web Server HTTP Requests is an externally developed code
package that will allow users, who are in environments outside of FasTrak, access to
FasTrak Issue data. This access is to include the submission of one to many new Issues
and checking the status of existing Issues. Bulk Insert will use Windows file I/O in a predefined target directory to interact with a “piggy-backed” addition to this API Interface.
2.1.2 File System Access
There will be a Windows User account that runs FasTrak TeamScripts, which will be the
scripted programs that define the Bulk Insert process outside of the API Interface. This
account must be provided with file-read and file-write access to the pre-defined target
directory.
If the API Interface runs under a Windows User account, this account must be provided
with file-read access to the pre-defined target directory.
Team: Transaction Services
106740250
Page 4 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
2.1.3 Naming Conventions
The Bulk Insert method and associated API Interface must agree on a naming convention
for drop-files and summary reports, as their interaction will be limited to a Windows file
exchange. The bulk file naming format will be “Submit_IssueID#_DUNS#_Filename”
(e.g. “Submit_00028_1234567890123_TestCWA.csv”). The “Filename” portion is
actually the internal FasTrak attachment name, which is derived from the filename
provided by the user. As the “IssueID#” will be unique within FasTrak, and the user can
only successfully submit a given Bulk Insert request once, the file will be guaranteed
unique. This format will readily indicate to which Bulk Insert issue a file is attached.
When dropping a results file into the MIR Drop-off directory, the filename must conform
to the MIR system, which is where Bulk Insert results reports will be stored. The results
file naming format will be “pfx.report_type.DUNS#.Submit_IssueID#_Filename” (e.g.
“ext.00000345.1234567890123.Submit_00028_TestCWA.csv”). The MIR system will
add a timestamp to the filename when loading the report into the database.
When a market participant views a results report through the Report Explorer GUI and
MIR, the report filename will not include the report-type or the DUNS, but it will include
the MIR timestamp. Using Report Explorer, the file naming format is “pfx.(mm-dd-yyyy
HH:MM:ss).Filename” (e.g. a results report file for the bulk file example above might be
“ext.(03-30-2006 14:15:22).Submit_00028_TestCWA.csv”).
2.1.4 TeamScript Performance
The design of this method relies on the use of FasTrak TeamScripts to handle reasonably
large “batch” files of Issue data, which presents a pair of unknown factors that can only
be determined through implementation. One is the total execution time of a TeamScript
when processing a maximum-length batch file, with the other being the simple ability of
a TeamScript to handle a file of that size.
2.2 Submission of Issues
The submission of an Issue or group of Issues to the FasTrak system will be done using a
few, very different methods. These will consist of two manual, browser-side processes GUI and Bulk Insert - and an external, API-driven web interface. Though the data
sources for these methods differ, they will all use the same submit-path in the FasTrak
system. This will provide common data validation and Issue verification processing.
Regardless of how it was created, every Issue will have a FasTrak status field to signify
which method (“source”) was used to create it. This will be done with a tri-state FasTrak
data field that has a distinct value for each of the three submission methods, with this
value being defaulted by the system per the method used.
2.2.1 Issue Creation Using the Bulk Insert Method
The Bulk Insert method will allow the user, through a web browser interface, to upload a
text file that contains the data for a batch of Issues to be created in FasTrak. This data
Team: Transaction Services
106740250
Page 5 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
will use the ERCOT-defined field ordering for the Issue type that is specified by the user.
It should be noted that this method is very dependent on the positional ordering of
fields in the text file. Care must be taken to ensure not only the correct number of
fields for an Issue Type but that they are also in the correct order. If a specific SubType doesn’t use a field then it should be left blank, though it still has a position in the
comma-delimited file. This would be done by having nothing preceding the comma for
that field (e.g. two in a row “,,” or nothing before the first or nothing after the last).
A Bulk Insert will be triggered during a FasTrak workflow “transition.” A transition is a
browser-state where the user is allowed to add to and modify data in the FasTrak system.
It is also how a FasTrak record will be navigated through an ERCOT workflow. The
ability to do this or any other transition, or to even “see” the Bulk Insert or any other
FasTrak process from the browser, is a FasTrak privilege that can be controlled on a per
user and/or per user group basis. Generally speaking, in a transition the user will have
the option of 1) fulfilling any transition-based requirements to complete the transition or
2) canceling out of the transition as if it had never been started.
To begin a Bulk Insert, the user must first create a record in FasTrak that will provide a
location or “container” for the Issue data file upload and access to the transition which
will trigger the Bulk Insert process.
Ex: A Bulk Insert is started by creating (“Submit”) a container record for that project (“Bulk Insert”).
Clicking these browser-links will create a Bulk Insert record, bringing the user to its
initial “Active” page - a FasTrak “state,” where a record isn’t moving in a workflow, isn’t
“changing state.” Generally speaking, whenever a FasTrak record is in any particular
state, the user can browse-away from that record, returning to it at any time later.
The “Active” page will have a transition button to upload an Issue data file and one that
will to start the Bulk Insert process, though FasTrak will disallow that process until one
and only one data file has been uploaded. Doing an upload will “attach” an uploaded
copy of a file to the Bulk Insert record; the original copy of the file on the user’s PC will
not be disturbed.
Team: Transaction Services
106740250
Page 6 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Ex: Initial “Active” state of a Bulk Insert record; note button to attach an Issue data file.
Clicking the button to “Attach/Update…” a file will bring the user to a transition “waitpage” where user-input will be required: the Issue type, the Issue sub type, and a file
attachment, the latter being an option in the “Actions:” dropdown.
Ex: “Add File” option of “Actions” dropdown in “Attach/Update Bulk File” transition.
Note red-italicized “File” to alert the user that a file must be attached.
The “Add File” option will launch a pop-up where the file to be attached is selected. The
user will provide a name for the file, used by FasTrak as a browser-side hyperlink to the
Team: Transaction Services
106740250
Page 7 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
attached file, and the full path of the file to be attached. The path can be typed in or the
user can click “Browse…” to select a file with the Windows file-locator dialog box.
Ex: File selection pop-up, where “Name” is for a GUI hyperlink to the attached file and “Path” specifies the file; before
(left) and after a file is successfully uploaded. The “Browse” button brings up a file-locator dialog box.
Clicking the GUI button “Upload & Attach File” will complete the file selection, starting
the file upload.
If the upload fails for any reason, the pop-up will remain, displaying a failure message.
The user will then have the option of 1) updating any pop-up selections and trying again
or 2) canceling out of the pop-up. If the latter is chosen, it will be likely that the user will
then need to cancel from the wait-page of the “Attach/Update…” transition.
If the upload is successful, the pop-up will display a success message and a button to
close the pop-up; the user can also click a checkbox to have the pop-up close
automatically if successful.
Ex: “Rows File” attachment, with “Issue Type” & “Sub-Type” selected. Note controls,
left to right, to “Edit” or “Delete” the attachment or email the user who attached the file.
Team: Transaction Services
106740250
Page 8 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
The user will then return to the “Attach/Update…” wait-page, where the added file now
appears as a link under “Attachments.” The file will remain attached, regardless of
whether the user then clicks the “OK” (to proceed) or “Cancel” (the transition) button; if
a user has the privileges, an attachment can be edited or deleted any time the controls for
that are showing.
Clicking “OK” will complete the transition with FasTrak then running an initial data
validation of the attachment, recording the data for later use when the Bulk Insert process
is selected. If there is no attachment, FasTrak will disallow the “OK” click, displaying a
message informing the user of the deficit; the user may then do a file attachment or click
“Cancel.” Note that a “Cancel” button can be clicked anytime one is available.
If there is an attachment, it has now been validated and pertinent data regarding the
attachment will be on-screen for the user to review. If this data is not what was expected,
the user will have several options for remedying the discrepancy: check that the selected
Issue Type and Sub-Type are correct for the attachment; check that the attached file is the
one expected; if the user has the privilege, edit the content of the attachment. After any
change is made, the “Attach/Update...” transition can be clicked once again to see if the
pertinent data becomes more as expected. If the Number of Rows displayed is not the
count the user expected, the user can view the attachment to see the invalid rows (which
will be prefixed with ”*ERR*”).
Having clicked either “OK” or “Cancel,” FasTrak will return the user to the Bulk Insert
“Active” page, where again will be the transition buttons to “Attach/Update…” an
attachment file and “Submit Bulk File” to start the Bulk Insert process. Selecting
“Attach/Update…” will allow an alternate file to be attached, or even for an attachment
to be deleted, with the user then electing to “Cancel” the transition. It should be iterated
that the “Attach/Update…” transition will not complete successfully unless there is one
and only one Issue data file attached; for more on adding and deleting attachments, see
File Attachment Details.
Ex: “Submit Bulk File” (left) to start Bulk Insert process; note 2 attachments which
would disallow this. “Archived” record (right) after process is selected.
Team: Transaction Services
106740250
Page 9 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Selecting “Submit Bulk File” will have FasTrak attempt to do the Bulk Insert process.
This process will first check for anything that might invalidate it: there isn’t one and only
one file attached, the attachment doesn’t match the data that “Attach/Update…” recorded;
see Bulk Insert Process for details.
If any errors or problems are found, the Bulk Insert process will halt the “Submit Bulk
File” transition, displaying text on the browser page detailing what was discovered. As
with any FasTrak transition, the wait-page will have both the “OK” (to proceed) and
“Cancel” (the transition) buttons. Attempting to proceed would be fruitless, as this would
just restart the Bulk Insert process, which would again report the same errors or
problems. The user will have to click “Cancel” to return to the “Active” page and
remedy what the process had reported.
If no errors or problems are found, the Bulk Insert record will automatically proceed to
the “Archived” state, becoming an inactive FasTrak record. The browser-side of this
process has been completed; there will be no other transitions to run. The API Interface
will have taken-over the bulk Issue create, processing the file produced by the Bulk Insert
Process and doing a submission to FasTrak for each valid Issue that is found. The file
attachment of the Bulk Insert record could be deleted, this is in order to save file storage
space on the web server, but it might also be kept for later reference.
2.2.2 Process Flow Diagram
Create Bulk Insert
FasTrak Record
SCREEN:
Select File to
Attach
Attach File to Bulk
Insert Record
API
Target
Directory
Extract Attached File to
API Target Directory
Read File from
API Target
Directory
FasTrak Bulk Insert
Complete
API Interface
to
Tibco Magic
Tibco Results
MIR Report
Team: Transaction Services
106740250
Page 10 of 23
FasTrak Issues
Created
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
2.2.3 File Attachment Details
The following are file attachment details of the FasTrak system.
 The FasTrak system cannot inherently restrict the number of files that may be
attached to any particular record; if one can be attached, so can 2 or more. It’s
none or many, depending on how a FasTrak project is configured.
 A file attachment can be added anytime that a user can see the “Add File” option
of the “Actions:” dropdown menu in the browser. For example, a file attachment
could be added to a record from any FasTrak “transition,” when a record is
moving in its workflow, or any FasTrak “state,” when a record is idle in its
workflow. With reference to the Issue Creation section for details, this means that
a file attachment could be added while in the “Active” state or even the
“Archived” state.
 Based on a user’s FasTrak privileges, a file attachment can be deleted anytime
from any record that the user can browse to.
When considering these points, both individually and collectively, there are many criteria
that a Bulk Insert record and its file attachment must satisfy before the Bulk Insert
process can be allowed to proceed. Please see the TeamScripting section and its subsections for details on how all of this is handled.
2.3 Issue Data Source
The first fundamental component of the Bulk Insert method is the data source to be used.
2.3.1 Issue Data Format
The data format that will be expected by this method will be a human-readable text file,
which will contain a text-row list of data for Issues of common type and subtype. A textrow will be a set of Comma Separated Value (CSV) data fields, with these fields in an
expected order based on their Issue type - see Appendix A for details. Any text-row that
does not conform to the expected order will not be processed by Bulk Insert, instead
generating an error in the summary report.
The data format is not Double Quote Comma Separated. In other words, any quotes will
be taken literally and commas may not appear in any field.
2.3.2 Issue Data Limits
At this time, the TeamScripts will not impose a limit on the number of issues processed
per bulk insert request.
2.3.3 Issue Data Storage in FasTrak
The FasTrak system will be configured to store attached files in a filesystem accessible
by the FasTrak web server. TeamScripting will need to access this directory to read and
write/update the contents of an attached file. It will be accessed via the worker process in
Team: Transaction Services
106740250
Page 11 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
IIS. Appropriate Windows file permissions will be necessary to enable the script to read
and write to the directory and contained files.
2.4 TeamScripting
TeamScripting will be used to resolve the complications listed in the following sections.
A TeamScript is a program script that is written to be run internally by the FasTrak
system, providing a way for FasTrak to accomplish very complex tasks as well as interact
with other, external systems.
2.4.1 Bulk Data Validation TeamScript
The Bulk Insert method is designed to handle only one batch of Issue data at a time, with
a batch being in the form of a text file that is attached to a FasTrak record in the Bulk
Insert project. The FasTrak system, however, cannot limit a record to one and only one
attached file (see File Attachment Details). And then there is the matter of validating the
data content of the attached file, verifying that the file is even worth sending to the Bulk
Insert process. FasTrak has no built-in procedure for doing this.
The Bulk Data Validation TeamScript will deliver file attachment validation and
preliminary Issue data verification. It will be triggered when a user clicks the browser
“OK” (to proceed) button to complete the “Attach/Update…” transition. At that point,
the presumption will be that the user has attached one and only one Issue data file that
needs to be verified, analyzed, and quantified.
Sequentially, this script will first check that there is only one file attached, disallowing
the “OK” click with an on-screen message if not. Next, the script will check the validity
of the attachment. This will involve retrieving the attachment data and doing a per-line
data field count, comparing these counts against the expected field count for the Issue
Type that the user has specified. It will tally the number of per-line Issues that appear to
be valid, storing this total to a data field of the Bulk Insert FasTrak record. This will
allow the user to check this total against the number of Issues that had been expected.
Any rows that were deemed not valid will be pre-pended with the tag “*ERR*”. This
will allow the user to easily spot the errors if they first wish to correct and upload and
attach the file again before actually submitting.
To finish, pertinent information that will uniquely identify the file attachment will be
gathered and recorded to data fields of the Bulk Insert FasTrak record; these fields will be
read-only to users. This information will include the user who did the file attach, the
exact time the attachment was done, and the Windows filename of the attachment. There
should be no way for all of this data to be duplicated, and it will be used by the Bulk
Insert Process to verify that nothing about the attachment has changed.
2.4.2 Bulk Data Validation TeamScript Pseudocode
The following, while not the actual code to be used, should illustrate the expected flow of
this TeamScript process.
Team: Transaction Services
106740250
Page 12 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
; Script entry, triggered by user browser action
record = GetCurrentRecord()
numAttached = RunDbQuery(record, “how many file attachments?”)
If (numAttached <> 1) Then
DisplayMsg(“Invalid number of file attachments”)
AbortTransition()
ExitScript
; Get data for attachment validity check
issueType = GetDataField(record, “Issue Type”)
issueSubType = GetDataField(record, “Issue Sub-Type”)
fieldCount = GetFieldCount(issueType)
dataAttached = RunDbQuery(record, “get attachment data”)
; Check data field count of Issue data text-rows in attachment
numValid = 0
While (dataAttached Not Empty)
nextLine = GetTextLine(dataAttached)
numFields = GetFieldCount(nextLine)
If (numFields = fieldCount) Then
numValid = numValid + 1
EndWhile
; Store the number of valid Issues found to the Bulk Insert record
SetDataField(record, “issueCount”, numValid)
; Gather other data to identify the attachment
userAttached = RunDbQuery(record, “get attachment user”)
timeAttached = RunDbQuery(record, “get attachment time”)
fileAttached = RunDbQuery(record, “get attachment file”)
; Store this data to the Bulk Insert record
SetDataField(record, “AttachmentUser”, userAttached)
SetDataField(record, “AttachmentTime”, timeAttached)
SetDataField(record, “AttachmentFile”, fileAttached)
ExitScript
2.4.3 Bulk Insert Process TeamScript
Presuming an attached batch file is ready to use, actually doing the Bulk Insert presents a
new set of difficulties. It must be verified that the attached file is still the same one that
Bulk Data Validation analyzed. The attached file needs to be read from the FasTrak
database. Various data fields of the Bulk Insert record will then need to prepend
themselves, properly formatted, to the just-read data. The data then needs to be written as
a text file somewhere in the Windows file system on the local machine. The FasTrak
system has no “boxed” knowledge of how to do these things. And consideration has yet
to be given to other automated tasks, such as following the Naming Conventions for the
text file name and the summary report that is to be generated external to FasTrak.
The Bulk Insert Process TeamScript will deliver a verified FasTrak file attachment to a
Windows file exchange with the API Interface. It will be triggered when a user clicks the
browser “Submit Bulk File” transition button. At that point, the presumption will be that
the user has confirmed as correct the one and only one Issue data file that is attached to
the Bulk Insert record, and is ready to run the bulk Issue submission.
Team: Transaction Services
106740250
Page 13 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Sequentially, this script will first check that there is only one file attached to the current
Bulk Insert record. The script will then check that, when last analyzed and validated, the
attachment at that time had a valid number of Issues for the Bulk Insert process. Next,
using the uniquely identifying information that was stored earlier, the script will verify
that it still has the same attachment as the one that was last analyzed and validated. If
any of these tests fail, the script will disallow the “Submit Bulk File” transition with an
on-screen message explaining the failure.
To finish, the script will build the Issue data text file, to be picked up by the API
Interface, in the pre-defined target directory. It will retrieve the attachment data and to
that copy of the data, it will then prepend information pertinent to the API Interface
submission process. This information will include the display values of the selected Issue
Type and Sub-Type, the User Login Name of the user who did the file attach, the API user
login name for the actual submitter’s entity, the FasTrak ProjectId relative to that subtype, and the FasTrak record id(ts_id) number of the operative Bulk Insert record. An
example of this pre-pended first line might be:
“D2D,Cancellation with Approval TDSP,1234567890$jdoe,1234567890$BulkInsert,9,1097”
The API user id will be acquired by looking for an exact DUNS number match using the
submitter’s login id (ie: 1234567890$jdoe) to find the API user that corresponds to the
same DUNS number (ie: 1234567890$BulkInsert).
The script will then use Naming Conventions to create a file name for the Issue data text
file, writing the updated attachment data to that file name in the pre-defined target
directory. If any of these steps fail, the script will disallow the “Submit Bulk File”
transition with an on-screen message explaining the failure. If nothing fails, the script
will finish successfully, allowing the Bulk Insert record to complete the “Submit Bulk
File” transition, bringing this record to the “Archived” state.
2.4.4 Bulk Insert Process TeamScript Pseudocode
The following, while not the actual code to be used, should illustrate the expected flow of
this TeamScript process.
; Script entry, triggered by user browser action
record = GetCurrentRecord()
numAttached = RunDbQuery(record, “how many file attachments?”)
If (numAttached <> 1) Then
DisplayMsg(“Invalid number of file attachments”)
AbortTransition()
ExitScript
; Get the number of valid Issues last found & check if within bounds
numValid = GetDataField(record, “issueCount”)
If (numValid < 1) Or (numValid > 1000) Then
Team: Transaction Services
106740250
Page 14 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
DisplayMsg(“Invalid number of Issues found”)
AbortTransition()
ExitScript
; Gather identifying data of the current attachment
userAttached = RunDbQuery(record, “get attachment user”)
timeAttached = RunDbQuery(record, “get attachment time”)
fileAttached = RunDbQuery(record, “get attachment file”)
; Get identifying data stored in the Bulk Insert record
userRec = GetDataField(record, “AttachmentUser”)
timeRec = GetDataField(record, “AttachmentTime”)
fileRec = GetDataField(record, “AttachmentFile”)
; Check identifying data against the data stored
If (userAttached <> userRec) Or (timeAttached <> timeRec) Or (fileAttached <> fileRec) Then
DisplayMsg(“File attachment has changed”)
AbortTransition()
ExitScript
; Get attachment data
dataAttached = RunDbQuery(record, “get attachment data”)
; Get data to prepend
issueType = GetDataField(record, “Issue Type”)
issueSubType = GetDataField(record, “Issue Sub-Type”)
userIdNum = GetDataField(record, “AttachmentUser”)
recordIdNum = GetDataField(record, “Record ID”)
; Prepend the data
preText = issueType + “,” + issueSubType + “,” + userIdNum + “,” + recordIdNum
dataAttached = preText + dataAttached
; Get file name to use
fileName = GetNameConv(record, “file”)
; Write out the file & check if any error
returnCode = WriteTextFile(predefinedPath, fileName, dataAttached)
If (returnCode Not “OK”) Then
DisplayMsg(“Failed writing Issues data file”)
AbortTransition()
ExitScript
ExitScript
2.4.5 Error Handling
How an error is reported on the browser-side by a TeamScript will depend on the
execution time of the script itself. The execution time threshold for reporting one way
versus another is usually determined by how long a browser will wait for a web server
response before timing-out. For Internet Explorer, this is typically 3 minutes.
 If an error is reported before browser time-out, any error messages will be
displayed directly to the browser page. As this type of error message is discarded
when the user does any browser activity, the following method could also be
used.
 Regardless of browser-side reporting and browser time-out, a TeamScript can
always report information, warnings, and errors to the Windows Event log.
Team: Transaction Services
106740250
Page 15 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
2.5 Results Status Report
As the API Interface does the submission to FasTrak of Issues that it processes, the
results of this process will be captured in a report on the MIR report server. This will
include reports on all invalid Issues, listing the reasons they were invalid.
This MIR report will be named using Naming Conventions that will be common to the
FasTrak system and the API Interface. When it goes to the “Archived” state, the FasTrak
Bulk Insert record that was created will display a browser link to the MIR report. A user
may click that link to be brought to the web page for the report, where the current status
and final results of the Bulk Insert process can be checked.
The result report will be an exact replica of the submitted file with the addition of comma
delimited fields appended to the end of each issue row. It is important to note that no
data is appended to the header row (first row) of the bulk request file. The additional
comma delimited fields are appended to the original row in the following order:
1. Status – The result for the issue (i.e. SUCCESS, FAIL, etc…)
2. ErrorCode – A valid error code is supplied if an error occurs, otherwise an
empty string is added so that their will be consecutive commas (…,,…) in the file/
3. Comments – Additional comments (error description, etc…) for the issue. An
empty string will be added of the comments does not apply (…,,…)
4. Timestamp – A timestamp when the issue was processed
5. Issue Number – the resulting issue number if the issue was processed
successfully. If it was not processed, an empty string is supplied (…,)
3
ALTERNATIVES
Following the consideration of the Bulk Insert method recommended above, other
possible methods are listed below. These methods might be used to achieve a better level
of performance, or could be implemented due to the absence of, or an incompatibility
with, other code packages that a method is dependent upon.
The alternatives listed represent the thought that is behind the selection and detailing of a
viable Bulk Insert method.
3.1 Emulation of Web Server HTTP Requests
The Bulk Insert method could be written to emulate the HTTP web requests received by
the API Interface, sending the Issue data that it is to create through that pipeline process.
A TeamScript will format into XML the Issue data fields that are retrieved from the Issue
Data Source. The format for this XML is defined by the API Interface.
Team: Transaction Services
106740250
Page 16 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
3.1.1 Assumptions
 To support this method, an “HTTP Interface” code package would need to be
developed to allow a TeamScript to communicate with the ERCOT web server.
 The machine running the TeamScript and its associated “HTTP Interface”
executable would need to see and post HTTP to the ERCOT web server.
3.1.2 Benefits
 Aside from needing to be compliant with the API Interface, this method would
share in its functionality. This eliminates having to script duplicate code to satisfy
the requirements and processes involved with that process.
3.1.3 Drawbacks
 This method is dependent on an externally developed code package.
3.2 Use of FasTrak API for Issue Create
The final step in the API Interface is the “FasTrak API for Issue Requests” package. This
could be called directly by a Bulk Insert TeamScript. This would eliminate the scripted
XML formatting of the Emulation of HTTP Requests alternative method. It would,
however, place a greater burden of data processing and validation on the script, which
would likely involve the redundant scripting functionality similar to what exists in the
API Interface.
3.2.1 Assumptions
 The “FasTrak API for Issue Requests” package is available to the Bulk Insert
TeamScript.
3.2.2 Benefits
 Aside from needing to be compliant with the “FasTrak API for Issue Requests”
package protocols, this method would share in its functionality. This eliminates
having to script duplicate code to satisfy the requirements and processes involved
with that process.
 ERCOT web server traffic could be reduced, since the Issue submits would be
going from the FasTrak server directly to an executable that would handle the
submit task.
3.2.3 Drawbacks
 This method is dependent on an externally developed code package.
 There would likely be scripting that is redundant to coding in the API Interface,
which could double the effort of code maintenance.
Team: Transaction Services
106740250
Page 17 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
3.3 Stand-alone Custom Executable
The Bulk Insert methods described up to this point have had the issue of being dependent
on other, externally developed code packages. If desired, this dependency could be
eliminated with the implementation of a separate code package to support the otherwise
similar TeamScript. A more specific run-time executable would be written to handle the
Issue submit to the FasTrak system.
3.3.1 Assumptions
 None. This alternative method would require nothing more than access to the
Issue data text files.
3.3.2 Benefits
 Aside from the FasTrak database, this method has no implementation
dependencies.
3.3.3 Drawbacks
 This would be a completely independent method, sharing nothing with the API
Interface or the associated “FasTrak API for Issue Requests” package. This
presents the cumbersome problem of having to maintain 2 methods of otherwise
similar functionality.
 As this method would need to handle all Bulk Insert operations, development time
would be significantly increased.
3.4 TeamScript Only
As any browser-driven Bulk Insert method will require TeamScript, this scripting could
be extended to handle all facets of the process: taking input from the Issue Data Source,
validating this data, and adding it as a FasTrak Issue record.
3.4.1 Assumptions
 None. This alternative method would require nothing more than access to the
Issue data text files.
3.4.2 Benefits
 This method would be fully contained, requiring no support from external
executables or other scripts.
 The script could be installed to the FasTrak database rather than a separate file,
making it “move” along with the FasTrak system whenever it is transferred –
from Test to Prod, for instance. Someone would have to remember to and do the
transfer of any external support files.
Team: Transaction Services
106740250
Page 18 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
3.4.3 Drawbacks
 Unlike the other methods, this method would not follow the regular FasTrak Row
submission path. This eliminates any built-in data field validation, default field
values, and submission path-specific operations that would have been provided by
FasTrak.
 This would be a completely independent method, sharing nothing with the API
Interface or the associated “FasTrak API for Issue Requests” package. This
presents the cumbersome problem of having to maintain 2 methods of otherwise
similar functionality.
 This type of scripting can be difficult to maintain, with this particular script being
large and complex.
 The run-time performance of TeamScript is known to suffer, at times excessively,
compared to more efficient API executables.
 As all Bulk Insert operations would have to be done by this script, development
time would be significantly increased.
4
DATA MODEL CHANGES
None identified thus far.
5
DOCUMENTATION UPDATES
Documentation for process and support information will be created and delivered as
needed.
6
TESTING
Testing documentation will be provided with project deliverables.
Team: Transaction Services
106740250
Page 19 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
APPENDIX A – ISSUE DATA FIELDS
The following tables detail the data fields, per Issue type and subtype, and the field
ordering, left to right, expected for the Issue Data Source. If a field is optional, this
means its value may be specified or left empty with a comma only, or nothing if at the
end of a text-row line.
Issue subtype
Row data field
Required (Req) Optional (Opt) Not Applicable (N/A) Read-Only (RO)
Required or Optional (R/O)
Submitted By/To - additional Issue subtype that may modify field requirements
Issue type: DEV Characteristics
New Station Code
New Zip Code
Comments
ESIID
UIDESIID
STARTTIME
STOPTIME
STATION CODE
PROFILE CODE
LOSS CODE
ADDTIME
REGION CODE
ZIP
Title
N/A
N/A
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
Req
N/A
Opt
Loss Code
Assignment
TDSP/ERCOT Req
or CR/TDSP
N/A
Req
N/A
N/A
N/A
Opt
Req
Req
Req
Opt
N/A
Req
Req
Req
N/A
N/A
Profile Code
Assignment
TDSP/ERCOT Req
or CR/TDSP
N/A
N/A
Req
N/A
N/A
Opt
Req
Req
Req
Opt
N/A
Req
Req
Req
N/A
Station Code
Assignment
TDSP/ERCOT Req
or CR/TDSP
N/A
N/A
N/A
Req
N/A
Opt
Req
Req
Req
Opt
Req
N/A
N/A
Req
ZIP
Assignment
TDSP/ERCOT Req
or CR/TDSP
N/A
N/A
N/A
N/A
Req
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
Team: Transaction Services
106740250
Page 20 of 23
ESIID Validation
New Profile Code
N/A
Assignee
New Loss Code
N/A
ESIID Duplicate
Check
New Region Code
Req
New Start Time
TDSP/ERCOT
Req
or CR/TDSP
Submitted By/To
Region Code
Assignment
Opt
Req
Opt
Opt Opt
Req
Opt
N/A
Opt
Opt
Req
Opt
N/A
N/A
Opt
Opt
Req
Opt
N/A
Req
Opt
Opt
Req
Opt
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Issue type: DEV non-IDR
New Total
New ONPK
New OFFPK
New MDPK
New SPK
Comments
ESIID
UIDESIID
STARTTIME
STOPTIME
TIMESTAMP
Meter Type
Total
Title
ESIID Duplicate
Check
Assignee
ESIID Validation
TDSP/ERCOT Req
or CR/TDSP
Req
Req
Opt
Opt
Opt
Opt
Opt
Req
N/A
N/A
N/A
N/A
Req
N/A
Opt
Opt
Req
Opt
In ERCOT system
not in TDSP
TDSP/ERCOT N/A
N/A
N/A
N/A
N/A
N/A
N/A
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
In both systems
with date issues
TDSP/ERCOT
N/A
or CR/TDSP
Req
Req
Opt
Opt
Opt
Opt
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
In both systems
with kWh or kW issues
TDSP/ERCOT N/A
N/A
Req
Opt
Opt
Opt
Opt
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
N/A
N/A
Req
Opt
Opt
Opt
Opt
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
In ERCOT system
not in MP
In both systems
with kWh or kW issues
CR/TDSP
CR/TDSP
New Start Time
New Stop Time
Submitted By/To
In MP system
not in ERCOT
Issue type: DEV IDR
Submitted By/To
New Start Time
New Stop Time
Comments
ESIID
STARTTIME
STOPTIME
UIDCHANNELCUT
CHANNEL
ADDTIME
Origin
Title
ESIID Duplicate
Check
Assignee
ESIID Validation
CR/TDSP
Req
Req
Opt
Req
N/A
N/A
N/A
Req
N/A
N/A
Opt
Opt
Req
Opt
In MP system
not in ERCOT
TDSP/ERCOT Req
Req
Opt
Req
N/A
N/A
N/A
Req
N/A
N/A
Opt
Opt
Req
Opt
In ERCOT system
not in MP
TDSP/ERCOT N/A
N/A
Opt
Req
Req
Req
Req
N/A
Req
Req
Opt
Opt
Req
Opt
N/A
N/A
Opt
Req
Req
Req
Req
N/A
Req
Req
Opt
Opt
Req
Opt
TDSP/ERCOT Req
Req
Opt
Req
Req
Req
Req
N/A
Req
Req
Opt
Opt
Req
Opt
Req
Req
Opt
Req
Req
Req
Req
N/A
Req
Req
Opt
Opt
Req
Opt
TDSP/ERCOT N/A
N/A
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
N/A
N/A
Opt
Req
Req
Req
Req
Req
Req
Req
Opt
Opt
Req
Opt
In MP system
not in ERCOT
In ERCOT system
not in MP
In both systems
with date issues
In both systems
with date issues
In both systems
with kWh issues
In both systems
with kWh issues
CR/TDSP
CR/TDSP
CR/TDSP
Team: Transaction Services
106740250
Page 21 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Issue type: DEV Existence
Comments
ESIID
UIDESIID
STARTTIME
STOP TIME
STATION CODE
PROFILE CODE
LOSS CODE
ADDTIME
REGION CODE
ZIP
Service Address
Premise Type
Title
ESIID Duplicate
Check
ESIID Validation
Service City
Service State
N/A
Opt
Req
N/A
Req
N/A
Req
Req
Req
RO
Req
Req
Req
Req
Opt
Opt
Opt
Req
Req
In ERCOT system
TDSP/ERCOT N/A
not TDSP
Req
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
N/A
In ERCOT with
start date issues
TDSP/ERCOT Req
N/A
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
N/A
Inactive record
has date issues
TDSP/ERCOT Req
N/A
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
N/A
Status Assignment TDSP/ERCOT Req
N/A
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
N/A
Un-Retire ESI ID 1 TDSP/ERCOT N/A
N/A
Opt
Req
Req
Req
Opt
N/A
N/A
N/A
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
N/A
New Start Time
New Stop Time
Submitted By/To
TDSP/ERCOT Opt
In TDSP system
not ERCOT
Issue type: DEV LSE
New Start Time
New Stop Time
Comments
ESIID
UIDESIID
STARTTIME
STOPTIME
PROFILE CODE
Rep Code
ADDTIME
STATUS
Title
ESIID Duplicate
Check
Assignee
ESIID Validation
LSE in MP sys not ERCOT:
active
Req
N/A
Opt
Req
Req
Req
N/A
Req
Req
Req
Req
Opt
Opt
Req
Opt
LSE in MP sys not ERCOT:
inactive
Req
Req
Opt
Req
Req
Req
Opt
Req
Req
Req
Req
Opt
Opt
Req
Opt
LSE in ERCOT system not MP
N/A
N/A
Opt
Req
Req
Req
Opt
Req
Req
Req
Req
Opt
Opt
Req
Opt
LSE date change: StartTime
Req
N/A
Opt
Req
Req
Req
Opt
Req
Req
Req
Req
Opt
Opt
Req
Opt
LSE date change: StopTime
N/A
Req
Opt
Req
Req
Req
Opt
Req
Req
Req
Req
Opt
Opt
Req
Opt
LSE date change: Start and Stop Req
Req
Opt
Req
Req
Req
Opt
Req
Req
Req
Req
Opt
Opt
Req
Opt
Team: Transaction Services
106740250
Page 22 of 23
7/18/2006
Version 25
PR50007
Design Document
Issue “Bulk Insert” Method
******************************************************
Issue type: D2D
ESI ID
Orig Tran ID
Tran Type
Gaining/Losing Rep
Date of Txn
(required if 867_03
or 867_03F)
GS Number
Comments
Title
ESIID Duplicate
Check
Global ID Duplicate
Check
Assignee
ESIID Validation
Inadvertent Switch
Req
Req
Opt
Req
Opt
N/A
Opt
Opt
Opt
Opt
N/A
Opt
Cancel With Approval
Req
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
Opt
Req
Opt
Cancel Without Approval
Req
Req
N/A
N/A
N/A
N/A
Opt
Opt
Opt
Opt
N/A
Opt
Missing TXNS
Req
Req
Req
N/A
Opt
N/A
Opt
Opt
Opt
Opt
Req
Opt
Usage/Billing Issues
Req
Req
Opt
N/A
R/O
N/A
Opt
Opt
Opt
Opt
Req
Opt
Reject TXNS
Req
Req
Req
N/A
Opt
N/A
Opt
Opt
Opt
Opt
Req
Opt
Rep of Record
Req
N/A
N/A
N/A
N/A
N/A
Opt
Opt
Opt
N/A
Req
Opt
Projects
Opt
Opt
Opt
N/A
Opt
Opt
Opt
Opt
Opt
Opt
Req
Opt
Siebel CHG/Info
Req
Req
N/A
N/A
Opt
N/A
Opt
Opt
Opt
Opt
Req
Opt
997 Issues
N/A
N/A
N/A
N/A
Req
Req
Opt
Opt
N/A
N/A
Req N/A
Other
Opt
Opt
Opt
N/A
Opt
Opt
Opt
Opt
Opt
Opt
Req
Team: Transaction Services
106740250
Page 23 of 23
Opt
7/18/2006
Version 25