Language-based Implementation
of Multi-Vendor Support in an
AdvancedTCA Shelf Manager
Sergey Zhukov
CEE-SECR 2010
October 13-15. Moscow, Russia
Introduction
 Auriga/Pigeon Point Systems (PPS) collaboration
- Dedicated engineering team for more than 10 years
- Specialized in management for telecommunications systems based on open
standards
 Current project: management for AdvancedTCA systems
- AdvancedTCA (ATCA) – an open standard for multi-board systems (shelves) used in
telecommunications, developed by PICMG standards body in 2003
- AdvancedTCA boards can be single-board computers or specialized boards (DSPs,
storage boards)
- Shelf can host boards from different vendors
- Management based on IPMI management specification from Intel
- MicroTCA – a smaller form-factor variant of AdvancedTCA for use in other areas
(industry, medical)
2
CEE-SECR 2010
Management Architecture in AdvancedTCA
3
CEE-SECR 2010
Pigeon Point Shelf Manager
 Hardware and software solution for management in AdvancedTCA
shelves
- Implemented on a small mezzanine single-board computer (ShMM)
- Runs embedded Linux
- Shelf Management software runs as a user-mode application
 Needs vendor-specific carrier board to plug into the shelf
- Interface between a carrier board and a shelf is vendor-specific
- Device population of a carrier board is vendor-specific
 Shelf Manager operates with carrier boards and shelves from
different vendors
4
CEE-SECR 2010
An AdvancedTCA Shelf with Shelf Manager
5
CEE-SECR 2010
Shelf Manager Software Adaptation Problem
 To provide multi-vendor support, Shelf Manager software needs
adaptation to a specific carrier and/or shelf
 Previously: carrier- and shelf-specific pieces of code in the Shelf
Manager software
- Does not scale well when number of supported vendors grows, which
happened over time
- However the most practical solution for not very powerful original
ShMMs
- But in newer ShMM models processing power increased
 A language-based solution to this problem has been introduced
6
CEE-SECR 2010
Language-Based Solution
 Define a language for describing carrier and shelves
- Hardware Platform Definition Language - HPDL
 Each language unit (module) describes a carrier or a shelf
- Created by PPS engineers or by shelf/carrier vendors
 Modules compiled into binary format by a special compiler
 Binary format loaded and interpreted by the Shelf Manager
software
7
CEE-SECR 2010
Storing Binary HPDL modules
 Preferred location – Carrier or Shelf FRU Information storage
- AdvancedTCA/IPMI standard storage
- Collocated with the respective carrier or shelf
- Coexists with other (standard) records that describe carrier and
shelf
- Carrier and shelf become self-describing
 Stored together with sensor data records (SDRs)
- SDRs are standardized descriptions of IPMI sensors
- Shelf Manager uses HPDL descriptions and SDRs together
8
CEE-SECR 2010
HPDL Features
 Describes following objects:
- Shelf Manager carrier board
- Other modules on the carrier board (mezzanines, RTMs)
- Non-intelligent modules in the shelf under direct control of the
Shelf Manager (power supplies, fan trays, alarm panels)
- I/O devices on the carrier board and in the shelf, controlled by
the Shelf Manager (mostly I2C devices)
- For each device, accessible signals and expressions over these
signals are defined
- IPMI sensors on the carrier board and in the shelf, bound to I/O
devices, signals and expressions
9
CEE-SECR 2010
HPDL Object Hierarchy
Device
Signal
1
*
*
FRU
*
1
1
1
*
Sensor
*
*
Expression
Fan Tray
10
CEE-SECR 2010
Describing FRUs
 FRU as an object is defined in the AdvancedTCA specification
- Hot swappable: can be replaced dynamically
- Power Management: dynamically powered on and off
- Supports other AdvancedTCA features – payload reset, etc.
 Carrier board, non-intelligent modules, mezzanines are represented
as FRUs
 In HPDL, described via “FRU” statement with subordinate clauses
and sub-statements
- Unique name
- Presence expression: indicates physical presence of the FRU
- Hot Swap latch expression and LED signal
11
CEE-SECR 2010
Describing FRUs (2)
 More clauses for “FRU” statement
- FRU Information location (normally an EEPROM)
- Signals for other standard and custom LEDs (besides Hot Swap)
- Initial and periodic actions (local FRU management policies)
- List of device descriptions for devices that belong to the FRU
- List of IPMI sensor descriptions for sensors that belong to the
FRU
- Amount of power consumed by the FRU (used in power budget
calculation and management)
- Some other spec-defined FRU properties
12
CEE-SECR 2010
Describing Fan Trays
 Fan tray is a specific case of FRU
- Includes one or more fans that are used for shelf cooling
 In HPDL, defined by “FAN TRAY” statement
 Includes all clauses of the “FRU” statement, plus:
- Fan speed properties: minimum, maximum and recommended
speed level
- A signal or set of signals to set fans to a certain speed level
- A signal to cause emergency shutdown (in case of fire in the
shelf)
- List of fan speed (tachometer) sensors, associated with each fan
13
CEE-SECR 2010
Describing Devices
 I/O devices on the Carrier board and in the shelf
- Directly controlled by the Shelf Manager (without IPMI)
- Mostly I2C devices, with some exceptions (direct ShMM GPIOs)
- Each device is associated with some FRU
 In HPDL, defined by “DEVICE” sub-statement
- In the scope of the corresponding “FRU” statement
 Includes the following clauses:
- Device type
- Device bus number and address
- List of used signals
14
CEE-SECR 2010
Describing Signals
 Set of signals represents the device interface
 A signal can represent
- An integer register
- A single bit in a register
- A data storage (e.g. an EEPROM)
 Signal can be read-write, read-only or write-only
 Internal signals a for device are defined by device type
 HPDL maps internal signals to externally visible signals
- Directly or using expressions
 Defined by “SIGNALS” clause
- In the scope of the corresponding “DEVICE” statement
15
CEE-SECR 2010
Handling Devices in the Shelf Manager
 Some devices need an HPDL driver
- A piece of code in the Shelf Manager software
- Defines internal signals that can be exposed
- Provides access to the actual device via the signals
 Some I2C devices do not need a special driver
- Use generic I2C driver in the Shelf Manager software
- Exposes 256 byte-sized signals (mapped to registers) and
256*8 one-bit signals
- Access via Read Byte/Write Byte SMB commands
16
CEE-SECR 2010
Describing Sensors
 AdvancedTCA uses IPMI sensor model
- Sensor properties described in Sensor Data Records (SDRs)
- Analog sensors
- Expose 1-byte raw value
- Translated to actual value using formulas in SDR
- Support thresholds and hysteresis
- Discrete sensors
- Up to 15 states, can be set simultaneously
- Settable sensors: value/states can be set via IPMI commands
 HPDL: mapping device signals to IPMI sensors
- To make signals accessible via standard IPMI mechanisms
17
CEE-SECR 2010
Describing Sensors (2)
 Described by “SENSORS” clause in a “FRU” statement
 For each sensor, following attributes are specified
- Sensor number and LUN (IPMI addressing information)
- Sensor value as expression or expressions over signals
- Initial state for discrete sensors
- Polling interval in milliseconds
- For settable sensors, signals to set when the value is set externally
 SDRs are compiled separately and stored in a parallel data storage
- SDR binary format defined by the IPMI specification
- Compiled by already existing (reused) compiler
- When implementing sensors, Shelf Manager software uses both HPDL and SDRs
- SDRs provide sensor properties, HPDL provides binding to hardware
18
CEE-SECR 2010
HPDL Example
FRU Carrier_FRU {
Main_3_3V: Main_3_3V;
SITE 3, 1;
Plus_5V: Plus_5V;
...
Minus_12V: Minus_12V;
POWER_LEVELS { 10000; };
Vbat: Vbat;
FRU_INFORMATION MasterADM1026.EEPROM;
# Tachometer signals 0 to 7
DEVICES {
Tachometer0: Tachometer0, "8";
DEVICE Multiplexer {
...
PCA9545: 0: 0x70;
Tachometer7: Tachometer7, "8";
SIGNALS { Channels:
# GPIOs: input, active low
Channels, "1", "2", "3", "4"; };
PwrGoodBus: GPIO12, in, low;
};
PwrGoodAcb: GPIO13, in, low;
DEVICE MasterADM1026 {
FanControl: PWM;
ADM1026: 0: 0x2e;
FanLocalControl: PWM_AFC;
SIGNALS {
EEPROM: EEPROM;
Temp1: IntTemp;
Temp2: ExtTemp1;
Temp3: ExtTemp2;
};
};
}; #
End of DEVICES statement
St_3_3V: Stdby_3_3V;
19
CEE-SECR 2010
HPDL Example (2)
...
SENSORS {
0x10,0,2: 3000: MasterADM1026.Temp1;
0x10,0,3: 3000: MasterADM1026.St_3_3V;
0x10,0,6: 3000: MasterADM1026.Vbat;
0x10,0,15: 3000: MasterADM1026.PwrGoodBus;
0x10,0,16: 3000: MasterADM1026.PwrGoodAcb;
# Expression used for the next sensor
0x10,0,17: 3000: !MasterADM1026.PwrGoodBus || MasterADM1026.PwrGoodAcb;
}; # End of SENSORS statement
}; # End of FRU statement
20
CEE-SECR 2010
ASN.1 as Binary HPDL Format
 ASN.1 BER (optionally compressed) as format for binary data
 Produced by HPDL compiler, parsed by the Shelf Manager software
 Motivations
- Backward compatibility (new Shelf Manager interprets old HPDL
modules)
- Forward compatibility (old Shelf Manager interprets new HPDL modules)
- Both can be achieved using ASN.1 tagged encoding
- Compactness of representation (EEPROMs are small)
- Decoding can be done efficiently (limited resources on ShMM)
- Availability of libraries that encode and decode BER
21
CEE-SECR 2010
HPDL Compiler Implementation
 Command-line program in two versions: for
Windows and for Linux
- Implemented in C++
- HPDL defined by a context-free grammar
- Bison used as a parser generator
- Can be compiled by GCC for Linux, by MSVC for Windows
- Special tools to convert output to IPMI FRU Information
format
22
CEE-SECR 2010
HPDL Introduction: Case Study: Client A
 Shelf and Carrier Board vendor, collaborates with PPS for a long time
- Carrier-specific code has been used to support their carriers and shelves
- Initialization parameters were used to specify actual carrier and shelf model
 Gradual transition to HPDL for this client
- HPDL now used to support all new shelf models and new generation Carrier boards
- Shelves became self-describing, no initialization parameters needed
- Carrier-specific code still exists to support legacy hardware but is not actively
maintained recently
- HPDL modules are developed by PPS engineers, Client engineers have a working
knowledge of them and sometimes make changes (in coordination with PPS)
- Many change requests from the client affect HPDL modules only and do not require
a new release of the Shelf Manager
23
CEE-SECR 2010
HPDL Introduction: Case Study: Client B
 Another shelf and Carrier board vendor, recently came to
AdvancedTCA market
- HPDL was already used in PPS
 No carrier-specific code for this client, only HPDL
 HPDL modules developed by engineers on the client side
- Made accessible to PPS to review and store in the common
repository of HPDL modules
 Client is self-sufficient with respect to support of their
hardware with PPS management solutions
24
CEE-SECR 2010
Conclusion. Advantages of Using HPDL
 No need of special Shelf Manager release to
support new carrier boards/shelves
 Dramatically reduced effort to implement support
for new carrier boards/shelves in PPS
 Carrier and shelf vendors can themselves use
HPDL for their products and take support activity
in their own hands
25
CEE-SECR 2010
Resources
 Auriga Inc.
- http://www.auriga.com
 Pigeon Point Systems
- http://www.pigeonpoint.com
 PICMG
- http://www.picmg.org
 IPMI
- http://www.intel.com/design/servers/ipmi
26
CEE-SECR 2010
Q&A; Auriga At a Glance
 World leader in R&D outsourcing
- Global Services 100 and Global Outsourcing 100
company
- Top 10 Offshore Eastern and Central Europe
 In business since 1990—first in Russia
 Incorporated in the U.S. in 1993
 5 engineering centers in Russia—Moscow (3), Nizhny
Novgorod (1), Rostov-on-Don (1)
 Engineering center in EU (Vilnius, Lithuania)
 Close relationship with the leading Chinese vendors
 250+ FTEs with low attrition & rotation
 CMMI Level 4 company
 SPICE (ISO 15504) assessed
- Life critical software compliant
 Focus on software R&D and product engineering for hightech clients
 Leader in system-level & embedded development services
 Proficient in Web & enterprise applications
 Mobile technologies & Knowledge management
solutions as outposts of further expertise development
27
CEE-SECR 2010