3700 San Martin Drive, Baltimore, Maryland 21218 Final External Report 01-045 Revision 01-045 WFPC-2 Shutter-A Position Sensing Error Fault Isolation Chris A. Long February 13, 2001 Wide Field and Planetary Camera-2 I. Problem Overview....................................................................................................................................... 1 II. Fault Isolation............................................................................................................................................. 1 A. System Level Isolation........................................................................................................................... 2 1. External to Spacecraft ......................................................................................................................... 2 2. Spacecraft (excluding payload)........................................................................................................... 2 3. Science Instruments ............................................................................................................................ 3 B. WFPC2 System Level Isolation ............................................................................................................. 3 1. Clock I/F circuits................................................................................................................................. 3 2. Mechanism Power Supply .................................................................................................................. 3 3. Engineering Telemetry........................................................................................................................ 3 4. Microprocessor ................................................................................................................................... 3 5. LED Drive Electronics........................................................................................................................ 3 6. Sensor-side Components..................................................................................................................... 3 C. Possible Faults on the Shutter Assembly................................................................................................ 4 1. Description.......................................................................................................................................... 4 2. List of Faults ....................................................................................................................................... 4 III. Targeted Fault Isolation ............................................................................................................................ 5 A. Mechanical Degradation/ Detent Translation......................................................................................... 5 1. Method of Elimination ........................................................................................................................ 5 2. Measuring Sensor-to-Encoder Alignment........................................................................................... 5 3. Sensor Alignment Evaluation ............................................................................................................. 7 B. Mechanical Degradation/ Detent Play.................................................................................................... 8 1. Method of Elimination ........................................................................................................................ 8 2. Detecting Mechanical Play ................................................................................................................. 8 C. Hard Mechanical Failures ...................................................................................................................... 9 1. Types of Failure .................................................................................................................................. 9 2. Alignment Offset ................................................................................................................................ 9 3. Blade Not in Detent .......................................................................................................................... 10 D. Evaluate Electrical Degradation........................................................................................................... 11 1. LED Degraded Output ...................................................................................................................... 11 2. Phototransistor Degraded Sensitivity................................................................................................ 12 3. Combined Effects in Sensor Package ............................................................................................... 12 E. Evaluate Hard Electrical Failure........................................................................................................... 12 1. LED Drive Resistors ......................................................................................................................... 12 IV. Remaining Faults.................................................................................................................................... 12 A. Evaluate Remaining Faults .................................................................................................................. 12 1. Sensor Package Degradation............................................................................................................. 12 2. Open Resistor in LED Drive Path..................................................................................................... 13 3. Sticktion near Open Detent ............................................................................................................... 13 V. Monitoring & Risk Reduction.................................................................................................................. 14 VI. Future Efforts.......................................................................................................................................... 14 VII. Conclusions ........................................................................................................................................... 15 VIII. Acknowledgements.............................................................................................................................. 15 IX. Appendix A: Interpolated Time Variance.................................................................................................ii A. Description.............................................................................................................................................ii 1. Data Manipulation ..............................................................................................................................ii 2. Plots ....................................................................................................................................................ii X. Appendix B: Interpolated and Speed Flag Timing..................................................................................... v A. Description............................................................................................................................................. v 1. Data Manipulation .............................................................................................................................. v 2. Plots .................................................................................................................................................... v XI. References .............................................................................................................................................xiii WFPC-2 Shutter-A Position Sensing Error Fault Isolation Chris A. Long Space Telescope Science Institute Baltimore, Maryland February 13, 2001 The Wide Field Planetary Camera II science instrument reported shutter errors in the summer and fall of 2000. No further errors have occurred since the RAM patch installation to advance the sensor turn-on time. Reduced photon coupling in the shutter position sensor has been identified as the cause of the errors. Fault analysis has limited the list of possible problems to the degradation of two electrical position-sensing components, an open resistor, or an occasional offset of the shutter blade position. No health and safety risks have been identified. I. Problem Overview The Wide Field Planetary Camera 2 (WFPC-2) issued an error message on August 31, 2000 (day 244). The message content of 0688066816 indicated that both shutter blades reported their position as "closed" during the pre-move check at Real-Time Interrupt (RTI) 1640i. The shutter's mechanical design makes it impossible for both blades to actually be in the closed position. The instrument recognized that the shutter condition was unknown and aborted commanding to open the shutter. The science data shows that the shutter was closed during the exposure. When the exposure reached it's scheduled end time the microprocessor commanded blade-A to close so that the shutter would enter a known condition. During this move the blade-A time-of-flight was higher than normal showing that the blade did close and that it was pushing blade-B open in the process. The Goddard Space Flight Center (GSFC) investigation revealed that the errors began earlier than the first one received at the console. Telemetry recorded previous errors on August 21 (234) and August 30 (243). Subsequent errors occurred on October 2 (276), 11 (285), 25 (299), 26 (300), 29 (303), 30 (304), and November 1 (306). WFPC-2 was safed on November 1 and a change to the flight software was installed before recovering the instrument on November 7. The software change (RAM patch) turns on the blade position sensors earlier in the process thereby increasing the time between energizing the LED/photodiode package and reading the blade position. Since this change there has been no further errors. An early evaluation of the problem led to the conclusion that the errors could be the result of mechanical degradation or failure. Certain failures in this category could lead to instrument damage unless identified and precluded by operational changes so the STScI Engineering Team deemed an in-depth analysis necessary. II. Fault Isolation This report describes the effort to answer the following questions: • What caused the error on August 31, 2000 (day 244)? • Is the cause of this error a health and safety concern? In a nutshell the health and safety boils down to a question of whether the cause is electrical or mechanical. If mechanical there may be a health and safety concern but if electrical, probably not. The analysis will largely be driven by the effort to eliminate one or the other. 1 A. System Level Isolation Each spacecraft system and outside systems that are related will be considered for the possibility that they could cause the error message on day 244. 1. External to Spacecraft a) Environment (non-thermal): The anomaly with 13 consecutive exposures reporting an error rules out a SEU problem. Had there been a SEU the blade position would have been correct on subsequent exposures. No other environmental possibilities have been identified. b) TDRSS: The only TDRSS function is passing data. Ruled out as with SIC+DH. c) White Sands: The only White Sands function is passing data. Ruled out as with SIC+DH. d) Ground-system (PASS, CCS, TRANS, RPS2): Data in and out has been eliminated. 2. Spacecraft (excluding payload) a) Transmitter: Data in and out has been eliminated. b) Safing: SMAC-20 is the only connection and is not relevant in this case. c) Thermal: Thermal cannot cause the error messages directly. d) PCS: WFPC-2 and PCS have no direct connection. e) EPS (PCU fault): Power is doing it's job, providing power f) 486: The TDF is not polled at the time that the shutters are checked and an error generated. There is nothing else in the PIT that WFPC2 checks and since this is the only communication between the two the 486 is not the problem. g) OTA: Photons are the only link. h) SIC+DH/NSSC-1: Commanding to the WFPC2 must be correct because the command sequences are occurring on time. Data coming out is not being corrupted because the science data is showing real problems (blank exposures). Timing errors would be generating their own problems so timing isn't the issue. Science data readout through the SIC+DH can be discounted. i) DMS: Ruled out because the only communications path is through the 486 and it has been eliminated. Cal Flip Mirror Mechanism HST WFPC-2 OTA Image SDF Bus A SDF Bus B F/24 Pickoff Mirror Mechanism UV Cal Lamps Pyr. Lamps SOFA Pyramid Mirror Shutter Assy. DSP/ADC (Bay 4) CH Elec. (Ch 1) CCD (Ch 1) CCD Window (Ch 1) PC Relay Optics (F/28.3) PC Fold Mirror (F/28.3) #1 AFM DSP/ADC (Bay 3) CH Elec. (Ch 2-4) CCD (Ch 2-4) CCD Window (Ch 2-4) WFC Relay Optics (F/12.9)) WFC Fold Mirrors (F/12.9) #2 Fixed #3/4 AFMS SDF Select SDF A Interface Electronics Clock A Clock B Clock I/F Select RIU C Clock I/F Circuit AFM Electronics Microprocessor PROM/RAM TEC (Ch 1-4) Heat Pipes (Ch 1-4) Radiated Heat to Space Radiator Sup/Rpl Bus B RIU Pw r B Sup/Rpl Bus A LEGEND Optical Thermal Electrical VIS Cal Lamps OR SDF A Interface Electronics Cal Lamps OR RIU B Logic Temperature Control TEC PS On/Off Bay 1-5 & Ch 1-4 RIU A Eng Data Mechanism Drivers Mechanism PS LVPS Bay 5 Heaters Repl. Htrs. On/Off Bay 1-4 & Ch 1-4 Heaters HP Htrs. On/Off HP 1-8 Heaters RIU Pw r A Pwr Bus A B Relays/ Selects Sensors Mechanisms & Lamps EMI Filter Box Power Select Mech PS Htrs OR On Power On/Off Off cal Figure 1: WFPC-2 functional block diagram with suspected systems shown in red. 2 RIU Serial Cmds Clock A Figure 2: WFPC-2 systems not yet eliminated from the list of possible faults. Clock B Power On/Off Microprocessor PROM/RAM Clock I/F Select Clock I/F Circuit Logic Shutter Enable LVPS Mechanism PS OR Eng Data A open/close B open/close Mechanism Drivers Solenoid drive Failsafe drive Mech PS Htrs Shutter Assy. +5V Sensors Sensor A Sensor B cal 3. Science Instruments a) COSTAR: Only shares power with WFPC2 that would be seen as an EPS problem. b) FOC: No direct connection to WFPC2. c) NICMOS: Only shares power with WFPC2 that would be seen as an EPS problem. d) STIS: No direct connection to WFPC2. e) WFPC2 The conclusion of the system level analysis is that the only possible location for the cause of this error is within the WFPC-2 instrument. B. WFPC2 System Level Isolation Most WFPC-2 systems can be eliminated on the basis that there is no interaction with those systems generating the shutter error message. When the obvious sub-systems have been eliminated the functional block diagram reduces as shown in figure 2. Those systems shown in red remain candidates for harboring the fault. 1. Clock I/F circuits a) The clock interface circuitii provides the timing from which microprocessor clocks are derived. It also supplies the clocking for CCD readout. Since the clamp-to-sample time and synchronization of WFPC2 electronics with the readout are confirmed to be correct by proper imaging the clock interface circuits and upstream components are not the source of the problem. 2. Mechanism Power Supply a) Providing the shutter drive pulse is the only mechanism power supplyiii function related to shutter operation. Mechanism power is applied to the shutter assemblyiv through connector M3A1/P1. There is a connection to the encoder circuit board but only as a place to mount the anti-kickback diodes. The error under consideration occurs during shutter movement setup before the move is commanded and a drive pulse is needed so the mechanism power supply and upstream OR circuit cannot be the source of the problem. The Low Voltage Power Supply cannot yet be eliminated as it has a separate connection to the sensors. 3. Engineering Telemetry a) The shutter position error occurs prior to any related engineering telemetry collection. As such engineering telemetry cannot be the source of the problem. 4. Microprocessor a) All physical constructs within the microprocessor are shared by many functions that would independently generate errors should a failure occur. Unique to blade-A are small code segments and RAM space. The code is verified to be good by ongoing checksum testing and the RAM by memory tests so the microprocessor is not the source of the problem. 5. LED Drive Electronics a) The sensor electronics schematic is shown in figure 3. Except for two resistors for each LED the drive electronicsiii power the LEDs for both blades in parallel. The blade-B sensor package has been working properly and therefore all components may be eliminated as a problem except for the resistors unique to the A-side, R5 and R105. This also eliminates the logic circuits, mechanism drivers, and power supplies associated with commanding the LED circuit. 6. Sensor-side Components a) Failure of the sensor noise reduction componentsiii R8 or C2 will affect both phototransistors. R9 failure will result in the output fixed at either the high or low rail. The U1 buffering device 3 Figure 3: WFPC-2 shutter encoder electronics. Excerpt from dwg10093548. must be working properly as it is too fast to "slow" the output signal and intermittent operation would show up in blade position telemetry. The register G002v (sheet 4) signals and the electronics to support them have already been eliminated since this is the same signal that drives each LED. Both G002 and J008 must be working properly as good telemetry is passing through this same path. This eliminates all receiver-side components and the supporting logic circuits except for the A-side phototransistor itself. The fault is now isolated to the shutter assembly. C. Possible Faults on the Shutter Assembly 1. Description a) The previous step concluded that only the shutter assembly remains a harbor for the fault. The possible problems on the shutter assembly that could cause the errors are shown in figure 4 and listed here: 2. List of Faults a) Mechanical (1) Blade-A stop integrity. (2) Encoder-to-Sensor alignment. (3) Sensor-A contamination. (4) Shutter-A drive mount integrity. (5) Magnetic detent integrity on drive-A. (6) Shutter-A drive arm integrity. (7) Shutter-A encoder arm integrity. (8) Shutter-A keel integrity. b) Electrical (1) LED-A degradation. (2) Phototransistor-A integrity. (3) Sensor-A wiring. (4) R5 or R105 open. 4 Electrical (E1) LED-A degradation. (E2) Phototransistor-A integrity. (E3) Sensor-A wiring. (E4) R5 or R105 open. SHUTTER-A OPEN (M1) (M8) (E4) (E1) LED Sensor (E3) Package (M7) (E2) PT (M6) Mechanical (M1) Blade-A stop integrity. (M2) Encoder-to-Sensor alignment. (M3) Sensor-A contamination. (M4) Shutter-A drive mount integrity. (M5) Magnetic detent integrity on drive-A. (M6) Shutter-A drive arm integrity. (M7) Shutter-A encoder arm integrity. (M8) Shutter-A keel integrity. N S (M4) (M2) (M3) 0 1 2 N (M5) S 3 Blade Open 4 5 6 cal Figure 4: Possible faults on the shutter assembly. III. Targeted Fault Isolation Mechanical degradation can cause the error messages only by changing the encoder disk to sensor alignment thereby interfering with the light passing through slot zero. As the system ages there will be wear in various components that could cause the open detent position to slowly translate away from the original rest position or the detent could become larger with the actual rest position become less well defined. A. Mechanical Degradation/ Detent Translation 1. Method of Elimination a) The key to eliminating degradation that causes mechanical translation from the list of possible failures is to prove that the encoder disk slot 0 to sensor package alignment remains unchanged. To prove this: (1) It will be shown that alignment changes can be measured with high precision. (2) Such measurements will show that the alignment is unchanged. b) Important note: The following argument is not commutative. If the position of the detent changes it must cause a change in the interpolated time. However, it is not true that a change in the interpolated time necessarily reflects a change in position! 2. Measuring Sensor-to-Encoder Alignment a) WFPC2 telemetry provides timing measurements that allow the relative position of the sensor package to encoder disk to be measured with significant precision. "Shutter A Interpolation" measures the time from the start of the current Major Frame to the encoder disk slot 1 passing by the LED. The only variable is encoder disk time-of-flight. b) When opening the shutter the interpolated time includes the encoder time-of-flight from the closed position to the slot 6 leading edge passing by the LED. Figure 6 shows this as the 5 Figure 5: Shutter-A encoder timeof-flight in milliseconds during blade closing on day 244, 2000. distance between the red (middle) and black (upper) lines. The closing interpolated time is the time-of-flight from the open (red, middle line) to the slot 1 trailing edge (blue, lower). c) Angular resolution of the encoder between slots 0 and 1 while opening. (1) Figure 5 plots the angular position of the blade-A encoder versus elapsed time in milliseconds for a blade opening shortly before the day 244, 2000 error message. The data denoted by squares is from the speed flags and is well defined. The ∆t between zero and the start of flag 1 is not directly known. This is the region of interpolated time that does not have a defined start time. To complete the curve the data is fitted to the diamonds then the interpolated ∆t is selected to meet the boundary condition that the starting point must be at the origin. (2) A 110ms drive pulse to the solenoid provides acceleration to the shutter blade throughout the flight shown in figure 5. The angular velocity therefore increases throughout the flight. The average angular velocity between slots 0 and 1 may be found using telemetry point USHRAINT, "Shutter A Interpolation." (3) The angular distance between slots 0 and 1 is 4° as shown in figure 7. When the encoder has traveled this distance the LED will be aligned with the middle of slot 1. The exact alignment between the LED and the slot when the sensor reads a "0" is unknown but will be assumed to be 1° further, or 5° from the starting point. At this point the LED will only have a small fraction of its area still exposed. A similar end point will be used for the speed flags. (4) From figure 5 the time-of-flight is 11.7ms. The average angular velocity is then ϖ 0→1 = ∆θ ∆t = 5°11.7ms = 0.43° / ms. (5) The resolution of the "Shutter A Interpolation" telemetry point USHRAINT is 40µs. At an angular velocity of 0.43°/ms the distance traveled during 40µs is ∆θ 40 µs = 0.43° / ms × 40 µs = 0.017° . (6) 0.017° is the angular motion that will occur during a single increment in interpolated time. Figure 7 provides that the slots travel on a radius of 0.8″ and each slot is 0.02″ widevi. The slot width in degrees is then wslot = w × 360° 0.02"×360° = = 1.43° π × 2r π × 2 × 0.8" (7) Dividing the 1.43° slot width by 0.017° of angular resolution yields 1/80th the distance across a single slot. This is the minimum shift in encoder position that will change the interpolated time. Figure 6: Shutter-A encoder in the open (blue) and close (black) positions. 6 Figure 7: Mechanical layout of the shutter-A encoder blade. d) Encoder alignment resolution (1) Interpolated time is able to detect positional changes as small as 0.017°. (a) Note that interpolated time does not reflect the actual time-of-flight between slots 0 and 1 as the timer is started at the last Major Frame boundary. The 10ms timer will have rolled over many times prior to capturing shutter motion, but since the motion is the only variable in timing the data will report time-of-flight changes with great precision. 3. Sensor Alignment Evaluation a) The mechanical issue in question is whether or not a change in the shutter-A encoder blade to sensor alignment is blocking the LED light resulting in a closed reading when the blade is actually open. The only speed flag data that is directly pertinent to this question is the shutter-A interpolated time while the blade is closing. Since minute shifts in encoder to sensor alignment will cause a change in interpolated time, interpolated time being stable will rule out all mechanical degradation causing drift as the source of the problem. b) Figure 8 shows the shutter-A interpolated time for microprocessor-controlled blade closing between the installation of WFPC-2 and the first occurrence of a shutter error. Mechanical degradation, or wear, that causes the open detent position to shift will result in a change to this data. If the detent position shifts to the left (figure 4) the encoder blade will rotate counterclockwise. For every 0.017° of rotation the interpolated time will increment by a 40µs LSB. Similarly the interpolated time will decrement if the detent position shifts to the right. Figure 8 clearly shows that the interpolated time has been stable over the span of the mission Figure 8: Shutter-A interpolated time for a microprocessor controlled closing. The black line is a 30-sample moving average filter. The data spans from launch to the 1st shutter error. 7 Figure 9: Shutter-A closing interpolated time variance from launch to the first shutter error. Data is biased to 11.7ms, normalized, and filtered with a 30sample sliding average. Appendix A contains information on shutterB and on the opening times. indicating that the open detent position is unchanged within 1/80th of a slot width. c) The stability of the interpolated time is sufficient to show that mechanical wear resulting in a shift of the open detent is not the cause of the shutter errors. There is a related failure mode that has not been ruled out. That is mechanical degradation that does not shift the position of the detent but instead results in excessive play of the detent position. This possibility will be evaluated in the next section. B. Mechanical Degradation/ Detent Play 1. Method of Elimination a) The key to eliminating degradation that causes mechanical play from the list of possible failures is to prove that the encoder disk slot 0 to sensor package alignment is repeatable. To prove this: (1) It will be shown that alignment changes can be measured with high precision. (2) Such measurements will show that the alignment repeatability is unchanged. b) The argument for monitoring play in the detent is similar to using the interpolated time mean to show that the detent position is unchanged. In this case the interpolated time variance will be used to show that the detent position is repeatable. The use of variance in this manner is meaningful but less rigorous than the connection between mean and position. As such the conclusions drawn are more open to interpretation. 2. Detecting Mechanical Play a) The telemetry used to evaluate the amount of play in the shutter-A interpolated time is described in the previous section. The interpolated time measures the time-of-flight from the open detent to the slot 1 trailing-edge (angular distance between the figure 6 red and blue lines). b) If the detent position changes by 0.017°, or 1/80th of the slot width, the interpolated time will increment so small uncertainties in the detent position will cause spreading of the interpolated time. To say it another way, if the exact positioning of the blade in the open detent is not always the same then the interpolated time will take on more values. Therefore if the detent is less well defined the interpolated time variance will become greater. c) The variance of the blade-A interpolated time has been evaluated for AP-17 commanded openings throughout the mission. Among other things this measurement will indicate if blade-A always rests in the same position when open. Figure 9 shows the normalized variance. To normalize the data the interpolated time (shutter-A AP-17 closing) has been biased upward so that the average value equals 11.7ms, the detent to slot 1 time-of-flight. The 20k points of interpolated time are then divided by a 30-point moving average to take out the slope of the line. A 30-point sliding variance is then applied to the result. d) The variance does not show degrading performance throughout the mission. This is not unexpected for the detent positioning. The detent position is most likely established by the blade resting against the stop or the physical carbon steel detent resting against the drive 8 Figure 10: Shutter-A closing interpolated time including shutter errors with a 20-point sliding average overlaid. solenoid. In each case the permanent magnet on the drive arm should hold the shutter against a well-defined stopping point so the detent position would remain repeatable and fixed. e) The result of this analysis is that the detent position is not showing signs of increased sloppiness or play. Combined with the detent position analysis of the previous section it can be concluded that the shutter-A encoder alignment is unchanged from launch up to the start of the shutter errors and therefore general mechanical degradation or wear can be ruled out as the source of the problem. f) Appendix B provides further details on the derivation of the normalized variance and shows the results for all 4 detents: Shutters A and B both open and close. C. Hard Mechanical Failures The analyses in the previous sections eliminate slow mechanical degradation as a source of the shutter errors. A mechanical fault coinciding with the onset of the errors is considered here. Whatever the source, a mechanical issue can only cause the shutter errors in question by changing the encoder-to-sensor alignment thereby imposing on the sensor light path. This type of problem can be either: 1. Types of Failure a) An alignment offset that partially blocks the light path reducing the effectiveness of the position sensors resulting in a system that works most of the time with occasional failures. b) A condition where blade-A does not always come to rest in the open detent. 2. Alignment Offset a) An offset in the encoder alignment brought on by a hard, or rapid, failure of the shutter-A assembly would be characterized by a step in the interpolated time at or near the first error. The step would be a result of the change in angular distance between the open detent and the slot 1 trailing edge. Figure 10 is the same as figure 8 but is extended to include the interpolated time data following the onset of shutter errors. The figure shows that there is a marked increase in the interpolated time coinciding with the shutter errors. However, the magnitude of the shift is very small in absolute terms. 9 b) The interpolated time while opening blade-A has incremented by approximately 3 least significant bits. This corresponds to a 120µs, or a 1%, increase in the time-offlight preceding slot 1. If the increase were due to the blade starting farther to the left (figure 4) this shift would only translate to an encoder rotation of 0.051°. Figure 11 depicts the magnitude of this shift relative to the slot and sensor dimensions (assuming the increase in interpolated time is due to a change in encoder alignment). The errors were occurring while the interpolated time had only changed by a small fraction of that represented in figure 11. The small physical shifts represented by the interpolated time change Figure 11: Lost LED area must raise doubt that a misalignment is responsible for the corresponding to a 120µs errors. change in interpolated time. c) For encoder flight following the interpolated time measurement a series of point-to-point timing measurements are made and reported as speed flags. The speed flags for blade-A closing also show changes that correspond to the changes in interpolated time and can help to determine what is happening. Table 1 compares the average time-of-flight for each speed flag before the onset of errors—July 2000—and then during January 2001. Flags 1, 2, and 4 show a small increase in the average time-of-flight while flags 3 and 5 a small decrease. Careful examination reveals that the actual time-of-flights between each slot is not changing but the varying outputs reflect a shift in the clock pulses that are intercepted for each measurement. This data confirms that the time to travel to slot 1 has actually increased as indicated by the interpolated time. Furthermore the data shows that the time to travel between slots 1 and 6 is unchanged and therefore the angular velocity has not changed (within the 625µs resolution of the speed flags). This may imply that the detent to slot 1 distance is increasing. However, it must be noted that these are very small numbers and therefore the conclusions be considered with some skepticism. (1) One other interpretation of the delayed arrival at slot 1 is that the blade is actually moving slower early in flight then recovers. If so then the slowing must be due to phenomena localized to beginning of flight such as increased friction just in this region. 3. Blade Not in Detent a) The final possibility for mechanical fault is that the shutter blade does not always end up in the open detent. When the blade rests in the detent the position is reported as open and the exposure proceeds normally. On the occasion when the blade is out of the detent the encoder is partially blocking the sensor light path making the blade report closed and an error is generated. The blockage would have to be partial, not full, since the RAM patch has eliminated the errors. b) A pulse is used to drive the blade to the open position. The pulse remains on even after the stop is reached thereby "sticking" the blade to the end of travel. Even if the blade bounces against the stop the drive pulse will simply push it back to the end. Therefore the blade will be sitting still against the stop when the drive pulse is removed. The blade will stay in this position due to the attraction of a permanent magnet in the drive arm assembly. The position of the blade will not change in the absence of a disturbing force. c) Between the times that blade-A is established in the open detent and when the error in Table 1: Shutter-A closing speed flag averages during July 2000 and January 2001. Telemetry Flag 1 Flag 2 Flag 3 Flag 4 Flag 5 Resolution 625µs 625µs 625µs 625µs 625µs TInterpolated 39µs Average ∆t (ms) Jul '00 Jan '01 15.686 15.693 11.294 11.407 9.412 9.379 8.157 8.600 7.529 6.970 52.078 52.049 0.21183 0.35409 10 measured position occurs a disturbance force is always applied to the blade. This occurs when blade-B is closed and hits blade-A at the end of travel. As long as the blade-B closed detent position is resting against blade-A then the earlier logic applies here and the B drive pulse will force both blades against the stop. The blade-B closing pulse will remain on while both blades settle against the stops and blade-A will again be in the open detent. However, A. H. Johnston, JPL if the blade-B springs back after impact and comes to rest separated from blade-A then Figure 12: LED light output as a function of blade-A would have to rely on the proton fluence. Reference (viii). permanent magnet to be returned to the open detent. The latter is the case that occurs with the WFPC-1 shutter as recorded during testing at GSFC in fall of 2000. The video shows that the strength of the permanent magnet is well in excess of that needed to restore blade-A to the detent with a normal system. d) The force provided by the permanent magnet to pull the shutter into detent is much less than the solenoid drive force. It is possible that sticktion1 in blade-A near the detent could prevent the permanent magnet from restoring the detent position. e) Following an error the blades are moved in a default sequence that involves one blade pushing the other. Due to this the timing data that would indicate the blade position during an error is altered by the unusual shutter operation. f) The WFPC-1 shutter-A does get knocked out of the open detent when blade-B closes and it must rely on the permanent magnet to restore proper positioning. It is likely that the WFPC-2 shutter experiences the same situation and conceivable that frictional changes near the detent could cause shutter-A to stick outside of the detent. Coupled with the lack of position information this possibility has yet to be ruled out. D. Evaluate Electrical Degradation Performance degradation of two components on the shutter assembly could cause the shutter errors, the shutter-A position sensor Light Emitting Diode (LED) and phototransistor. Shutter errors will be generated if the coupling between the two devices is reduced to a level that the phototransistor does not "see" the light from the LED through encoder slot 0. This condition would occur if the LED light output power drops, the ability of the phototransistor to convert light to electrical signals is reduced, or a combination of the two. 1. LED Degraded Output a) There are numerous publications that show that exposure to ionizing radiation will cause a decrease in luminance from gallium arsenide (GaAs) infrared-emitting (IR) diodes. When discussing IR LEDs Siemensvii states that "A high degree of crystal perfection is a precondition for the creation of effectively radiant recombination as crystal defects act as centers for non-radiating recombination." Since ionizing radiation damage will generate such crystal defects this provides an explanation for the loss in light output. b) Figure 12, taken from a JPL reportviii on Figure 13: Trapped proton fluence at the radiation effects, shows LED output surface of HST. Reference (x). 1 Sticktion: A contemporary term that has many definitions in technical circles. In this report sticktion will be defined as a localized region of high friction in a mechanism. 11 degradation as a function of proton fluence for an OD880 GaAlAs device. The output drops sharply for the first 5x1010 neutrons/cm2 and then continues to degrade at a slower rate with continued exposure. This plot is useful in that the effects are shown for dosing with the part deenergized, as is the case for WFPC-2. According to the data the light output will halve for a proton dose of 2x1010 protons/cm2. This behavior is consistent with the results of radiation effects testingix undertaken by GSFC that show the output drops to half for 1x1011 neutrons/cm2 for an operating LED. c) The proton fluencex external to the spacecraft has been estimated for the MLI degradation investigation and is given in figure 13. From this graph the WFPC-2 dose, at the onset of shutter errors, was approximately 2x1010 protons/cm2 divided by the spacecraft shielding effect. Until shielding effects are calculated a dose of 1x1010 protons/cm2 will be assumed. At this dose the LED output can now be expected to be 75% of the launch luminance. 2. Phototransistor Degraded Sensitivity a) Phototransistor response to radiation damage has not been analyzed but can be assumed to be a reduction in sensitivity. 3. Combined Effects in Sensor Package a) Although the on-orbit response of the phototransistor has not been demonstrated, the ageing effect of the sensor pair will be a loss of coupling greater than the loss of LED output alone. For that reason the degradation of either component will be considered in the context of a loss of sensitivity of the sensor pair. Sensor package degradation is largely consistent with the symptoms of the shutter errors and has not been ruled out as the cause. E. Evaluate Hard Electrical Failure In the "Fault Isolation" section it was shown that the only electrical components that could fail in a hard manner and still cause the shutter errors are the two parallel resistors in the current path to the side-A Light Emitting Diode (LED), R105 and R5. 1. LED Drive Resistors a) The two resistors form a 79Ω resistance in series with a 2N2907A transistor and the TIL252 LED. The series circuit is across 0 to +5V with a 0.18V drop across the saturated transistorxi and 1.5V across the forward biased LED. Therefore the current is set by the remaining 3.32V through the 79Ω, or 42mA. The TIL25 datasheetxii provides that for this current the LED will be operating at 80% of the rated 0.75mW output power. If either resistor were to fail open the circuit resistance would double to 158Ω thereby reducing the current to 21mA and the output power to 34%. This value would be further reduced by the loss of output due to on-orbit radiation damage. b) There is insufficient information available to confirm if the system would operate with the LED these output levels. This problem could be ruled out with ground testing. Neutral density filters placed as interference in the WFPC-1 sensor assembly would indicate the reduction in photon coupling at which the sensor package would no longer work. IV. Remaining Faults A single cause of the errors has not been identified but the field of choices has been limited to the degradation of the two position-sensing components, a failed resistor, or an occasional offset in the blade-A position when open. Table 2 summarizes these findings and provides arguments for and against each possibility. It is known that reduced coupling in the sensor package is the source of the errors. The question that remains open is whether the reduced coupling is due to a less sensitive sensor package or is the encoder blade interfering with the light path. A. Evaluate Remaining Faults 1. Sensor Package Degradation a) The LED and phototransistor are known to be degrading and that the coupling between them is decreasing even if the light path is unchanged. It is not known if the degradation has reached 2 JPL drawing 10093548, "Shutter Optical Encoder," specifies a TIL24 LED. This is superceded by the WFPCII shutter encoder quality assurance document CL-SH-13-0011, "Assembly and Inspection Data Sheet," that reports TIL25 devices were used. 12 Failure LED and/or phototransistor degradation Pros Cons 1) Documentation supports this effect. 1) Sudden appearance of errors. 2) Ground testing supports this effect. 2) Would not explain timing trends. 3) The patch is working. 3) No long-term interpolated time drift. R105 or R5 open 1) Consistent with sudden onset of errors. 1) Unlikely failure mode. 2) Would not explain timing trends. Sticktion near blade-A 1) A is knocked out of detent by B 1) Small offsets have not shown up in open detent (causing an closing on WFPC-1. telemetry. occasional offset) 2) Due to blade gap on WFPC-1, B does not restore A to the detent. 3) Higher Tintpl TOF could be due to sticktion near the open-detent. Table 2: The remaining three problems that could generate the shutter errors with arguments for and against each being the culprit. a level sufficient to cause the shutter errors. Ground testing that varies the lamp intensity or reduces photon coupling using neutral density filters would help to make this determination. However, there remain several points that weigh in favor of this problem and some that weigh against. b) The best argument for this problem is that the lamp output is known to be decreasing. Also the fact that the November 2000 patch was designed to fix this problem and is working adds to the credibility. c) Working against the sensor degradation argument is the timing trends that coincide with the onset of the shutter errors. If sensor degradation is accepted as the fault then these trends cannot be related to the problem. It is possible to explain how degraded lamp output can cause a shift in interpolated time. However, this is a slow process that would have to cause a long-term trend beginning well before the onset of errors and could not generate a sudden knee in the data. It is certainly possible that the errors and the timing trends are unrelated but the coincidence shown by figure 10 is unsettling. d) A second argument against the lamp explanation is the suddenness with which the errors became present. Problems that develop due to a slow decline of some parameter usually have a slow onset then increase in frequency. To the contrary, the shutter errors developed rapidly in a period of weeks. The lamp effect may be subject to environmental influences such a change in temperature or a new unidentified noise source but no such correlation has been found. 2. Open Resistor in LED Drive Path a) The arguments for this failure are the same as for sensor package degradation. In addition this failure would be consistent with the rapid onset of errors. The previously described ground testing involving filters on the WFPC-1 shutter would also shed light on this type of failure. b) This type of failure does not explain the recent timing trends. One can concoct a scenario in which the resistor opens slowly or a resistor solder joint is slowly degrading but such a situation is somewhat farfetched and will not be considered here in any further detail. 3. Sticktion near Open Detent a) Surprisingly, this is the one failure that is consistent with all of the data. However difficult it is to believe that such a specific problem could be found through analysis the following arguments can be made. (1) Ground testing with the WFPC-1 shutter shows that blade-A bounces out of the open detent when blade-B closes. Once out of detent blade-B does not push blade-A back to the correct position due to a gap between the blades, the permanent magnets must apply the restoring force. (2) Increased friction near the start of flight would increase the time-of-flight to slot 1 but would have a negligible effect on the blade speed thereafter. This is consistent with the interpolated time and speed flag data. (3) This would cause a small binary effect on the open detent. There would be the normal detent and a second resting spot nearby that causes a partial blockage of the sensor light path. 13 Figure 14: Shutter-A and shutter-B interpolated times since launch. This could cause both the occasional shutter errors and then be corrected by the RAM patch the same as a dimming lamp. V. Monitoring & Risk Reduction Although a single fault for the shutter errors has not been identified there is enough understanding of the issues to make a knowledgeable assessment of the current risks. Analysis of the available data continues in an effort to assess any ongoing risk to the instrument. The electrical possibilities pose no threat to the health of the instrument however operations could again be impacted. If the sensors are losing sensitivity the A-blade may degrade to the point to overcome the software patch benefits and the errors would reoccur. Furthermore, if the A-side sensors have experienced this condition the B-side sensors can be expected to have similar problems in the future. Although there has been speculation that LED degradation could be detected in the timing data, no such results have been proven to date. Currently there exist no monitors of the sensor package health that have been shown to provide early warning of a problem or predict if and when another problem will occur. If a mechanical problem is responsible for the errors there is still a possibility of a health risk but currently the risk appears to be small. Unlike the electrical issues there is very good telemetry in the form of timing data that can be used to monitor the shutter's mechanical health. Regular monitoring of this and other data has commenced and will continue while the WFPC-2 operates. The usefulness of the data to predict problems has yet to be proven but as an analysis aid the information content is excellent. Software has been developed to allow regular monitoring of the timing data. Following further refinement the code it will be incorporated into the STScI Telemetry Evaluation System (TES). VI. Future Efforts 1. Further analysis may reveal offsets in the open detent that can be measured as a result of the RAM patch. Such offsets would reduce the interpolated time and would now be reported in the absence of errors. 2. Neutral density filter testing of the WFPC-1 shutter would provide insight into the electrical issues. 14 3. The shutter speed history could be constructed using the methods that created figure 5. 4. Improve the radiation dose calculations to better predict LED degradation. VII. Conclusions The source of the error is reduced coupling in the blade-A position sensor. Depicted in figure 15, the issue is LED/phototransistor pair decreased output and sensitivity or an encoder blade offset that interferes with the light path. Remaining are two possibilities that could cause the LED to dim and one problem that could cause the offset. The analysis to date shows minimal risks to the WFPC-2 instrument. Close monitoring of the shutter timing data will continue while the instrument remains on orbit. Automation of the data analysis provides a wealth of insight into the health of the shutter and should offer warning of mechanical degradation. This report is not the final word on the shutter errors. The effort will continue at STScI to refine the analyses of this problem and provide more conclusive evidence. Others are encouraged to view these findings with a critical eye and to explore any possibilities that have been overlooked. The discussion has often returned to the trends in interpolated time. In closing I will put the amplitude of the recent trends into perspective by comparing it to the history of the other three interpolated times: shutter-A open, shutter-B open, and shutter-B close. The shutter-A closing is shown as the bottom (blue) trace in figure 14. Clearly it has been the best-behaved interpolated time over the life of the mission. Hopefully this can be used to allay fears of a major mechanical failure. Appendix A provides further information on the interpolated time variance calculations. Appendix B contains interpolated time and speed flag history plots. VIII. Acknowledgements This report reflects the work of the entire STScI Engineering Team. The progress is a credit to Colleen Townsley's ability to find the manpower with limited staffing. The circuit analysis by Tom Wheeler has been invaluable. The early insight provided by Tom Bickler of JPL and the Anomaly Review Board brought the problem into focus and rapidly provided a repair to restore the integrity of the science program. Thank you to the GSFC mechanisms group for providing video documentation of the WFPC-1 testing. Figure 15: Source of shutter errors: dimmed lamp or blocked light path? 15 IX. Appendix A: Interpolated Time Variance A. Description 1. Data Manipulation a) Data since launch is segregated into streams of interpolated time. Each interpolated time is then separated by open or close, and AP17 or microprocessor commanding. The software determines the type of move based upon patterns of independent telemetry. The output is then filtered to remove unwanted data such as repeated records. b) The interpolated times for each blade open and close are then considered. Since the starting point of the timer is unknown the time-of-flight is not represented by interpolated time. To correct for this the data is biased to a lifetime average flight of 11.7ms. This value represents the blade-A closing time-of-flight in the interpolated region as found on the chart below. c) Dividing each point by a 30-point moving average then normalizes the biased data. d) A 30-point moving variance is then taken from the normalized data. 2. Plots a) The following plots are provided in this appendix: (1) Top/Left: Interpolated time. (2) Bottom/Left: The variance of the interpolated time. (3) Top/Right: Biased and normalized interpolated time. (4) Bottom/Right: The variance of the biased and normalized interpolated time ii iii iv X. Appendix B: Interpolated and Speed Flag Timing A. Description 1. Data Manipulation a) Data since launch is segregated into streams of interpolated time. Each interpolated time is then separated by open or close, and AP17 or microprocessor commanding. The software determines the type of move based upon patterns of independent telemetry. The output is then filtered to remove unwanted data such as repeated records. 2. Plots a) The red triangle on each plot indicates the first occurrence of the shutter errors. b) All plots encompass the period from February 1997 to February 2001. c) All plots use equal ranges for the y-axis scale except for U170 uProc close. v Blade-A Microprocessor Controlled Closing vi Blade-A AP17 Controlled Opening vii Blade-A Microprocessor Controlled Opening viii Blade-B AP17 Controlled Closing ix Blade-B Microprocessor Controlled Closing x Blade-B AP17 Controlled Opening xi Blade-B Microprocessor Controlled Opening xii XI. References i Ken Stowers, "Appendix VI to the SI to SIC&DH Interface Control Document," ST ICD-08 revision A, July 1993, table VI-3. ii Jet Propulsion Laboratory, "Wide Field Planetary Camera II Instrument Description and User Handbook," JPL D-11212, Appendix C, December 1993, California Institute of Technology. iii Jet Propulsion Laboratory, "Schematic Diagram, Mechanical Drivers," WFPC-II schematic number 10093516 revision H, 1980. iv Jet Propulsion Laboratory, "Shutter Assembly," WFPC-II schematic number 10093548 revision D, 1980. v Jet Propulsion Laboratory, "Schematic Diagram, Logic Circuit Boards," WFPC-II schematic number 10093506 revision D, 1983. vi Jet Propulsion Laboratory, "Disc, Encoder," WFPC-II schematic number 10085196, revision B, 1980. vii Siemens, "General IR and Photodetector Information," Appnote 37, http://webook.fset.de/20091999PHCHO/OPTOELECTRONICA/app37_1.pdf viii A. Johnston, "Recent Work on Radiation Effects in Microelectronics at JPL," http://rd49.web.cern.ch/RD49/RD49News/Allan_Johnston.pdf ix R. Reed, K. LaBel, H. Kim, H. Leidecker, and J. Lohr; "Test Report of Proton and Neutron Exposures of Devices that Utilize Optical Components and are Contained in the CIRS Instrument;" http://radhome.gsfc.nasa.gov/radhome/papers/i090397.html x J. Dever, K. de Groh, B. Banks, J. Townsend, J. Barth, S. Thomson, T. Gregory, and W. Savage "Environmental Exposure Conditions for Teflon FEP on the Hubble Space Telescope Investigated," http://www.grc.nasa.gov/WWW/RT1999/5000/5480dever2.html xi Semicona Semiconductors, "Type 2N2907A," data sheet number 2N2907A. xii Texas Instruments, "Types TIL23, TIL24, TIL25 P-N Gallium Arsenide Infrared-Emitting Diodes," bulletin No. DL-S 11312, February 1970—Revised January 1976. xiii