Bits is Bits? Right?
Check Again.
George Blood, Safe Sound Archive
Joint Technical Symposium
Oslo, Norway
Monday, May 3, 2010
Typical Playback Chain
Analog source
Analog-to-digital converter
Digital Audio Workstation
Digital File
Physical Carrier of Files
Typical Playback Chain
Analog source
Analog-to-digital converter
Digital Audio Workstation
Digital File
Physical Carrier of Files
}
From ADC to Digital File
ADC Digital Output (AES-3)
Cable/patching/switching
Digital Audio Workstation:
Interface: AES-3 to FW/USB/etc.
Driver
Computer: CPU/Bus/etc.
Software
OS/Drivers
Storage interface/drivers: FW, USB, SATA, FC, iSCSI
Storage Device (HDD)
Digital File
Disappointments
• Don’t mention brands
• Don’t tell you our solution
• Very pervasive problem
• http://www.avpreserve.com/wpcontent/uploads/2010/01/Digital_Audio_Inter
stitial_Errors.pdf
Disappointments
• Planned to announce a software
tool to automate this testing….
Tested Hardware
•
•
•
•
•
•
Lynx AES16
M-Audio (96 and 192)
M-Box
Metric Halo ULN-2
Nagra V
SonicStudio 302 (aka ULN2+DSP)
Tested Software
•
•
•
•
•
•
•
Boom
Metric Halo Console
ProTools LE (Mac)
Sonic Solutions HDSP
Sonic Studio soundBlade
SoundForge
WaveLab
Tested OS
•
•
•
•
•
Mac OS9
Mac OS X
Windows 98
Windows Home
Windows ME
Tested Drivers
• Stock drivers (mostly)
– OS stock drivers
– Hardware bundled drivers
• Dedicated Drivers
– OS 9 (Sonic Solutions)
– OS X (SonicStudio & MH)
Tested Storage
•
•
•
•
•
•
Fat32
FibreChannel (2Gb, RAID5)
FireWire 400
HFS+ (SCSI Ultra160)
HFS-extended
SATA
Not Tested
•
•
•
•
•
•
•
•
•
Audacity
AudioCube
[Bias Peak]
iSCSI
ProTools HD
ProTools LE (PC)
Pyramix
RME
ADC to FireWire
Testing Multiple
Components
Simultaneously
•
•
•
•
•
•
Ab1
Ab2
Bb1
Bb2
Aa1
Aa2
Finding errors
• 100% Listen
– Inefficient
– Unreliable
• Software verification
– Doesn’t certify authenticity
– Limited parameter testing
– Unreliable
Finding Errors
• Null Test:
– 2 presumed identical files
– Sample-accurate alignment
– Invert polarity
– Add together
• If a1=a2, then a1 - a2=0
Aligning 2 stereo pairs
(1)
Aligning 2 stereo pairs
(2)
Aligning 2 stereo pairs
(3)
Aligning 2 stereo pairs
(4)
Aligning 2 stereo pairs
(5)
Aligning 2 stereo pairs
(6)
…then invert 1 pair
Testing for 2 parameters
• Time
– Number of samples
• Amplitude
– Number of bits
Null if mis-aligned by
1 sample
Error of 76 samples
dropped
from: http://www.avpreserve.com/wpcontent/uploads/2010/01/Digital_Audio_Interstitial_Errors.pdf
Bits vs. samples missing
• 1010101010101010
• 1010101011101010
Bits vs. samples missing
• 1010101010101010
• 1010101011101010
Null if gain changed
What happened?
• New Hardware
– Failure
• Reviewed set up with engineer
– Failure
• Reviewed old systems
– Failure
• Simplified Test
– Finally some passes
– Failures continue between identically
configured systems
Qualifying a Control
• Round Robin Test
– A≠B
– B≠C
– A=C
– …B is wrong
Qualifying a Control
• Assumes only one variable at
play
– Host
– I/O Hardware
– Driver
– Application
…test configuration
Hardware in hand
• SonicStudio
–
–
–
–
–
Great editor
Semi-proprietary file types
Non-destructive editing
No support for WAV or BWF output
No AES-31 ADL support
• Nagra V
– Simple
– Self-contained
– Can’t change variables
• (other than resolution)
Results
•
•
•
•
Null tests passed
Every time
Always
Validates testing procedures
Talent Pool
• Head of QA with physics
background
• 2 experienced audio engineers
• 2 dealers
• 2 outside technical engineers
Tech “support”
•
•
•
•
•
•
Universally appalling
Short attention span
Don’t understand concepts
Always some other link in chain
Regional sales reps care more
Engineer curious but unable to help
Computer code “traps”
•
•
•
•
Intractable problem buried in code
Build routine to watch for it
Applies fix
Efficient solution
Computer code “traps”
• Masking (aka “hiding”) problem!
– Little crossfades at glitches
– Other lies…
True Nulls
Very small error in Null
Very small error in Null
Close up
Incomplete Null
Incomplete Null
Non-zero offset
Calculating size of error
• “Normalize” difference signal
– Increases gain to zero headroom
• 1 bit = ~ 6dB
• Normalizing amplitude
divided by 6dB
equals # of bits (amplitude) of difference…
Calculating size of error
• 90dB / 6 dB = 15 bits
Incomplete Null
Spectrum of difference signal
Conclusion
• Not passing 24 bits
• Truncating at 16 bits
• Dithering 16th - 24th bits
– More noise but Lower distortion
than truncating at 16 bits and
dithering 17-24
Struggling to keep up
Different bit depth, all non-zero
Example 3 drops samples
Struggling to keep up
Example 1 & 2 Normalized -- “properly” dithered
“Well-dithered lie”
Toggling single bit
Are there no limits?
“Fatter” difference on L
Display higher resolution on R
Are there no limits?
“Fatter” difference on L
“Full” null - absurd gain change
- on R
Are there no limits?
Different residual on L!
More absurdities
• Same hardware, different ages,
different results
• Same configuration, different
results on different days
• Same hardware, good results
with one software; poor results
with different software
Conclusions
• Get what you pay for
– More expensive hardware & software performs better in
Null tests
• Stock drivers aren’t as good as hardware specific
drivers
– keep up to date, especially after OS upgrades
• Non-host-based systems perform significantly better
• Mac-based systems performed better than PC-based
systems, but are far from flawless
• Worst PC and Worst Mac systems equally bad
• These problems are widely known by manufacturers
and blatantly hidden