Android Media Player Development
Suyash Gupta
Android Developer
Talentica Software
Agenda
1.Introduction
2.Characteristics of Video streams
3.Android Media Player Development
4.Advanced Media Player Development
1. Introduction
• Video - Most Popular and common way of entertainment.
• Boom in mobile market in past 5 years.
• 3g services revolutionized the telecommunication industry.
1) Video - Journey So Far
●
●
Video technology was first developed for (CRT) televisions.
Video recorders were sold for $50,000 in 1956, and videotapes
cost $300 per one-hour reel.
●
Sony introduced VCR in 1971.
●
Then came CD's in 1997.
●
Followed by blue ray disc in 2006.
●
Computers to capture, store, edit and transmit video clips.
●
Smartphones users grew to a increase to a limit of 80%.
●
Video watching became very user friendly.
Note: In Real-time entertainment the major share is made by Youtube, Netflix and Pandora.
2. Characteristics of video streams
a) Frame rate
b) Interlaced VS progressive
c) Aspect ratio
d) Color depth
e) Digital video
f) Video compression method (digital only)
g) Containers for Codecs
a) Frame Rate
The number of still pictures per unit of time of video.
a) Frame Rate
• To identify each frame individually min 10-12 FPS.
• Early silent films had a frame rate from 14 to 24 FPS but it
was perceived as jerky motion.
• There are three main frame rate standards in
the TV and digital cinema business: 24p, 25p, and 30p.
• 48p is a progressive format and is currently being trailed
in the film industry.(The Hobbit)
b) Video can be interlaced or progressive.
Interlaced video:
●
●
●
●
●
Every frame has horizontal lines.
Each frame is divided into 2 fields.
Odd field and even field.
Interlacing is a technique of doubling the FR
within the limitations of a narrow bandwidth.
Image Detail is sacrificed.
b) Interlaced and progressive frames
Odd Field
Even Field
b) Interlaced video fields
b) Interlace video Frame
b) Progressive video
●
●
●
●
Progressive is a method for displaying, storing or
transmitting moving images.
Each Frame resembles an image.
Progressive scan is used in most CRTs used as
computer Monitors.
It is also becoming increasingly common in highend Television equipment, which is often capable
of performing de-interlacing so that Interlaced
video can still be viewed.
b) Progressive video Frame
b) Progressive Video
Advantages of progressive scan include:
1. Better quality of video.
2. No flickering of narrow horizontal patterns.
3. Simpler video processing equipment.
4. Easier compression than interlaced.
c) Aspect ratio
●
●
●
●
Aspect ratio describes the dimensions of
video screens.
Traditional television screen is 4:3.
High definition televisions(720p and 1080p)
16:9.
AR 1.85:1 and 35 mm US and UK widescreen
standard for theatrical film.
c) Some common aspect ratios
d) Color depth
The number of bits used to indicate the color of a
single pixel, in a video frame buffer.
Bit is a storage value of a pixel.
1-bit color (2^1 = 2 colors): monochrome, black and
white
2-bit color (2^2 = 4 colors): CGA, gray-scale early
NeXTstation, color Macintoshes ect.
3-bit color (2^3 = 8 colors)
d) Color depth
6-bit color (2^6 = 64 colors):
8-bit color (2^8 = 256 colors): most early color Unix
workstations, VGA at low resolution, video games.
16-bit color (2^16 = 32768 colors): some Silicon
Graphics systems
True color supports 24-bit for three RGB colors.
d) Color depths examples
1 bit (2 colors)
2 bits (4 colors)
4 bits (16 colors)
d) Color Depths
8 bits (256 colors)
24 bits (16,777,216
colors, "truecolor")
e) Digital Video
●
●
Analog video stores video signal on magnetic tape.
When copies are made of an Analog signal, degradation is
introduced due to loss of video signal in the copying.
●
Whereas in digital data every bit has a binary value.
●
So copying is 100% precise.
Analog solution to video recording
Digital video storage medium
e) Size of raw digital video
Digital Video in its raw form can be huge.
An example video can have a duration of (3600sec), a
frame size of 640x480 (WxH) at a color depth of
24bits and a frame rate of 25fps.
●
●
●
●
pixels per frame = 640 * 480 = 307,200
bits per frame = 307,200 * 24 = 7,372,800 =
7.37Mbits
bit rate (BR) = 7.37 * 25 = 184.25Mbits/sec
video size (VS) = 184Mbits/sec * 3600sec =
662,400Mbits = 82,800Mbytes = 82.8Gbytes
e) Issues
Leads to two interrelated problems:
1) Large storage requirements.
2) High Bandwidth requirement during playback and
recording.
f) Video Compression
1. Conversion of data to a format that requires fewer
bits.
2. The inverse process is known as decompression.
3.Decompression may or may not yield an exact copy
of the original data.
f) Video Codec
A codec is a compression algorithm, used to reduce
the size of a stream. The compression is
usually lossy.
•
•
Lossless compression: It is a class of data
compression algorithms that allows the original
data to be perfectly reconstructed from the
compressed data.
“Lossy" compression is a data encoding method
that compresses data by discarding (losing) some
of it.
f) Comparison of lossy compression
80%
compression
90%
compression
96%
compression
f) Different Codecs
MPEG-1(1993)(Moving Pictures Expert Group)
[Old, supported by everything (at least up to 352x240)]
MPEG-2 codecs
A good quality of compression. This is what is used for DVD.
MPEG-4 (1998)AVC (DivX codec)
A good quality compression with maintaining the quality of video.
H.263/MPEG-4 Part 2 codecs
DivX Pro Codec: Compresses video down to very small file sizesexcellent for web use but loses quality.
Thats why not used for the television purpose
Xvid: Free/open-source implementation of MPEG-4 ASP, originally
based on the OpenDivX project. Similar as DivX Pro Codec.
FFmpeg MPEG-4: open-source libavcodec codec library
Used by default for decoding or encoding in many open-source
video players such as MXPlayer, VLC, MPlayer and GStreamer.
WMV (Windows Media Video): Microsoft's family of proprietary
video codec designs including WMV 7, WMV 8, and WMV 9.
Google (On2) codecs:
VP6, VP6-E, VP6-S, VP7, VP8 VP9 : Proprietary high definition
video compression formats and codecs developed
by On2 Technologies
g) Containers for Codecs
The container describes the structure of the file:
where the various pieces are stored and how they
are made.
I) AVI (standard Microsoft Win container)
(mostly used in digital cameras)
II) MOV (standard QuickTime container)
Carries the widest range of codecs
III) MPEG-2 TS (acronym of Transport Stream,
standard container for digital broadcasting)
g) Containers for Codecs
IV) MP4 (standard container for the MPEG-4
multimedia portfolio)
V) RealMedia (standard container for RealVideo and
RealAudio)
VI) 3gp (used by many mobile phones)
3. Android Media Player
a)Android supported formats
b)Media player API and Sample Code
c)Limitations of Media Player
3. a) Android supported formats
Type
Format /
Codec
Vide
o
H.263
H.264 AVC
Encoder
Decoder
Supported File
Type(s) /
Container
Formats
•
•
• 3GPP (.3gp)
• MPEG-4 (.mp4)
•
(Android 3.0+)
•
Baseline Profile • 3GPP (.3gp)
(BP)
• MPEG-4 (.mp4)
• MPEG-TS (.ts,
AAC audio only,
not seekable,
Android 3.0+)
MPEG-4 SP
VP8
Details
•
•
(Android 4.3+)
3GPP (.3gp)
•
Streamable
• WebM (.webm
(Android 2.3.3+) only in Android )
4.0 and above • Matroska
(.mkv, Android
4.0+)
3. b) Media Player API
Audio / Video
Playback
Record
android.media.MediaPlayer
android.media.MediaRecorder
b) Playback: MediaPlayer Basics
Play Media From:
• Resource Folder
• File System Path
• URL
Basically
• Create new instance of MediaPlayer
• Call prepare()
• Call start()
b) Playback View
Playing a video File
b) Layout View
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:id="@+id/RelativeLayout1"
android:layout_width="match_parent"
android:layout_height="fill_parent"
android:orientation="horizontal" >
<SurfaceView
android:id="@+id/surfaceView1"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_above="@+id/seekBar1"
android:layout_alignParentTop="true"
android:layout_weight="0.67" />
</RelativeLayout>
b) MediaPlayer Class
Create Using:
Get:
• URI
• Resource ID
• Position
• Duration
• isPlaying
Set:
Output Display
Looping
•
Data Source
•
Wake Mode
States:
•
•
•
•
•
•
•
•
Start
Prepare
Pause
Release
Reset
SeekTo
Stop
Error
Inherits from
java.lang.Object
Listeners:
•
Buffering Update
•
Completion
•
Error
•
When Prepared
b) Activity Code
public class MainActivity extends Activity implements SurfaceHolder.Callback,
OnPreparedListener, OnErrorListener, OnInfoListener {
MediaPlayer mediaPlayer;
SurfaceHolder surfaceHolder;
SurfaceView playerSurfaceView;
String videoSrc = "rtsp://v6.cache1.c.youtube.com/CjYLENy73wIaLQkDsLHya4Z9hMYDSANFEIJbXYtZ29vZ2xlSARSBXdhdGNoYKX4k4uBjbOiUQw=/0/0/0/video.3gp";
//String dataSrc = "http://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4";
//int resId = R.raw.sample;//.mp4
//int String dataSrc = "/sdcard/music/sample_music.mp3";
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main2);
playerSurfaceView = (SurfaceView) findViewById(R.id.playersurface);
surfaceHolder = playerSurfaceView.getHolder();
surfaceHolder.addCallback(this);
}
b) Activity Code
@Override
public void surfaceCreated(SurfaceHolder arg0) {
try {
mediaPlayer = new MediaPlayer();
mediaPlayer.setDisplay(surfaceHolder);
mediaPlayer.setDataSource(videoSrc);
mediaPlayer.setOnPreparedListener(this);
mediaPlayer.setOnErrorListener(this);
mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
mediaPlayer.prepare();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (SecurityException e) {
e.printStackTrace();
} catch (IllegalStateException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
b) Activity Code
@Override
public void onPrepared(MediaPlayer mp) {
mediaPlayer.start();
}
@Override
public boolean onError(MediaPlayer mp, int what, int extra) {
Log.e("MainActivity", "what="+what+" extra="+extra);
return false;
}
@Override
public boolean onInfo(MediaPlayer mp, int what, int extra) {
Log.i("MainActivity", "what="+what+" extra="+extra);
return false;
}
Media Player State Diagram
c) Limitations of Media Player
• Any time media player changes state, check before selecting any
action or you may fall into IllegalStateException.
• Error when the orientation of screen is change, even if you manage
the tag screenOrientation=“orientation” in manifest.xml
• When activity goes in background and MediaPlayer is playing, even if
you have handled onPause and onResume in some devices it goes to
error state.
• There are 30 to 34 types of possible error starting from -1 to -34.
For e.g. Error: the video container is not valid for progressive
playback (-32);
Error due to resource being in wrong state to handle request (-14)
Error due to general data processing(-18)
c) Limitations of Media Player
• public static void playMusic(int id) { // Not a good practice
MediaPlayer mediaPlayer = MediaPlayer.create(context, id);
mediaPlayer.setLooping(true); mediaPlayer.start();
} // Here the media player will be removed by GC even if it is in play
mode.
• The error value (1, -2147483648), the '1' value corresponds to the
constant in MediaPlayer.MEDIA_ERROR_UNKNOWN. (This might
happen)
-2147483648 corresponds to hexadecimal 0x80000000 which is
defined as UNKNOWN_ERROR
4. Advanced Media Player Development
a) Customization Features
a) Benefits of Customization
a) Drawbacks
a) FFMPEG Library
a) Customization Features
We can Customize the Media Player and achieve :
We can enhance security.
a) Playing video in chunks
b) Prevent video from getting stolen.
c) Encrypt the InputStream with Cipher.
•Real Time manipulation.(Play ads in between)
Adaptive Streaming.(Configurable bandwidth)
•Multi Format Playing.(discussed later)
b) Benefits of Customization
• Your video content is secure.
• Easily applicable to other java platforms too for e.g.
J2ME
• Saved video can be played only by our media player.
• Configurable bandwidth according to network
speed.
• Since playback is chunk by chunk you can show
video ads in between them too.
• Event handling during playback is possible for e.g. at
what point the user stops the video or how much
percentage watched.
c) Drawbacks
• Decoding of Android media player is bad, may fall
into lot of errors and you might not get possible
solution anywhere.(Limited number of file
formats)
• The total chunk playback duration and the clip
playback duration should match or the player will
again throw errors like ChunkNotFound (Custom
exception).
d) FFMPEG Library
• Almost every android phone user is friendly with MX Player
which is a pretty famous player.
• MX Player is build using FFMpeg library which we will discuss
later.
• It is user-friendly, plays almost all formats and possesses great
build-in features.
d)Features of mx player
• Plays almost every movie files including .3gp .avi .divx
.f4v .flv .mkv .mp4 .mpeg .mov .vob .wmv .webm .xvid
and many more.
• Hardware decoding (with h/w+ decoder)
• Processor optimization
• High speed rendering for ARM® NEON™ compliant
processors.
• Provides processor specific Codecs including Tegra 2
processor.
• Move forward/backward (Optional)
• Aspect ratio selection.
How to build FFmpeg for Android
Following conditions are required to run included shell script:
•Linux or Cygwin.
•Latest Google NDK.
Download ffmpeg source code
1)In config-ffmpeg.sh, change "NDK" variable to point NDK path.
2)Add this line NDK=$HOME/Desktop/adt/android-ndk-r9
3)In build.sh, change "MAKE" variable to point ndk-build script of NDK directory.
4) Run the shell scripts by command ./build.sh
5) The build can take a while to finish depending on your computer speed.
6) Make ffmpeg Libraries available for Your Projects
FFFMPEG Project
Live Demo
Thank You
Questions??
References
Video Characteristics:
http://en.wikipedia.org/wiki/Video
Android Developer Site:
http://developer.android.com/reference/android/media/MediaPlayer.html
FFMPEG Build:
http://www.roman10.net/how-to-build-ffmpeg-with-ndk-r9/