Spark Streaming Setup Notes
Doug Chang
References:
https://github.com/holdenk/elasticsearchspark/blob/master/src/main/scala/com/holdenkarau/es
spark/IndexTweetsLive.scala
Goal: create a spark streaming program using the Twitter feed.
This is one use case where Spark did a better job than Hadoop in packaging the components for
data acquisition. Subscribe to the twitter developer api and show a stream of data being processsed
by spark.
Debug Step 1: verify you can subscribe to the twitter feed. The developer API and subsequent
websearches aren't that clear on how to sign up to get both a consumer key/secret and token
key/secret. This is the base starting point for the spark tutorials.
Download the Twitter4j api or create a maven project with the twitter 4j jars. There are different
versions of the api where the more recent versions have refactored the 4j jar names.
Do not use my keys. I changed the keys below so nobody can copy them and get me introuble with
Twitter.
package com.example;
import twitter4j.StallWarning;
import twitter4j.Status;
import twitter4j.StatusDeletionNotice;
import twitter4j.StatusListener;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.TwitterStream;
import twitter4j.TwitterStreamFactory;
import twitter4j.auth.Authorization;
import twitter4j.conf.Configuration;
import twitter4j.conf.ConfigurationBuilder;
public class TestTwitterLogin {
public static void main(String []args) throws IllegalStateException, TwitterException{
System.setProperty("twitter4j.oauth.consumerKey", "eFIaiOuxsny01VVQ2QWasdf");
System.setProperty("twitter4j.oauth.consumerSecret",
"gDQI5EiCMJJaaNI8XVNhfZXwuCOYfeJ3XsOUNHvsXqgq0asdf");
System.setProperty("twitter4j.oauth.accessToken", "76976448-Otz8w4yMKx6yCEWTH3dNTfuF8LYeLgqdoDrasdf");
System.setProperty("twitter4j.oauth.accessTokenSecret",
"NFPFe2EzuKWuzRKmY1RENUBfQzGeGbAS1JzjX3asdf");
TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
StatusListener listener = new StatusListener() {
@Override
public void onStatus(Status status) {
System.out.println("----------STATUS---------");
System.out.println("@" + status.getUser().getScreenName() + " - " + status.getText());
}
@Override
public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
System.out.println("----------ONDELETIONNOTICE---------");
System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
}
@Override
public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
System.out.println("----------LIMITATION---------");
System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
}
@Override
public void onScrubGeo(long userId, long upToStatusId) {
System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
}
public void onStallWarning(StallWarning warning) {
System.out.println("Got stall warning:" + warning);
}
@Override
public void onException(Exception ex) {
ex.printStackTrace();
}
};
twitterStream.addListener(listener);
twitterStream.sample();
}
}
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>TestJavaTwitter4J</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-stream</artifactId>
<version>4.0.1</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-examples</artifactId>
<version>4.0.1</version>
</dependency>
</dependencies>
</project>
Verify you can get a twitter printout to prove your api subscription is working:
Debug Step #2a: Verify you can create a simple spark program using SBT. We will use this to
verify we can print out the twitter stream. Use as a starting point the documentation on the spark
website for creating simple.sbt: http://spark.apache.org/docs/0.9.1/quick-start.html#astandalone-app-in-scala
dc@localhost spark]$ cat simple.sbt
name := "Simple Project"
version := "1.0"
scalaVersion:="2.10.3"
libraryDependencies += "org.apache.spark" %% "spark-core" % "0.9.1"
resolvers += "Akka Repository" at "http://repo.akka.io/releases"
libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.3.0"
[dc@localhost spark]$ ls src/main/scala/SimpleApp.scala
src/main/scala/SimpleApp.scala
[dc@localhost spark]$ cat src/main/scala/SimpleApp.scala
/*** SimpleApp.scala ***/
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
object SimpleApp {
def main(args: Array[String]) {
val logFile = "/usr/lib/spark/README.md" // Should be some file on your system
val sc = new SparkContext("local", "Simple App", "/usr/lib/spark",
List("target/scala-2.10/simple-project_2.10-1.0.jar"))
val logData = sc.textFile(logFile, 2).cache()
val numAs = logData.filter(line => line.contains("a")).count()
val numBs = logData.filter(line => line.contains("b")).count()
println("Lines with a: %s, Lines with b: %s".format(numAs, numBs))
}
}
Output:
If you are using the bigtop distribution then you have to add sudo.
Run:
>sudo sbt/sbt run
[dc@localhost spark]$ sudo sbt/sbt run
Launching sbt from sbt/sbt-launch-0.12.4.jar
[info] Loading project definition from /usr/lib/spark/project
[info] Set current project to Simple Project (in build file:/usr/lib/spark/)
[info] Running SimpleApp
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/spark/lib/slf4j-log4j121.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j121.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/06/24 13:53:11 INFO Utils: Using Spark's default log4j profile: org/apache/spark/log4jdefaults.properties
14/06/24 13:53:11 WARN Utils: Your hostname, localhost.localdomain resolves to a loopback
address: 127.0.0.1; using 192.168.171.1 instead (on interface vmnet8)
14/06/24 13:53:11 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
14/06/24 13:53:12 INFO Slf4jLogger: Slf4jLogger started
14/06/24 13:53:12 INFO Remoting: Starting remoting
14/06/24 13:53:12 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://spark@192.168.171.1:45736]
14/06/24 13:53:12 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://spark@192.168.171.1:45736]
14/06/24 13:53:12 INFO SparkEnv: Registering BlockManagerMaster
14/06/24 13:53:13 INFO DiskBlockManager: Created local directory at /tmp/spark-local20140624135313-32ec
14/06/24 13:53:13 INFO MemoryStore: MemoryStore started with capacity 640.0 MB.
14/06/24 13:53:13 INFO ConnectionManager: Bound socket to port 34930 with id =
ConnectionManagerId(192.168.171.1,34930)
14/06/24 13:53:13 INFO BlockManagerMaster: Trying to register BlockManager
14/06/24 13:53:13 INFO BlockManagerMasterActor$BlockManagerInfo: Registering block manager
192.168.171.1:34930 with 640.0 MB RAM
14/06/24 13:53:13 INFO BlockManagerMaster: Registered BlockManager
14/06/24 13:53:13 INFO HttpServer: Starting HTTP Server
14/06/24 13:53:13 INFO HttpBroadcast: Broadcast server started at http://192.168.171.1:45476
14/06/24 13:53:13 INFO SparkEnv: Registering MapOutputTracker
14/06/24 13:53:13 INFO HttpFileServer: HTTP File server directory is /tmp/spark-06c01edada6d-4dd3-af93-3e441204cea0
14/06/24 13:53:13 INFO HttpServer: Starting HTTP Server
14/06/24 13:53:13 INFO SparkUI: Started Spark Web UI at http://192.168.171.1:4040
14/06/24 13:53:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
14/06/24 13:53:14 INFO SparkContext: Added JAR target/scala-2.10/simple-project_2.10-1.0.jar
at http://192.168.171.1:39534/jars/simple-project_2.10-1.0.jar with timestamp 1403643194173
14/06/24 13:53:14 INFO MemoryStore: ensureFreeSpace(152694) called with curMem=0,
maxMem=671101747
14/06/24 13:53:14 INFO MemoryStore: Block broadcast_0 stored as values to memory (estimated
size 149.1 KB, free 639.9 MB)
14/06/24 13:53:14 INFO FileInputFormat: Total input paths to process : 1
14/06/24 13:53:14 INFO SparkContext: Starting job: count at SimpleApp.scala:11
14/06/24 13:53:14 INFO DAGScheduler: Got job 0 (count at SimpleApp.scala:11) with 2 output
partitions (allowLocal=false)
14/06/24 13:53:14 INFO DAGScheduler: Final stage: Stage 0 (count at SimpleApp.scala:11)
14/06/24 13:53:14 INFO DAGScheduler: Parents of final stage: List()
14/06/24 13:53:14 INFO DAGScheduler: Missing parents: List()
14/06/24 13:53:14 INFO DAGScheduler: Submitting Stage 0 (FilteredRDD[2] at filter at
SimpleApp.scala:11), which has no missing parents
14/06/24 13:53:14 INFO DAGScheduler: Submitting 2 missing tasks from Stage 0 (FilteredRDD[2]
at filter at SimpleApp.scala:11)
14/06/24 13:53:14 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks
14/06/24 13:53:14 INFO TaskSetManager: Starting task 0.0:0 as TID 0 on executor localhost:
localhost (PROCESS_LOCAL)
14/06/24 13:53:14 INFO TaskSetManager: Serialized task 0.0:0 as 1690 bytes in 6 ms
14/06/24 13:53:14 INFO Executor: Running task ID 0
14/06/24 13:53:14 INFO Executor: Fetching http://192.168.171.1:39534/jars/simpleproject_2.10-1.0.jar with timestamp 1403643194173
14/06/24 13:53:14 INFO Utils: Fetching http://192.168.171.1:39534/jars/simple-project_2.101.0.jar to /tmp/fetchFileTemp8920244719454374851.tmp
14/06/24 13:53:15 INFO Executor: Adding file:/tmp/spark-14342d4f-45d7-41ce-a90261dbb6983db3/simple-project_2.10-1.0.jar to class loader
14/06/24 13:53:15 INFO BlockManager: Found block broadcast_0 locally
14/06/24 13:53:15 INFO CacheManager: Partition rdd_1_0 not found, computing it
14/06/24 13:53:15 INFO HadoopRDD: Input split: file:/usr/lib/spark/README.md:0+51
14/06/24 13:53:15 INFO MemoryStore: ensureFreeSpace(384) called with curMem=152694,
maxMem=671101747
14/06/24 13:53:15 INFO MemoryStore: Block rdd_1_0 stored as values to memory (estimated size
384.0 B, free 639.9 MB)
14/06/24 13:53:15 INFO BlockManagerMasterActor$BlockManagerInfo: Added rdd_1_0 in memory
on 192.168.171.1:34930 (size: 384.0 B, free: 640.0 MB)
14/06/24 13:53:15 INFO BlockManagerMaster: Updated info of block rdd_1_0
14/06/24 13:53:15 INFO Executor: Serialized size of result for 0 is 563
14/06/24 13:53:15 INFO Executor: Sending result for 0 directly to driver
14/06/24 13:53:15 INFO Executor: Finished task ID 0
14/06/24 13:53:15 INFO TaskSetManager: Starting task 0.0:1 as TID 1 on executor localhost:
localhost (PROCESS_LOCAL)
14/06/24 13:53:15 INFO TaskSetManager: Serialized task 0.0:1 as 1690 bytes in 1 ms
14/06/24 13:53:15 INFO Executor: Running task ID 1
14/06/24 13:53:15 INFO BlockManager: Found block broadcast_0 locally
14/06/24 13:53:15 INFO TaskSetManager: Finished TID 0 in 250 ms on localhost (progress: 1/2)
14/06/24 13:53:15 INFO CacheManager: Partition rdd_1_1 not found, computing it
14/06/24 13:53:15 INFO HadoopRDD: Input split: file:/usr/lib/spark/README.md:51+51
14/06/24 13:53:15 INFO MemoryStore: ensureFreeSpace(544) called with curMem=153078,
maxMem=671101747
14/06/24 13:53:15 INFO MemoryStore: Block rdd_1_1 stored as values to memory (estimated size
544.0 B, free 639.9 MB)
14/06/24 13:53:15 INFO DAGScheduler: Completed ResultTask(0, 0)
14/06/24 13:53:15 INFO BlockManagerMasterActor$BlockManagerInfo: Added rdd_1_1 in memory
on 192.168.171.1:34930 (size: 544.0 B, free: 640.0 MB)
14/06/24 13:53:15 INFO BlockManagerMaster: Updated info of block rdd_1_1
14/06/24 13:53:15 INFO Executor: Serialized size of result for 1 is 563
14/06/24 13:53:15 INFO Executor: Sending result for 1 directly to driver
14/06/24 13:53:15 INFO Executor: Finished task ID 1
14/06/24 13:53:15 INFO DAGScheduler: Completed ResultTask(0, 1)
14/06/24 13:53:15 INFO TaskSetManager: Finished TID 1 in 24 ms on localhost (progress: 2/2)
14/06/24 13:53:15 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all
completed, from pool
14/06/24 13:53:15 INFO DAGScheduler: Stage 0 (count at SimpleApp.scala:11) finished in 0.292 s
14/06/24 13:53:15 INFO SparkContext: Job finished: count at SimpleApp.scala:11, took
0.458186941 s
14/06/24 13:53:15 INFO SparkContext: Starting job: count at SimpleApp.scala:12
14/06/24 13:53:15 INFO DAGScheduler: Got job 1 (count at SimpleApp.scala:12) with 2 output
partitions (allowLocal=false)
14/06/24 13:53:15 INFO DAGScheduler: Final stage: Stage 1 (count at SimpleApp.scala:12)
14/06/24 13:53:15 INFO DAGScheduler: Parents of final stage: List()
14/06/24 13:53:15 INFO DAGScheduler: Missing parents: List()
14/06/24 13:53:15 INFO DAGScheduler: Submitting Stage 1 (FilteredRDD[3] at filter at
SimpleApp.scala:12), which has no missing parents
14/06/24 13:53:15 INFO DAGScheduler: Submitting 2 missing tasks from Stage 1 (FilteredRDD[3]
at filter at SimpleApp.scala:12)
14/06/24 13:53:15 INFO TaskSchedulerImpl: Adding task set 1.0 with 2 tasks
14/06/24 13:53:15 INFO TaskSetManager: Starting task 1.0:0 as TID 2 on executor localhost:
localhost (PROCESS_LOCAL)
14/06/24 13:53:15 INFO TaskSetManager: Serialized task 1.0:0 as 1693 bytes in 0 ms
14/06/24 13:53:15 INFO Executor: Running task ID 2
14/06/24 13:53:15 INFO BlockManager: Found block broadcast_0 locally
14/06/24 13:53:15 INFO BlockManager: Found block rdd_1_0 locally
14/06/24 13:53:15 INFO Executor: Serialized size of result for 2 is 563
14/06/24 13:53:15 INFO Executor: Sending result for 2 directly to driver
14/06/24 13:53:15 INFO Executor: Finished task ID 2
14/06/24 13:53:15 INFO TaskSetManager: Starting task 1.0:1 as TID 3 on executor localhost:
localhost (PROCESS_LOCAL)
14/06/24 13:53:15 INFO TaskSetManager: Serialized task 1.0:1 as 1693 bytes in 0 ms
14/06/24 13:53:15 INFO Executor: Running task ID 3
14/06/24 13:53:15 INFO DAGScheduler: Completed ResultTask(1, 0)
14/06/24 13:53:15 INFO TaskSetManager: Finished TID 2 in 9 ms on localhost (progress: 1/2)
14/06/24 13:53:15 INFO BlockManager: Found block broadcast_0 locally
14/06/24 13:53:15 INFO BlockManager: Found block rdd_1_1 locally
14/06/24 13:53:15 INFO Executor: Serialized size of result for 3 is 563
14/06/24 13:53:15 INFO Executor: Sending result for 3 directly to driver
14/06/24 13:53:15 INFO Executor: Finished task ID 3
14/06/24 13:53:15 INFO DAGScheduler: Completed ResultTask(1, 1)
14/06/24 13:53:15 INFO DAGScheduler: Stage 1 (count at SimpleApp.scala:12) finished in 0.017 s
14/06/24 13:53:15 INFO TaskSetManager: Finished TID 3 in 9 ms on localhost (progress: 2/2)
14/06/24 13:53:15 INFO SparkContext: Job finished: count at SimpleApp.scala:12, took
0.028886185 s
14/06/24 13:53:15 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all
completed, from pool
Lines with a: 8, Lines with b: 5
14/06/24 13:53:15 INFO ConnectionManager: Selector thread was interrupted!
[success] Total time: 5 s, completed Jun 24, 2014 1:53:15 PM
[dc@localhost spark]$
The input file is:
[dc@localhost spark]$ cat README.md
thsi si a test
this is a test for lines with a
a
a
a
a
and for the letter b
b
b
b
another b with a
Notes: bigtop doesn't install sbt, you will have to install sbt under /usr/lib/spark to be like the
spark distribution. There is a build.properies error message:
[dc@localhost spark]$ sbt/sbt run
awk: cmd. line:1: fatal: cannot open file `./project/build.properties' for reading (No such file or
directory)
Launching sbt from sbt/sbt-launch-.jar
Error: Invalid or corrupt jarfile sbt/sbt-launch-.jar
Create a project directory and copy build.properties from the source distribution or create your
own. Update the sbt version inside build.properties to the version of sbt you are using. SBT goes
and fetches this version before starting. Will probably still work if you don't do this.
[dc@localhost spark]$ cat project/build.properties
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
#
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
sbt.version=0.13.5
Debug Step #2b: modify the above to support streaming.
Modify the sbt file to include dependencies:
[dc@localhost spark]$ cat simple.sbt
name := "Simple Project"
version := "1.0"
scalaVersion:="2.10.3"
libraryDependencies += "org.apache.spark" %% "spark-core" % "0.9.1"
resolvers += "Akka Repository" at "http://repo.akka.io/releases"
libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.3.0"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "0.9.1"
libraryDependencies += "org.twitter4j" % "twitter4j-stream" % "3.0.3"
Modify the SimpleApp.scala file to add streaming and print out the tweets. Copy the TwitterUtil
scala files and TwitterDStream class files from the 0.9.1 source code or impor the external Twitter
jar.
My source dir looks like:
[dc@localhost spark]$ ls src/main/scala/
SimpleApp.scala TwitterInputDStream.scala TwitterUtils.scala
[dc@localhost spark]$
[dc@localhost spark]$ cat src/main/scala/SimpleApp.scala
/*** SimpleApp.scala ***/
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.streaming._
object SimpleApp {
def main(args: Array[String]) {
val logFile = "/usr/lib/spark/README.md" // Should be some file on your system
//val sc = new SparkContext("local", "Simple App", "/usr/lib/spark",
// List("target/scala-2.10/simple-project_2.10-1.0.jar"))
//val logData = sc.textFile(logFile, 2).cache()
//val numAs = logData.filter(line => line.contains("a")).count()
//val numBs = logData.filter(line => line.contains("b")).count()
//println("Lines with a: %s, Lines with b: %s".format(numAs, numBs))
System.setProperty("twitter4j.oauth.consumerKey", "eFIaiOuxsny01VVQ2QWISK1Mw")
System.setProperty("twitter4j.oauth.consumerSecret",
"gDQI5EiCMJJaaNI8XVNhfZXwuCOYfeJ3XsOUNHvsXqgq0Hoj9T")
System.setProperty("twitter4j.oauth.accessToken",
Otz8w4yMKx6yCEWTH3dNTfuF8LYeLgqdoDrcl0oBK")
System.setProperty("twitter4j.oauth.accessTokenSecret",
"NFPFe2EzuKWuzRKmY1RENUBfQzGeGbAS1JzjX3Eu3GwDE")
//scaladocs not accurate, follow holden's exampple
//spark://192.168.171.1:7077
val stream = new StreamingContext("local","Simple App", Seconds(1))
"76976448-
val tweets= TwitterUtils.createStream(stream,None)
tweets.print()
stream.start()
println("+++++++++++++++++++++++++++++");
stream.awaitTermination()
//sc.stop()
}
}
Output:
こんな楽しいゲームはこれが初めて！！！ http://t.co/JqhnaWj8Gj', source='<a
href="https://twitter.com/" rel="nofollow">ネットワーク50</a>', isTruncated=false,
inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null',
geoLocation=null, place=null, retweetCount=97, isPossiblySensitive=false,
contributorsIDs=[J@ef3bee1, retweetedStatus=null, userMentionEntities=[],
urlEntities=[URLEntityJSONImpl{url='http://t.co/SbJlVMIzjc',
expandedURL='http://urx.nu/9CfR', displayURL='urx.nu/9CfR'}], hashtagEntities=[],
mediaEntities=[MediaEntityJSONImpl{id=481602776594513920, url=http://t.co/JqhnaWj8Gj,
mediaURL=http://pbs.twimg.com/media/Bq7_LHXCAAAXhzh.jpg,
mediaURLHttps=https://pbs.twimg.com/media/Bq7_LHXCAAAXhzh.jpg,
expandedURL=http://twitter.com/ninkiap/status/481602776774877185/photo/1,
displayURL='pic.twitter.com/JqhnaWj8Gj', sizes={0=Size{width=150, height=150, resize=101},
1=Size{width=340, height=408, resize=100}, 2=Size{width=500, height=600, resize=100},
3=Size{width=500, height=600, resize=100}}, type=photo}], currentUserRetweetId=-1,
user=UserJSONImpl{id=2515434492, name='【公式】大人気ゲームアプリ速報 ',
screenName='ninkiap', location='', description='オススメの無料アプリや知って得するアプリ情
報をアップしていきます。
知ってる人はＲＴしてみんなに教えてあげよ♪', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/474527354694283264/n0MTRAd_normal.jpeg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/474527354694283264/n0MTRAd_normal.jpeg', url='null', isProtected=false, followersCount=103, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=167, createdAt=Thu
May 22 06:29:24 PDT 2014, favouritesCount=0, utcOffset=32400, timeZone='Irkutsk',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='ja', statusesCount=1824, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=1, isFollowRequestSent=false}},
userMentionEntities=[UserMentionEntityJSONImpl{name='【公式】大人気ゲームアプリ速報 ',
screenName='ninkiap', id=2515434492}],
urlEntities=[URLEntityJSONImpl{url='http://t.co/SbJlVMIzjc',
expandedURL='http://urx.nu/9CfR', displayURL='urx.nu/9CfR'}], hashtagEntities=[],
mediaEntities=[MediaEntityJSONImpl{id=481602776594513920, url=http://t.co/JqhnaWj8Gj,
mediaURL=http://pbs.twimg.com/media/Bq7_LHXCAAAXhzh.jpg,
mediaURLHttps=https://pbs.twimg.com/media/Bq7_LHXCAAAXhzh.jpg,
expandedURL=http://twitter.com/ninkiap/status/481602776774877185/photo/1,
displayURL='pic.twitter.com/JqhnaWj8Gj', sizes={0=Size{width=150, height=150, resize=101},
1=Size{width=340, height=408, resize=100}, 2=Size{width=500, height=600, resize=100},
3=Size{width=500, height=600, resize=100}}, type=photo}], currentUserRetweetId=-1,
user=UserJSONImpl{id=2515560282, name='花言葉bot', screenName='hanakotobaw', location='',
description='花言葉をつぶやきます。 誰かにお花を贈る時などに参考になれば嬉しいです。
知らなかったらRTお願いします。', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/480690879175200772/6yVJqjie_normal.jpeg'
,
profileImageUrlHttps='https://pbs.twimg.com/profile_images/480690879175200772/6yVJqjie_norm
al.jpeg', url='null', isProtected=false, followersCount=52, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=446, createdAt=Thu
May 22 07:31:15 PDT 2014, favouritesCount=0, utcOffset=32400, timeZone='Irkutsk',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='ja', statusesCount=223, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=1, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:45 PDT 2014, id=481602865991929857, text='規制
垢【@ syara04282】からの無言フォローお許しください <BoT>', source='<a
href="http://autotweety.net" rel="nofollow">autotweety.net</a>', isTruncated=false,
inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null',
geoLocation=null, place=null, retweetCount=0, isPossiblySensitive=false,
contributorsIDs=[J@2478ad72, retweetedStatus=null, userMentionEntities=[], urlEntities=[],
hashtagEntities=[], mediaEntities=[], currentUserRetweetId=-1,
user=UserJSONImpl{id=1249201381, name='紗螺@しゃららん', screenName='syara0428',
location='室内', description='名前はしゃらです。 fate/弾丸論破/FF/マギ/銀魂…etcが好き。
なりきりさんや関係の一般さんと絡む用。
アイコン頂き物ヘッダートレスにつき持ち帰りはご遠慮ください',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/481454291480088578/TMgfgESb_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/481454291480088578/TMgfgESb_nor
mal.jpeg', url='http://twpf.jp/syara0428', isProtected=false, followersCount=281, status=null,
profileBackgroundColor='0099B9', profileTextColor='3C3940', profileLinkColor='0099B9',
profileSidebarFillColor='95E8EC', profileSidebarBorderColor='5ED4DC',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=288, createdAt=Thu Mar
07 06:32:19 PST 2013, favouritesCount=12857, utcOffset=32400, timeZone='Irkutsk',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme4/bg.gif',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme4/bg.gif',
profileBackgroundTiled=false, lang='ja', statusesCount=79682, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=9, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:45 PDT 2014, id=481602865987727361, text='RT
@AnfasA2014: ‫ هههه هه ههههه ههههه هههه ه ه ههههه ههه ههه ههه ههه هههه ه‬, ‫هه هههه ه‬
‫'~ه هههه هههه‬, source='<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for
iPhone</a>', isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@6ec32a12,
retweetedStatus=StatusJSONImpl{createdAt=Tue Jun 24 17:18:29 PDT 2014,
id=481592229014691841, text=' ‫ هههه هه ههههه ههههه هههه ه ه ههههه ههه ههه‬, ‫هه هههه ه‬
‫'~ههه ههه هههه ه ه هههه هههه‬, source='<a href="http://blackberry.com/twitter"
rel="nofollow">Twitter for BlackBerry®</a>', isTruncated=false, inReplyToStatusId=-1,
inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null', geoLocation=null, place=null,
retweetCount=3, isPossiblySensitive=false, contributorsIDs=[J@4eff3c8d, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=2291801757, name='‫'هههههه ههههههه‬,
screenName='AnfasA2014', location=' ~ ‫'~ هههه هههه هههه‬, description=' ‫هههه ههه هههههه‬
‫هههههه هههه هه هه ههههه ههههههههه ههههه ههه ههههه ههههه هه ههههههههه هههه ههه‬
‫'~ ههههه ههههه ههههههه هه ههههه ههههههههه‬, isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/477632234808045568/0oCxt5Gb_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/477632234808045568/0oCxt5Gb_nor
mal.jpeg', url='null', isProtected=false, followersCount=4127, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=4478, createdAt=Sun
Jan 19 06:55:15 PST 2014, favouritesCount=537, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='ar', statusesCount=9958, isGeoEnabled=true, isVerified=false,
translator=false, listedCount=4, isFollowRequestSent=false}},
userMentionEntities=[UserMentionEntityJSONImpl{name='‫'هههههه ههههههه‬,
screenName='AnfasA2014', id=2291801757}], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=2560181521, name='‫'هههههههه‬,
screenName='Froo7a_1981', location='', description=' ‫ههه هههههه هههههه هههههه ههههههههههه‬
‫'هههههه هههههه هههه ههههههههههههه‬, isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/481549523127242752/oB420nQW_normal.jp
eg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/481549523127242752/oB420nQW_no
rmal.jpeg', url='null', isProtected=false, followersCount=1200, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=1843, createdAt=Tue
Jun 10 17:11:42 PDT 2014, favouritesCount=48, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='ar', statusesCount=241, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:45 PDT 2014, id=481602866012905473, text='Can I
move out of this state and disconnect my phone ', source='<a
href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@56e0757f, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=1942450526, name='Cheyenne De Jarnett',
screenName='cheydejarnett', location='', description='Happy. Living life to the fullest. Fresno
State Bound', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/476827081678934016/5GHrOkF__normal.jp
eg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/476827081678934016/5GHrOkF__nor
mal.jpeg', url='null', isProtected=false, followersCount=156, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=164, createdAt=Sun Oct
06 17:38:42 PDT 2013, favouritesCount=1097, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='en', statusesCount=1553, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=0, isFollowRequestSent=false}}
...
14/06/24 18:01:48 INFO JobScheduler: Finished job streaming job 1403658108000 ms.0 from job
set of time 1403658108000 ms
14/06/24 18:01:48 INFO JobScheduler: Total delay: 0.069 s for time 1403658108000 ms
(execution: 0.062 s)
14/06/24 18:01:48 INFO MemoryStore: ensureFreeSpace(10958) called with curMem=458774,
maxMem=671101747
14/06/24 18:01:48 INFO MemoryStore: Block input-0-1403658108000 stored as bytes to memory
(size 10.7 KB, free 639.6 MB)
14/06/24 18:01:48 INFO BlockManagerMasterActor$BlockManagerInfo: Added input-01403658108000 in memory on 192.168.171.1:45241 (size: 10.7 KB, free: 639.6 MB)
14/06/24 18:01:48 INFO BlockManagerMaster: Updated info of block input-0-1403658108000
14/06/24 18:01:48 WARN BlockManager: Block input-0-1403658108000 already exists on this
machine; not re-adding it
14/06/24 18:01:48 INFO MemoryStore: ensureFreeSpace(2720) called with curMem=469732,
maxMem=671101747
14/06/24 18:01:48 INFO MemoryStore: Block input-0-1403658108200 stored as bytes to memory
(size 2.7 KB, free 639.6 MB)
14/06/24 18:01:48 INFO BlockManagerMasterActor$BlockManagerInfo: Added input-01403658108200 in memory on 192.168.171.1:45241 (size: 2.7 KB, free: 639.6 MB)
14/06/24 18:01:48 INFO BlockManagerMaster: Updated info of block input-0-1403658108200
14/06/24 18:01:48 WARN BlockManager: Block input-0-1403658108200 already exists on this
machine; not re-adding it
14/06/24 18:01:49 INFO NetworkInputTracker: Stream 0 received 3 blocks
14/06/24 18:01:49 INFO JobScheduler: Added jobs for time 1403658109000 ms
14/06/24 18:01:49 INFO JobScheduler: Starting job streaming job 1403658109000 ms.0 from job
set of time 1403658109000 ms
14/06/24 18:01:49 INFO SparkContext: Starting job: take at DStream.scala:586
14/06/24 18:01:49 INFO DAGScheduler: Got job 5 (take at DStream.scala:586) with 1 output
partitions (allowLocal=true)
14/06/24 18:01:49 INFO DAGScheduler: Final stage: Stage 5 (take at DStream.scala:586)
14/06/24 18:01:49 INFO DAGScheduler: Parents of final stage: List()
14/06/24 18:01:49 INFO DAGScheduler: Missing parents: List()
14/06/24 18:01:49 INFO DAGScheduler: Computing the requested partition locally
14/06/24 18:01:49 INFO BlockManager: Found block input-0-1403658107800 locally
14/06/24 18:01:49 INFO MemoryStore: ensureFreeSpace(87290) called with curMem=472452,
maxMem=671101747
14/06/24 18:01:49 INFO MemoryStore: Block input-0-1403658108800 stored as bytes to memory
(size 85.2 KB, free 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMasterActor$BlockManagerInfo: Added input-01403658108800 in memory on 192.168.171.1:45241 (size: 85.2 KB, free: 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMaster: Updated info of block input-0-1403658108800
14/06/24 18:01:49 INFO SparkContext: Job finished: take at DStream.scala:586, took
0.010101289 s
------------------------------------------Time: 1403658109000 ms
------------------------------------------StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870190428162, text='Just
because I don't start the conversation, doesn't mean I'm not dying to speak to you.', source='<a
href="http://dlvr.it" rel="nofollow">dlvr.it</a>', isTruncated=false, inReplyToStatusId=-1,
inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null', geoLocation=null, place=null,
retweetCount=0, isPossiblySensitive=false, contributorsIDs=[J@fbfc6d1, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=448879104, name='Vicky Edward',
screenName='VickyEdwardXVOZ', location='Columbus', description='#FootballIsLife #TrackNation
#D1Bound #Team$YB', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/378800000827538916/71f1b81d02a9c60c91
e24ca0ca293514_normal.jpeg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/378800000827538916/71f1b81d02a9
c60c91e24ca0ca293514_normal.jpeg', url='null', isProtected=false, followersCount=128,
status=null, profileBackgroundColor='C0DEED', profileTextColor='333333',
profileLinkColor='0084B4', profileSidebarFillColor='DDEEF6',
profileSidebarBorderColor='C0DEED', profileUseBackgroundImage=true, showAllInlineMedia=false,
friendsCount=351, createdAt=Wed Dec 28 05:39:39 PST 2011, favouritesCount=0, utcOffset=-1,
timeZone='null',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/3788000001361945
86/cuHHTVgb.jpeg',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/37880000013
6194586/cuHHTVgb.jpeg', profileBackgroundTiled=false, lang='en', statusesCount=995,
isGeoEnabled=false, isVerified=false, translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870194610176, text='Kill
me.', source='<a href="http://twitter.com/download/android" rel="nofollow">Twitter for
Android</a>', isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@7795cb6a, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=77515175, name='Fick Dich ♥',
screenName='grobesMadchen', location='In your nightmares.', description='Avenged Sevenfold Its
my fucking life *ww* blink-182, Slipknot, BFMV, Iwrestledabearonce, SOAD. Intento de baterista.
Futura médico con tatuajes. foREVer.', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/480581704914984960/m7pm__zu_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/480581704914984960/m7pm__zu_nor
mal.jpeg', url='https://www.facebook.com/Miriam.A7XfoREVer?ref=tn_tnmn', isProtected=false,
followersCount=464, status=null, profileBackgroundColor='642D8B', profileTextColor='333333',
profileLinkColor='FF0000', profileSidebarFillColor='EADEAA',
profileSidebarBorderColor='000000', profileUseBackgroundImage=true, showAllInlineMedia=false,
friendsCount=244, createdAt=Sat Sep 26 09:39:53 PDT 2009, favouritesCount=1808, utcOffset=18000, timeZone='Guadalajara',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/881363929/a2ce570
3a72339f6c5e746f0be4fcdd0.jpeg',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/881363929/a
2ce5703a72339f6c5e746f0be4fcdd0.jpeg', profileBackgroundTiled=true, lang='es',
statusesCount=43372, isGeoEnabled=true, isVerified=false, translator=false, listedCount=13,
isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870202990592, text='First
comer requisition prefixes - be pinched till follow upheave corpuscle cellphone reckoning
information?:... http://t.co/RBdjLpKMzF', source='<a href="http://dlvr.it"
rel="nofollow">dlvr.it</a>', isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1,
isFavorited=false, inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@b5dd97c, retweetedStatus=null,
userMentionEntities=[], urlEntities=[URLEntityJSONImpl{url='http://t.co/RBdjLpKMzF',
expandedURL='http://dlvr.it/66Dph2', displayURL='dlvr.it/66Dph2'}], hashtagEntities=[],
mediaEntities=[], currentUserRetweetId=-1, user=UserJSONImpl{id=1217647434,
name='MabelElizabeth', screenName='MabelElizabeth3', location='', description='null',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/3333464139/17eadb1ae785bf8b893b2a98efc
d8051_normal.jpeg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/3333464139/17eadb1ae785bf8b893b2
a98efcd8051_normal.jpeg', url='null', isProtected=false, followersCount=69, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=0, createdAt=Mon Feb
25 00:29:17 PST 2013, favouritesCount=0, utcOffset=3600, timeZone='London',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='en', statusesCount=22099, isGeoEnabled=false,
isVerified=false, translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870198820864, text='mewek
di bilang *sudaaahh*, nah udah gini balik kayak AC 5 PK, apaan :| ⚡
', source='<a
href="https://path.com/" rel="nofollow">Path</a>', isTruncated=false, inReplyToStatusId=-1,
inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null', geoLocation=null, place=null,
retweetCount=0, isPossiblySensitive=false, contributorsIDs=[J@3e7161b8, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=1622263176, name='NoerainiRSH',
screenName='NoerainiRSH', location='Bandung, Ina', description='capra's #yogurtarian',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/471094644160204801/J7qxSdlA_normal.jpeg
',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/471094644160204801/J7qxSdlA_norm
al.jpeg', url='null', isProtected=false, followersCount=82, status=null,
profileBackgroundColor='709397', profileTextColor='333333', profileLinkColor='FF3300',
profileSidebarFillColor='A0C5C7', profileSidebarBorderColor='86A4A6',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=113, createdAt=Fri Jul
26 00:14:09 PDT 2013, favouritesCount=11, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme6/bg.gif',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme6/bg.gif',
profileBackgroundTiled=false, lang='en', statusesCount=263, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870202994688,
text='¿Como hare para probar otro cuerpo sin pensar que estoy contigo ? (8.', source='<a
href="https://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@5c6970f, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=2482851636, name='PerraPalga♡',
screenName='LpdayRiver', location='', description='Larga vida al Rock anda Roll ♥',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/478749964815069184/GtecywS7_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/478749964815069184/GtecywS7_nor
mal.jpeg', url='null', isProtected=false, followersCount=49, status=null,
profileBackgroundColor='FF05C5', profileTextColor='333333', profileLinkColor='B30045',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='FFFFFF',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=53, createdAt=Wed May
07 15:54:39 PDT 2014, favouritesCount=288, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/4659640735326535
68/GTk_cMGE.jpeg',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/46596407353
2653568/GTk_cMGE.jpeg', profileBackgroundTiled=true, lang='es', statusesCount=2852,
isGeoEnabled=false, isVerified=false, translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870203002880, text='RT
@FoxwoodEllery: @omaReal @StevenCanales1 I am also up for this.', source='<a
href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@30e75cbf,
retweetedStatus=StatusJSONImpl{createdAt=Tue Jun 24 17:55:30 PDT 2014,
id=481601542785466368, text='@omaReal @StevenCanales1 I am also up for this.', source='<a
href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
isTruncated=false, inReplyToStatusId=481599100303843328, inReplyToUserId=61323546,
isFavorited=false, inReplyToScreenName='omaReal', geoLocation=null, place=null,
retweetCount=1, isPossiblySensitive=false, contributorsIDs=[J@257a18ae, retweetedStatus=null,
userMentionEntities=[UserMentionEntityJSONImpl{name='PerfectO', screenName='omaReal',
id=61323546}, UserMentionEntityJSONImpl{name='StevenCanales', screenName='StevenCanales1',
id=315476114}], urlEntities=[], hashtagEntities=[], mediaEntities=[], currentUserRetweetId=-1,
user=UserJSONImpl{id=543834968, name='Christian McCarver', screenName='FoxwoodEllery',
location='', description='Ay Real Shit I'm Fucking Really Fucking Real, Yeah Really. Instagram@christianmccarver', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/481131876074008576/FjpDpOzk_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/481131876074008576/FjpDpOzk_nor
mal.jpeg', url='null', isProtected=false, followersCount=489, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='FFFFFF',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=375, createdAt=Mon
Apr 02 17:14:42 PDT 2012, favouritesCount=11070, utcOffset=-14400, timeZone='Eastern Time
(US & Canada)',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/668796768/4f3e85b
d28e1261f1a150fc18b9234de.jpeg',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/668796768/4
f3e85bd28e1261f1a150fc18b9234de.jpeg', profileBackgroundTiled=true, lang='en',
statusesCount=16336, isGeoEnabled=true, isVerified=false, translator=false, listedCount=0,
isFollowRequestSent=false}}, userMentionEntities=[UserMentionEntityJSONImpl{name='Christian
McCarver', screenName='FoxwoodEllery', id=543834968},
UserMentionEntityJSONImpl{name='PerfectO', screenName='omaReal', id=61323546},
UserMentionEntityJSONImpl{name='StevenCanales', screenName='StevenCanales1',
id=315476114}], urlEntities=[], hashtagEntities=[], mediaEntities=[], currentUserRetweetId=-1,
user=UserJSONImpl{id=315476114, name='StevenCanales', screenName='StevenCanales1',
location='', description='NLS Co-leader DADDY 1st, Friend 2nd', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/467404013504196608/kEq857Wn_normal.jp
eg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/467404013504196608/kEq857Wn_nor
mal.jpeg', url='null', isProtected=false, followersCount=149, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=162, createdAt=Sat Jun
11 16:07:08 PDT 2011, favouritesCount=349, utcOffset=-18000, timeZone='Central Time (US &
Canada)', profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='en', statusesCount=4708, isGeoEnabled=true, isVerified=false,
translator=false, listedCount=0, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870219792385, text='next
week. OMG ', source='<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter
for iPhone</a>', isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@72ea5aba, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=1113240404, name='k a t i e ',
screenName='its_lucyhale', location='another girl obsessed w/ Lucy', description='• put up your
damn umbrella & get on with it •', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/476803290257043456/7PZ43QT2_normal.jp
eg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/476803290257043456/7PZ43QT2_no
rmal.jpeg', url='null', isProtected=false, followersCount=1288, status=null,
profileBackgroundColor='642D8B', profileTextColor='333333', profileLinkColor='FF0073',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='FFFFFF',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=1185, createdAt=Tue
Jan 22 18:37:54 PST 2013, favouritesCount=673, utcOffset=-14400, timeZone='Eastern Time (US
& Canada)',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/4461367112783626
25/OwhDD4WT.jpeg',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/44613671127
8362625/OwhDD4WT.jpeg', profileBackgroundTiled=true, lang='en', statusesCount=5460,
isGeoEnabled=false, isVerified=false, translator=false, listedCount=4, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870215585793, text='Otp
smlm :p', source='<a href="http://www.samsungmobile.com" rel="nofollow">Samsung Mobile</a>',
isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false,
inReplyToScreenName='null', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@77eaa446, retweetedStatus=null,
userMentionEntities=[], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=1430196182, name='▶ STRONG_GIRL ◀',
screenName='ViennaVegas', location='KUALA TERENGGANU ', description='◀ Bila orang dah tak
suka , jangan merayu-rayu suruh dia terima kita lagi , Lupakan dia . Mungkin Allah da sediakan
seseorang yang lebih baik :') ▶', isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/461940355118546945/mI7_S8vp_normal.jpeg
',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/461940355118546945/mI7_S8vp_norm
al.jpeg', url='null', isProtected=false, followersCount=253, status=null,
profileBackgroundColor='050505', profileTextColor='333333', profileLinkColor='FA7DC4',
profileSidebarFillColor='EFEFEF', profileSidebarBorderColor='000000',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=246, createdAt=Wed
May 15 03:50:51 PDT 2013, favouritesCount=17, utcOffset=28800, timeZone='Kuala Lumpur',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/4421735416019312
64/410cqTOa.png',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/44217354160
1931264/410cqTOa.png', profileBackgroundTiled=true, lang='en', statusesCount=2807,
isGeoEnabled=true, isVerified=false, translator=false, listedCount=1, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870194630656,
text='@agion_yorg いい天気だな。アギオン、手合わせついでに久々に遠乗りに出かけない
か？', source='<a href="http://twittbot.net/" rel="nofollow">twittbot.net</a>', isTruncated=false,
inReplyToStatusId=-1, inReplyToUserId=163849985, isFavorited=false,
inReplyToScreenName='agion_yorg', geoLocation=null, place=null, retweetCount=0,
isPossiblySensitive=false, contributorsIDs=[J@4dc72d2b, retweetedStatus=null,
userMentionEntities=[UserMentionEntityJSONImpl{name='アギオン・ヨルク',
screenName='agion_yorg', id=163849985}], urlEntities=[], hashtagEntities=[], mediaEntities=[],
currentUserRetweetId=-1, user=UserJSONImpl{id=419631245, name='ティレル・ライトウィング
', screenName='tyrel_lightwing', location='執務室', description='司令官様の親友のティレルの自
動botです。時々、司令官様(@agion_yorg)と会話しています。反応語句は下記参照。',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/2679766397/935dbccc7d41105c047e1e683f
6c16ff_normal.jpeg',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/2679766397/935dbccc7d41105c047e
1e683f6c16ff_normal.jpeg', url='http://m-cld.sakura.ne.jp/ew_bot.html', isProtected=false,
followersCount=38, status=null, profileBackgroundColor='FCFCFC', profileTextColor='7D4230',
profileLinkColor='C4875E', profileSidebarFillColor='9AE6E2',
profileSidebarBorderColor='9AE6E2', profileUseBackgroundImage=true, showAllInlineMedia=false,
friendsCount=22, createdAt=Wed Nov 23 07:56:18 PST 2011, favouritesCount=0, utcOffset=36000, timeZone='Hawaii',
profileBackgroundImageUrl='http://pbs.twimg.com/profile_background_images/370607107/fu14.jpg
',
profileBackgroundImageUrlHttps='https://pbs.twimg.com/profile_background_images/370607107/f
u14.jpg', profileBackgroundTiled=false, lang='ja', statusesCount=33383, isGeoEnabled=false,
isVerified=false, translator=false, listedCount=2, isFollowRequestSent=false}}
StatusJSONImpl{createdAt=Tue Jun 24 18:00:46 PDT 2014, id=481602870186229761, text='好き
なアニメあればRT！ #RTした人全員フォローする #相互希望 超電磁砲 中二病 おにあい はが
ない けいおん ニャル子 さくら荘 俺ガイル ジョジョ イニD 進撃 ロンパ 恋愛ラボ きんモザ
脳コメ 妹ちょ みでし 生徒会役員共 ディーふらぐ そにアニ', source='<a
href="http://twittbot.net/" rel="nofollow">twittbot.net</a>', isTruncated=false,
inReplyToStatusId=-1, inReplyToUserId=-1, isFavorited=false, inReplyToScreenName='null',
geoLocation=null, place=null, retweetCount=0, isPossiblySensitive=false,
contributorsIDs=[J@4f6689f1, retweetedStatus=null, userMentionEntities=[], urlEntities=[],
hashtagEntities=[HashtagEntityJSONImpl{text='RTした人全員フォローする'},
HashtagEntityJSONImpl{text='相互希望'}], mediaEntities=[], currentUserRetweetId=-1,
user=UserJSONImpl{id=2426069922, name='夏輝＠相互フォロー支援',
screenName='sougo_natuki_', location='日本', description='アニメ好き、ボカロ好き、東方好き
のための相互フォロー支援垢です！ フォロバ100％ですがずっと浮上してるわけではないの
でフォロー遅れる場合があります。 フォローをお待ちしております',
isContributorsEnabled=false,
profileImageUrl='http://pbs.twimg.com/profile_images/451801811393855488/twoye3k3_normal.jpe
g',
profileImageUrlHttps='https://pbs.twimg.com/profile_images/451801811393855488/twoye3k3_nor
mal.jpeg', url='null', isProtected=false, followersCount=441, status=null,
profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4',
profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED',
profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=451, createdAt=Thu Apr
03 12:15:12 PDT 2014, favouritesCount=0, utcOffset=-1, timeZone='null',
profileBackgroundImageUrl='http://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundImageUrlHttps='https://abs.twimg.com/images/themes/theme1/bg.png',
profileBackgroundTiled=false, lang='ja', statusesCount=247, isGeoEnabled=false, isVerified=false,
translator=false, listedCount=0, isFollowRequestSent=false}}14/06/24 18:01:49 WARN
BlockManager: Block input-0-1403658108800 already exists on this machine; not re-adding it
...
14/06/24 18:01:49 INFO JobScheduler: Finished job streaming job 1403658109000 ms.0 from job
set of time 1403658109000 ms
14/06/24 18:01:49 INFO JobScheduler: Total delay: 0.040 s for time 1403658109000 ms
(execution: 0.038 s)
14/06/24 18:01:49 INFO MemoryStore: ensureFreeSpace(22562) called with curMem=559742,
maxMem=671101747
14/06/24 18:01:49 INFO MemoryStore: Block input-0-1403658109000 stored as bytes to memory
(size 22.0 KB, free 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMasterActor$BlockManagerInfo: Added input-01403658109000 in memory on 192.168.171.1:45241 (size: 22.0 KB, free: 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMaster: Updated info of block input-0-1403658109000
14/06/24 18:01:49 WARN BlockManager: Block input-0-1403658109000 already exists on this
machine; not re-adding it
14/06/24 18:01:49 INFO MemoryStore: ensureFreeSpace(4412) called with curMem=582304,
maxMem=671101747
14/06/24 18:01:49 INFO MemoryStore: Block input-0-1403658109200 stored as bytes to memory
(size 4.3 KB, free 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMasterActor$BlockManagerInfo: Added input-01403658109200 in memory on 192.168.171.1:45241 (size: 4.3 KB, free: 639.5 MB)
14/06/24 18:01:49 INFO BlockManagerMaster: Updated info of block input-0-1403658109200
14/06/24 18:01:49 WARN BlockManager: Block input-0-1403658109200 already exists on this
machine; not re-adding it
^C[dc@localhost spark]$