1、环境信息
2、安装zookeeper 3、安装druid 4、启动druid 5、验证druid 6、遇到的坑1、环境信息
Centos6.5 32GB 8C Zookeeper 3.4.5 Druid 0.9.22、安装zookeeper
tar -xzf zookeeper-3.4.6.tar.gzcd zookeeper-3.4.6cp conf/zoo_sample.cfg conf/zoo.cfg./bin/zkServer.sh start
3、安装 druid
3、1解压druid 3、2 执行 bin/init 初始化化环境4、启动druid
4、1启动5个终端执行下面5个命名，或者启动一个将命令执行丢入后台java `cat conf-quickstart/druid/historical/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historicaljava `cat conf-quickstart/druid/broker/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server brokerjava `cat conf-quickstart/druid/coordinator/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/coordinator:lib/*" io.druid.cli.Main server coordinatorjava `cat conf-quickstart/druid/overlord/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/overlord:lib/*" io.druid.cli.Main server overlordjava `cat conf-quickstart/druid/middleManager/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/middleManager:lib/*" io.druid.cli.Main server middleManager
也可以用nohup在后台运行
4、2验证是否正常
执行完成以后可以查看服务是否启动索引任务界面
5、验证druid
5、1 修改wikiticker-index.json 文件为本地local模式的{ "type" : "index", "spec" : { "ioConfig" : { "type" : "index", "firehose" : { "type" : "local", "baseDir" : "/home/hadoop/druid-0.9.2/quickstart", "filter" : "wikiticker-2015-09-12-sampled.json" } }, "dataSchema" : { "dataSource" : "wikiticker", "granularitySpec" : { "type" : "uniform", "segmentGranularity" : "day", "queryGranularity" : "none", "intervals" : ["2015-09-12/2015-09-13"] }, "parser" : { "type" : "string", "parseSpec" : { "format" : "json", "dimensionsSpec" : { "dimensions" : [ "channel", "cityName", "comment", "countryIsoCode", "countryName", "isAnonymous", "isMinor", "isNew", "isRobot", "isUnpatrolled", "metroCode", "namespace", "page", "regionIsoCode", "regionName", "user" ] }, "timestampSpec" : { "format" : "auto", "column" : "time" } } }, "metricsSpec" : [ { "name" : "count", "type" : "count" }, { "name" : "added", "type" : "longSum", "fieldName" : "added" }, { "name" : "deleted", "type" : "longSum", "fieldName" : "deleted" }, { "name" : "delta", "type" : "longSum", "fieldName" : "delta" }, { "name" : "user_unique", "type" : "hyperUnique", "fieldName" : "user" } ] }, "tuningConfig" : { "type" : "index", "partitionsSpec" : { "type" : "hashed", "targetPartitionSize" : 5000000 }, "jobProperties" : {} } }} 5、2 执行 curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @quickstart/wikiticker-index.json localhost:8090/druid/indexer/v1/task 命令来overlord数据
[hadoop@SZB-L0038787 druid-0.9.2]$ curl -X 'POST' -H 'Content-Type:application/json' -d @quickstart/wikiticker-index.json localhost:8090/druid/indexer/v1/task{"task":"index_wikiticker_2017-05-22T02:06:52.225Z"} 5、3 查看运行的情况，可以在console上面查看是否运行成功
5、4查看数据
运行如下命令curl -L -H'Content-Type: application/json' -XPOST --data-binary @quickstart/wikiticker-top-pages.json http://localhost:8082/druid/v2/?pretty
[ { "timestamp" : "2015-09-12T00:46:58.771Z", "result" : [ { "edits" : 33, "page" : "Wikipedia:Vandalismusmeldung" }, { "edits" : 28, "page" : "User:Cyde/List of candidates for speedy deletion/Subpage" }, { "edits" : 27, "page" : "Jeremy Corbyn" }, { "edits" : 21, "page" : "Wikipedia:Administrators' noticeboard/Incidents" }, { "edits" : 20, "page" : "Flavia Pennetta" }, { "edits" : 18, "page" : "Total Drama Presents: The Ridonculous Race" }, { "edits" : 18, "page" : "User talk:Dudeperson176123" }, { "edits" : 18, "page" : "Wikipédia:Le Bistro/12 septembre 2015" }, { "edits" : 17, "page" : "Wikipedia:In the news/Candidates" }, { "edits" : 17, "page" : "Wikipedia:Requests for page protection" }, { "edits" : 16, "page" : "Utente:Giulio Mainardi/Sandbox" }, { "edits" : 16, "page" : "Wikipedia:Administrator intervention against vandalism" }, { "edits" : 15, "page" : "Anthony Martial" }, { "edits" : 13, "page" : "Template talk:Connected contributor" }, { "edits" : 12, "page" : "Chronologie de la Lorraine" }, { "edits" : 12, "page" : "Wikipedia:Files for deletion/2015 September 12" }, { "edits" : 12, "page" : "Гомосексуальный образ жизни" }, { "edits" : 11, "page" : "Constructive vote of no confidence" }, { "edits" : 11, "page" : "Homo naledi" }, { "edits" : 11, "page" : "Kim Davis (county clerk)" }, { "edits" : 11, "page" : "Vorlage:Revert-Statistik" }, { "edits" : 11, "page" : "Конституция Японской империи" }, { "edits" : 10, "page" : "The Naked Brothers Band (TV series)" }, { "edits" : 10, "page" : "User talk:Buster40004" }, { "edits" : 10, "page" : "User:Valmir144/sandbox" } ]} ] 6、遇到的坑
源文件的路径，在druid中配置路径baseDir 这个建议填写绝对路径，filter这个只填写文件名称。如果你需要填写信息的话filter这个参数带的路径是没有实际意义的
2017-05-22T01:34:40,650 INFO [task-runner-0-priority-0] io.druid.segment.realtime.firehose.LocalFirehoseFactory - Searching for all [quickstart/wikiticker-2015-09-12-sampled.json] in and beneath [/home/hadoop/druid-0.9.2/.]2017-05-22T01:34:40,673 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[IndexTask{id=index_wikiticker_2017-05-22T01:34:36.301Z, type=index, dataSource=wikiticker}]com.metamx.common.ISE: Found no files to ingest! Check your schema. at io.druid.segment.realtime.firehose.LocalFirehoseFactory.connect(LocalFirehoseFactory.java:100) ~[druid-server-0.9.2.jar:0.9.2] at io.druid.segment.realtime.firehose.LocalFirehoseFactory.connect(LocalFirehoseFactory.java:46) ~[druid-server-0.9.2.jar:0.9.2] at io.druid.indexing.common.task.IndexTask.getDataIntervals(IndexTask.java:255) ~[druid-indexing-service-0.9.2.jar:0.9.2] at io.druid.indexing.common.task.IndexTask.run(IndexTask.java:201) ~[druid-indexing-service-0.9.2.jar:0.9.2] at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.9.2.jar:0.9.2] at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.9.2.jar:0.9.2] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]2017-05-22T01:34:40,678 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_wikiticker_2017-05-22T01:34:36.301Z] status changed to [FAILED].2017-05-22T01:34:40,680 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: { "id" : "index_wikiticker_2017-05-22T01:34:36.301Z", "status" : "FAILED", "duration" : 232}