Hadoop을 사용하는 상용 솔루션인 Splunk를 정리 합니다.
로그 수집 및 로그 기반 통계 전문 솔루션인 Splunk는 간단한 ETL 및 자동으로 내용에 대한 Index를 생성 합니다.
### wget http://download.splunk.com/releases/5.0.3/splunk/linux/splunk-5.0.3-163460-linux-2.6-x86_64.rpm
### wget http://download.splunk.com/releases/6.0/splunk/linux/splunk-6.0-182037-linux-2.6-x86_64.rpm
wget http://download.splunk.com/releases/6.0.2/splunk/linux/splunk-6.0.2-196940-linux-2.6-x86_64.rpm
rpm -i splunk-6.0-182037-linux-2.6-x86_64.rpm
### rpm -i --prefix=~ splunk-5.0.3-163460-linux-2.6-x86_64.rpm #--- 설치되는 폴더 변경시
### rpm -U splunk-5.0.3-163460-linux-2.6-x86_64.rpm #--- Splunk upgrade
### rpm -U --prefix=~ splunk-5.0.3-163460-linux-2.6-x86_64.rpm #--- SPlunk upgrade### ----------------------------------------------------------------------------
### Splunk 설정
### ----------------------------------------------------------------------------
export SPLUNK_HOME=/opt/splunk
export PATH=${PATH}:${SPLUNK_HOME}/bin
CDPATH=${CDPATH}:/optsplunk start --accept-license #--- Splunk 시작 (첫 시작시)
### splunk enable boot-start #--- CentOS 기동시 자동 시작 설정
### splunk enable boot-start -user splunk wget http://download.splunk.com/releases/5.0.4/universalforwarder/linux/splunkforwarder-5.0.4-172409-linux-2.6-x86_64.rpm?ac=get_splunk_downloadsplunk
splunk help cheatsheetsplunk set web-port 9000
splunk set splunkd-port 9089eval sTime=strftime(time / 1000000, "%Y-%m-%d %H:%M:%S.%6N") error OR failed OR severe OR (sourcetype=access_* (404 OR 500 OR 503))
error OR failed OR severe OR (sourcetype=access_* (status=404 OR status=500 OR status=503))
sourcetype=access_* action=purchase | top category_idstart-dfs.shERROR: SIC ERROR 119 - SIC Error for lea: Client could not choose an authentication method for service lea
splunk internal call command: $SPLUNK_HOME/bin/splunk _internal call /servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@opsec
splunk output: QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@opsec'
FAILED: 'HTTP/1.1 404 Not Found'
Content:
In handler 'log_status': Could not find object id=1@opsec
$SPLUNK_HOME/Splunk_TA_opseclea_linux22/local/opsec.conf
lea_server_auth_port = 18185
lea_server_auth_type = sslca
lea_server_ip = 10.0.xxx.5$CHECKPOINT_HOME/conf/fwopsec.conf
lea_server auth_port 18185
lea_server auth_type sslcasslca 대신 ssl_opsec 인증 방식을 지원할 경우
cd etc/apps/Splunk_TA_opseclea_linux22/opsec-tools
./opsec_putkey port 18185 CMA_IP
authkey.C 파일이 생성됨
etc/apps/lea-loggrabber-splunk/bin 폴더에 authkey.C 파일 복사
vi etc/apps/Splunk_TA_opseclea_linux22/default/lea.conf
lea_server auth_type auth_opsec
Splunk restart
splunk stop
splunk clean eventdata 인덱스명
splunk start"설정 -> 사용자 인터페이스 -> 탐색 메뉴" 메뉴에서 앱 컨텍스트로 "CustomApp(CustomApp)"을 선택 합니다.
"default"를 선택하여 메뉴을 수정 합니다.
메뉴 관리
View 화면에서 "편집 -> 패널 편집" 메뉴를 선택 합니다.
"패널 추가" 버튼을 선택 합니다.
HTML Element 추가
명명 규칙
<div class="dashboard-cell" style="width: 50%;">
<div class="dashboard-panel clearfix">
<div class="panel-element-row">
<div class="dashboard-element html" id="element2" style="width: 100%">
<div class="panel-body html">
<div><p>HTML ElementJavaScript에서 HTML Element 노드 찾기
require(
["splunkjs/mvc/utils", "splunkjs/mvc/tokenutils", "underscore", "jquery"]("splunkjs/mvc",),
function(mvc, utils, TokenUtils, _, $) {
var node = null;
node = $("#element2 .panel-body");
alert(node.html());
}
);JavaScript에서 d3.js 파일 사용 방법
RequireJS의 Base 폴더 : $SPLUNK_HOME/share/splunk/search_mrsparkle/exposed/js/
$SPLUNK_HOME/share/splunk/search_mrsparkle/exposed/jslib/d3/d3.js 파일이 존재할 경우
require(
["splunkjs/mvc", "splunkjs/mvc/utils", "splunkjs/mvc/tokenutils", "underscore", "jquery",
"../jslib/d3/d3"],
function(mvc, utils, TokenUtils, _, $, d3) {
var node = null;
viewTemplate();
viewJavaScriptObject(window, "window");
chart001(d3);
}
);Search 생성
var search004 = new SearchManager({
"id": "search004 ",
"search": "index=json | head 10 | table _time guid gubun filename maxDuration source sourcetype",
"earliest_time": "-2400h",
"latest_time": "now",
"preview": true
});Search 결과 가져오기
var manager = splunkjs.mvc.Components.getInstance('search004'); #--- SearchManager 인스턴스 생성
var data = manager.data('results', { #--- SplunkResultsModel 생성 (events, preview, results, summary)
output_mode: 'json_rows',
count: 0 #--- 가져올 레코드 수 지정 (0. 모두 가져오기)
});
data.on('data', function(results) { #--- results : SplunkResultsModel
#--- 데이터를 가져올 때까지 반복적으로 호출됨
if (!data.hasData()) {
return;
}
var collection = results.collection().toJSON();
#--- collection에 배열 형태로 레코드가 저장됨. 각 레코드는 object 형태를 가짐 ("{name: value}")
for (var idx = 0; idx < collection.length; idx++) {
var item = collection[idx](idx.md);
window.alert("item[+ idx + "](") = " + JSONtoString(item));
}
});
manager.startSearch(); cd /nas/appl/splunk/etc/apps/framework
./splunkdj createapp zzapps
splunk stop
splunk start
cd /nas/appl/splunk/etc/apps/zzappsvi /nas/appl/splunk/etc/apps/zzapps/django/zzapps/templates/home001.html
{# Boilerplate for a Django Bindings template #}
{% extends "splunkdj:base_with_app_bar.html" %}
{% load splunkmvc %}
{% block title %}{ {app_name}} Your page title goes here{% endblock title %}
{% block css %}
<!-- Style sheets are loaded here -->
<link rel="stylesheet" type="text/css" href="{ {STATIC_URL}}{ {app_name}}/custom.css" />
<link rel="stylesheet" type="text/css" href="{ {STATIC_URL}}splunkjs/css/dashboard.css" />
<link rel="stylesheet" href="http://code.jquery.com/ui/1.10.0/themes/base/jquery-ui.css" />
<style>
/* Define any page styles here*/
</style>
{% endblock css %}
{% block content %}
<!-- You can use HTML and <div> tags for layout -->
{# Splunk views go here #}
<!-- http://dev.splunk.com/view/SP-CAAAEQA -->
<div class="dashboard-body container-fluid main-section-body">
<div class="row">
<div class="dashboard-header clearfix">
<h2>Add views to a template using Django template tags</h2>
</div>
</div>
<div class="fieldset">
{% dropdown id="drop_headcount" default="1" value="$headcount$"|token_safe %}
{% radiogroup id="radio_sourcetype" managerid="search_sourcetypes"
labelField="sourcetype" valueField="sourcetype"
value="$sourcetype$"|token_safe
default="splunkd"%}
<div class="input form-submit" id="search_btn">
<button class="btn btn-primary submit">Search</button>
</div>
</div>
<div class="dashboard-row">
<div class="dashboard-cell" style="width: 50%;">
<div class="dashboard-panel">
<div class="panel-head">
<h3>Chart</h3>
</div>
<div class="panel-body">
<p>view chart</p>
{% chart id="chart1" managerid="search1" type="bar" %}
</div>
</div>
</div>
<div class="dashboard-cell" style="width: 50%;">
<div class="dashboard-panel">
<div class="panel-head">
<h3>Chart</h3>
</div>
<div class="panel-body">
<p>view chart</p>
{% chart id="chart_sourcetype" managerid="search_chart" type="pie" %}
</div>
</div>
</div>
</div>
<div class="dashboard-row">
<div class="dashboard-cell" style="width: 100%;">
<div class="dashboard-panel">
<div class="panel-head">
<h3>Table</h3>
</div>
<div class="panel-body">
<p>view table : $sourcetype$</p>
{% table id="table_searchresults" managerid="search_resulttable" %}
</div>
</div>
</div>
</div>
<div class="dashboard-row">
<div class="dashboard-cell" style="width: 30%;">
<div class="dashboard-panel">
<div class="panel-head">
<h3>Timeline</h3>
</div>
<div class="panel-body">
<p>view timeline</p>
{% timeline id="timeline1" managerid="search1" %}
</div>
</div>
</div>
<div class="dashboard-cell" style="width: 70%;">
<div class="dashboard-panel">
<div class="panel-head">
<h3>Event Viewer</h3>
</div>
<div class="panel-body">
<p>view events</p>
{% eventsviewer id="eviewer1" managerid="search1" %}
</div>
</div>
</div>
</div>
</div>
{% endblock content%}
{% block managers %}
{# Search managers go here #}
{% searchmanager id="search1"
search="index=_internal | head 1000 | stats count by sourcetype"
preview=True
required_field_list="*"
status_buckets=300 %}
{% savedsearchmanager
id="search2"
searchname="Top five sourcetypes"
app="search" %}
{% searchmanager
id="search_resulttable"
search="index=_internal sourcetype=$sourcetype$ | head $headcount$"|token_safe
cache=True
%}
{% searchmanager
id="search_chart"
search="index=_internal | head 1000 | stats count by sourcetype"
cache=True
%}
{% searchmanager
id="search_sourcetypes"
search="index=_internal | head 1000 | top sourcetype"
cache=True
%}
{% endblock managers %}
{% block js %}
{# JavaScript goes here #}
<script>
var deps = [
"splunkjs/ready!",
"splunkjs/mvc/radiogroupview"
];
require(deps, function(mvc) {
var choices_headcount = [
{label:"1", value: "1"},
{label:"2", value: "2"},
{label:"3", value: "3"}
];
splunkjs.mvc.Components.getInstance("drop_headcount").settings.set("choices", choices_headcount);
});
</script>
{% endblock js %} <earliestTime>rt-5m</earliestTime>
<latestTime>rt</latestTime> <li>최대 초당 6,000개의 서비스, 서비스 당 2 이벤트</li>
<li>
<b>
<font color="blue">최대 분당 720,000 인덱싱을 하여야 함</font>
</b>
</li> <li>서비스별 IO폼 파일을 프로그램으로 자동 파싱하여 데이터베이스에 저장함</li>
<li>
<b>
<font color="blue">데이터베이스 저장된 정보를 사용하여 서비스 로그 인덱싱 자동화함</font>
</b>
</li><searchString>index=_internal sourcetype=splunkd series=json | bucket _time span=1m | eval epm = eps * 60 | eval kbpm = kbps * 60 | stats avg(epm) as "분당 인덱싱 건수" by _time</searchString>
<earliestTime>$earliest$</earliestTime>
<latestTime>$latest$</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">none</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option><title>분당 인덱싱 현황</title>
<searchString>index=_internal sourcetype=splunkd series=json | bucket _time span=1m | eval epm = eps * 60 | eval kbpm = kbps * 60 | stats avg(epm) as "분당 인덱싱 건수" avg(kbpm) as "분당 인덱싱 용량(KB)" by _time splunk_server | rename splunk_server as "인덱스 서버"</searchString>
<earliestTime>$earliest$</earliestTime>
<latestTime>$latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">none</option>
<option name="count">10</option>
yum install httpd httpd-*
yum install mod_sslvi /etc/httpd/conf/httpd.conf
ProxyRequests On
ProxyVia On
Order deny,allow
Allow from all
SSLProxyEngine On
ProxyPass /proxy https://node201.hadoop.com:8089/ retry=0 timeout=5
ProxyPassReverse /proxy/ https://node201.hadoop.com:8089/
같은 장비의 다른 서비스 호출시 SELinux에 의해 차단됨
Permission denied: proxy: HTTP: attempt to connect to 127.0.0.1:8888 (*) failed/usr/sbin/setsebool httpd_can_network_connect true참고 문헌