Python for Hive

skybin090804

<br>
當使用hadoop成為web日志或生產(chǎn)日志的存儲池后，我們可以使用./hive登錄到hive shell中<br>
進行查詢操作。但為了方便,在查詢方面用python進行封裝好相應(yīng)方法進行使用，用java進行我們的統(tǒng)計分析。<br>
<br>
使用hive客戶端的相關(guān)功能只需要將${hive_home}/lib/py路徑加入但python的運行環(huán)境路徑中就行了。<br>
如以下腳本：<br>
<br>
<br>
#/usr/bin/env python<br>
#-*- coding: utf-8 -*-<br>
<br>
import os,sys<br>
<br>
dir = os.path.dirname(os.path.realpath(sys.argv[0]))<br>
<br>
sys.path.insert(0, dir + '/lib/py')&nbsp; <font color="#f00000">####引入相關(guān)python for hive 的相關(guān)環(huán)境</font><br>
<br>
<br>
from hive_service import ThriftHive<br>
from hive_service.ttypes import HiveServerException<br>
from thrift import Thrift<br>
from thrift.transport import TSocket<br>
from thrift.transport import TTransport<br>
from thrift.protocol import TBinaryProtocol<br>
<br>
<br>
def hiveExe(sql):<br>
&nbsp;&nbsp;&nbsp; try:<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; transport = TSocket.TSocket('192.168.237.13', 10000) <br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; transport = TTransport.TBufferedTransport(transport)<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; protocol = TBinaryProtocol.TBinaryProtocol(transport)<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; client = ThriftHive.Client(protocol)<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; transport.open()<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; client.execute(sql)<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; print "The return value is : "+client.fetchOne()<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; transport.close()<br>
&nbsp;&nbsp;&nbsp; except Thrift.TException, tx:<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; print '%s' % (tx.message)<br>
<br>
if __name__ == '__main__':<br>
&nbsp;&nbsp;&nbsp;&nbsp; print hiveExe("select * from img_ana limit 2")<br>