基于hadoop大規(guī)模數(shù)據(jù)排序算法---韓旭紅組 第二次報(bào)告

xuyuanchao_cnu

<DIV>
<P style="TEXT-ALIGN: center; mso-line-height-alt: 15.6pt" align=center><FONT face=宋體><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 22pt">基于<SPAN lang=EN-US>hadoop</SPAN>的大規(guī)模數(shù)據(jù)排序算法</SPAN></B><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US></SPAN></FONT></P>
<P style="TEXT-ALIGN: center; mso-line-height-alt: 15.6pt" align=center><FONT face=宋體><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 18pt" lang=EN-US>(</SPAN></B><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 18pt">第二次報(bào)告）</SPAN></B><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US></SPAN></FONT></P>
<P style="mso-line-height-alt: 15.6pt"><FONT face=宋體><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 22pt" lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </SPAN></SPAN></B><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 15pt" lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp;&nbsp;</SPAN></SPAN></B><B style="mso-bidi-font-weight: normal"><SPAN style="FONT-SIZE: 13.5pt" lang=EN-US>-------2011.9.24</SPAN></B></FONT></P>
<P style="LINE-HEIGHT: 15.6pt"><FONT size=3><FONT face=宋體><SPAN lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp; </SPAN></SPAN>小組成員：<SPAN style="FONT-SIZE: 10.5pt" lang=EN-US></SPAN></FONT></FONT></P>
<P style="LINE-HEIGHT: 15.6pt"><FONT size=3><FONT face=宋體><SPAN lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </SPAN></SPAN>組長(zhǎng)：韓旭紅<SPAN lang=EN-US> 1091000161</SPAN><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US></SPAN></FONT></FONT></P>
<P style="LINE-HEIGHT: 15.6pt"><FONT size=3><FONT face=宋體><SPAN lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </SPAN></SPAN>組員：李巍<SPAN lang=EN-US> 1091000167&nbsp;&nbsp; </SPAN>李越<SPAN lang=EN-US> 1091000169<SPAN style="mso-spacerun: yes">&nbsp; </SPAN></SPAN>閆悅<SPAN lang=EN-US> 1091000178</SPAN></FONT></FONT></P>
<P style="LINE-HEIGHT: 15.6pt"><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US><FONT face=宋體>&nbsp;</FONT></SPAN></P>
<P style="TEXT-INDENT: -17.1pt; MARGIN-LEFT: 53.2pt; mso-line-height-alt: 15.6pt; mso-para-margin-left: 3.44gd; mso-char-indent-count: -.95"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-bidi-font-family: 幼圓">本周內(nèi)容：<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-INDENT: 27pt; MARGIN-LEFT: 53.25pt; mso-line-height-alt: 15.6pt; mso-para-margin-left: 5.07gd; mso-char-indent-count: 1.5"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-bidi-font-family: 幼圓">這周我們對(duì)<SPAN lang=EN-US>Hadoop MapReduce </SPAN>做了研究，對(duì)<SPAN lang=EN-US>mapreduce</SPAN>有了初步的了解，并為以后工作做了一些規(guī)劃。</SPAN><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US></SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-hansi-font-family: 宋體; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">一．概述</SPAN><B><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 18pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US></SPAN></B></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>Hadoop Map/Reduce</SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">是一個(gè)使用簡(jiǎn)易的軟件框架，基于它寫出來(lái)的應(yīng)用程序能夠運(yùn)行在由上千個(gè)商用機(jī)器組成的大型集群上，并以一種可靠容錯(cuò)的方式并行處理上<SPAN lang=EN-US>T</SPAN>級(jí)別的數(shù)據(jù)集。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="LINE-HEIGHT: 15.6pt; TEXT-INDENT: -53.25pt; MARGIN-LEFT: 53.25pt"><SPAN style="FONT-SIZE: 10.5pt" lang=EN-US><FONT face=宋體>&nbsp;</FONT></SPAN></P>
<P style="TEXT-INDENT: -53.25pt; MARGIN-LEFT: 53.25pt; mso-line-height-alt: 15.6pt"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-bidi-font-family: 幼圓">二．</SPAN><SPAN style="FONT-FAMILY: 'Times New Roman','serif'; FONT-SIZE: 7pt; mso-bidi-font-weight: bold; mso-fareast-font-family: 幼圓" lang=EN-US>&nbsp;&nbsp;&nbsp;&nbsp; </SPAN><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold">簡(jiǎn)單工作原理<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-INDENT: -53.25pt; mso-line-height-alt: 15.6pt"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold" lang=EN-US><SPAN style="mso-spacerun: yes">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </SPAN></SPAN><FONT size=3 face=宋體>一個(gè)<SPAN lang=EN-US>Map/Reduce </SPAN><I>作業(yè)（<SPAN lang=EN-US>job</SPAN>）</I> 通常會(huì)把輸入的數(shù)據(jù)集切分為若干獨(dú)立的數(shù)據(jù)塊，由 <I><SPAN lang=EN-US>map</SPAN>任務(wù)（<SPAN lang=EN-US>task</SPAN>）</I>以完全并行的方式處理它們�？蚣軙�(huì)對(duì)<SPAN lang=EN-US>map</SPAN>的輸出先進(jìn)行排序，然后把結(jié)果輸入給<I><SPAN lang=EN-US>reduce</SPAN>任務(wù)</I>。通常作業(yè)的輸入和輸出都會(huì)被存儲(chǔ)在文件系統(tǒng)中。 整個(gè)框架負(fù)責(zé)任務(wù)的調(diào)度和監(jiān)控，以及重新執(zhí)行已經(jīng)失敗的任務(wù)。</FONT></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">通常，<SPAN lang=EN-US>Map/Reduce</SPAN>框架和分布式文件系統(tǒng)是運(yùn)行在一組相同的節(jié)點(diǎn)上的，也就是說(shuō)，計(jì)算節(jié)點(diǎn)和存儲(chǔ)節(jié)點(diǎn)通常在一起。這種配置允許框架在那些已經(jīng)存好數(shù)據(jù)的節(jié)點(diǎn)上高效地調(diào)度任務(wù)，這可以使整個(gè)集群的網(wǎng)絡(luò)帶寬被非常高效地利用。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>Map/Reduce</SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">框架由一個(gè)單獨(dú)的<SPAN lang=EN-US>master JobTracker </SPAN>和每個(gè)集群節(jié)點(diǎn)一個(gè)<SPAN lang=EN-US>slave TaskTracker</SPAN>共同組成。<SPAN lang=EN-US>master</SPAN>負(fù)責(zé)調(diào)度構(gòu)成一個(gè)作業(yè)的所有任務(wù)，這些任務(wù)分布在不同的<SPAN lang=EN-US>slave</SPAN>上，<SPAN lang=EN-US>master</SPAN>監(jiān)控它們的執(zhí)行，重新執(zhí)行已經(jīng)失敗的任務(wù)。而<SPAN lang=EN-US>slave</SPAN>僅負(fù)責(zé)執(zhí)行由<SPAN lang=EN-US>master</SPAN>指派的任務(wù)。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">應(yīng)用程序至少應(yīng)該指明輸入<SPAN lang=EN-US>/</SPAN>輸出的位置（路徑），并通過(guò)實(shí)現(xiàn)合適的接口或抽象類提供<SPAN lang=EN-US>map</SPAN>和<SPAN lang=EN-US>reduce</SPAN>函數(shù)。再加上其他作業(yè)的參數(shù)，就構(gòu)成了<I>作業(yè)配置（<SPAN lang=EN-US>job configuration</SPAN>）</I>。然后，<SPAN lang=EN-US>Hadoop</SPAN>的 <I><SPAN lang=EN-US>job client</SPAN></I>提交作業(yè)（<SPAN lang=EN-US>jar</SPAN>包<SPAN lang=EN-US>/</SPAN>可執(zhí)行程序等）和配置信息給<SPAN lang=EN-US>JobTracker</SPAN>，后者負(fù)責(zé)分發(fā)這些軟件和配置信息給<SPAN lang=EN-US>slave</SPAN>、調(diào)度任務(wù)并監(jiān)控它們的執(zhí)行，同時(shí)提供狀態(tài)和診斷信息給<SPAN lang=EN-US>job-client</SPAN>。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="LINE-HEIGHT: 18pt; MARGIN: 0cm 0cm 0pt; BACKGROUND: white" class=MsoNormal><SPAN style="FONT-FAMILY: 宋體; LETTER-SPACING: 0.4pt; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">　　　</SPAN><SPAN lang=EN-US><FONT face=Calibri>Hadoop </FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">有許多元素構(gòu)成。其最底部是</SPAN><SPAN lang=EN-US><FONT face=Calibri>HDFS</FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">，它存儲(chǔ)</SPAN><SPAN lang=EN-US><FONT face=Calibri> Hadoop </FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">集群中所有存儲(chǔ)節(jié)點(diǎn)上的文件。</SPAN><SPAN lang=EN-US><FONT face=Calibri>HDFS</FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">的上一層是</SPAN><SPAN lang=EN-US><FONT face=Calibri> MapReduce </FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">引擎，該引擎由</SPAN><SPAN lang=EN-US><FONT face=Calibri> JobTrackers </FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">和</SPAN><SPAN lang=EN-US><FONT face=Calibri> TaskTrackers </FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin">組成。</SPAN><SPAN style="FONT-FAMILY: 宋體; LETTER-SPACING: 0.4pt; FONT-SIZE: 12pt; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin" lang=EN-US></SPAN></P>
<P style="TEXT-INDENT: -53.25pt; MARGIN-LEFT: 53.25pt; mso-line-height-alt: 15.6pt"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-bidi-font-family: 幼圓">三． 運(yùn)行環(huán)境</SPAN><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold" lang=EN-US></SPAN></P>
<UL type=disc>
<LI style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; tab-stops: list 36.0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-list: l0 level1 lfo1" class=MsoNormal><SPAN style="mso-bidi-font-size: 10.5pt" lang=EN-US><FONT face=Calibri>&nbsp;</FONT></SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>Hadoop Streaming</SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">是一種運(yùn)行作業(yè)的實(shí)用工具，它允許用戶創(chuàng)建和運(yùn)行任何可執(zhí)行程序 （例如：<SPAN lang=EN-US>Shell</SPAN>工具）來(lái)做為<SPAN lang=EN-US>mapper</SPAN>和<SPAN lang=EN-US>reducer</SPAN>。<SPAN lang=EN-US></SPAN></SPAN></LI>
<LI style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; tab-stops: list 36.0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-list: l0 level1 lfo1" class=MsoNormal><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>Hadoop Pipes</SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">是一個(gè)與<SPAN lang=EN-US>SWIG</SPAN>兼容的<SPAN lang=EN-US>C++ API </SPAN>（沒(méi)有基于<SPAN lang=EN-US>JNI<SUP>TM</SUP></SPAN>技術(shù)），它也可用于實(shí)現(xiàn)<SPAN lang=EN-US>Map/Reduce</SPAN>應(yīng)用程序。<SPAN lang=EN-US></SPAN></SPAN></LI></UL>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold">四．</SPAN><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold; mso-hansi-font-family: 宋體; mso-bidi-font-family: 幼圓; mso-font-kerning: 0pt">輸入與輸出</SPAN><B><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 18pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US></SPAN></B></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>Map/Reduce</SPAN><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">框架運(yùn)轉(zhuǎn)在<SPAN lang=EN-US>&lt;key, value&gt; </SPAN>鍵值對(duì)上，也就是說(shuō)， 框架把作業(yè)的輸入看為是一組<SPAN lang=EN-US>&lt;key, value&gt; </SPAN>鍵值對(duì)，同樣也產(chǎn)出一組<SPAN lang=EN-US> &lt;key, value&gt; </SPAN>鍵值對(duì)做為作業(yè)的輸出，這兩組鍵值對(duì)的類型可能不同。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">框架需要對(duì)<SPAN lang=EN-US>key</SPAN>和<SPAN lang=EN-US>value</SPAN>的類<SPAN lang=EN-US>(class)</SPAN>進(jìn)行序列化操作， 因此，這些類需要實(shí)現(xiàn) <SPAN lang=EN-US>Writable</SPAN>接口。 另外，為了方便框架執(zhí)行排序操作，<SPAN lang=EN-US>key</SPAN>類必須實(shí)現(xiàn) <SPAN lang=EN-US>WritableComparable</SPAN>接口。 <SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">一個(gè)<SPAN lang=EN-US>Map/Reduce </SPAN>作業(yè)的輸入和輸出類型如下所示：<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US>(input) &lt;k1, v1&gt; -&gt; <B>map</B> -&gt; &lt;k2, v2&gt; -&gt; <B>combine</B> -&gt; &lt;k2, v2&gt; -&gt; <B>reduce</B> -&gt; &lt;k3, v3&gt; (output) </SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold">五．<SPAN lang=EN-US>Map/Reduce - </SPAN>用戶界面</SPAN><B><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 18pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US></SPAN></B></P>
<P style="mso-line-height-alt: 15.6pt"><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold" lang=EN-US>&nbsp;</SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">這部分文檔為用戶將會(huì)面臨的<SPAN lang=EN-US>Map/Reduce</SPAN>框架中的各個(gè)環(huán)節(jié)提供了適當(dāng)?shù)募?xì)節(jié)。這應(yīng)該會(huì)幫助用戶更細(xì)粒度地去實(shí)現(xiàn)、配置和調(diào)優(yōu)作業(yè)。然而，需要注意每個(gè)類<SPAN lang=EN-US>/</SPAN>接口的<SPAN lang=EN-US>javadoc</SPAN>文檔提供最全面的文檔。 <SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">我們會(huì)先看看<SPAN lang=EN-US>Mapper</SPAN>和<SPAN lang=EN-US>Reducer</SPAN>接口。應(yīng)用程序通常會(huì)通過(guò)提供<SPAN lang=EN-US>map</SPAN>和<SPAN lang=EN-US>reduce</SPAN>方法來(lái)實(shí)現(xiàn)它們。 <SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">然后，我們會(huì)討論其他的核心接口，其中包括：<SPAN lang=EN-US> JobConf</SPAN>，<SPAN lang=EN-US>JobClient</SPAN>，<SPAN lang=EN-US>Partitioner</SPAN>，<SPAN lang=EN-US> OutputCollector</SPAN>，<SPAN lang=EN-US>Reporter</SPAN>，<SPAN lang=EN-US> InputFormat</SPAN>，<SPAN lang=EN-US>OutputFormat</SPAN>等等。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">最后，我們將通過(guò)討論框架中一些有用的功能點(diǎn)（例如：<SPAN lang=EN-US>DistributedCache</SPAN>，<SPAN lang=EN-US> IsolationRunner</SPAN>等等）來(lái)收尾。<SPAN lang=EN-US></SPAN></SPAN></P>
<P style="TEXT-ALIGN: left; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 3" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 幼圓; FONT-SIZE: 18pt; mso-bidi-font-weight: bold">六．下一步計(jì)劃</SPAN><B><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 13.5pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt" lang=EN-US></SPAN></B></P>
<P style="TEXT-ALIGN: left; TEXT-INDENT: 24pt; MARGIN: 0cm 0cm 0pt; mso-pagination: widow-orphan; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-char-indent-count: 2.0" class=MsoNormal align=left><SPAN style="FONT-FAMILY: 宋體; FONT-SIZE: 12pt; mso-fareast-font-family: 宋體; mso-fareast-theme-font: minor-fareast; mso-bidi-font-family: 宋體; mso-font-kerning: 0pt">下周我們將對(duì)<SPAN lang=EN-US>mapreduce </SPAN>的核心功能進(jìn)行探究，對(duì)其作業(yè)配置，任務(wù)的執(zhí)行和環(huán)境進(jìn)行進(jìn)一步的了解。</SPAN><SPAN style="mso-bidi-font-size: 10.5pt" lang=EN-US></SPAN></P>
<P style="MARGIN: 0cm 0cm 0pt" class=MsoNormal><SPAN lang=EN-US><FONT face=Calibri>&nbsp;</FONT></SPAN></P></DIV>