Amazon Ads

2015年5月26日 星期二


metricsHDFS與MapReduce daemon會收集一些與事件以及量測相關的資訊,這些總稱為metrics例如,datanode會收集已寫入的byte數、已複製的block總數等metrics
data locality optimization Hadoop在執行map task時,會盡全力將該task放到存放輸入該task資料的那台node去執行,以免使用到寶貴的頻寬進行資料的傳送
shuffle A proccess by which the system performs the sort and transfers the map outputs to the reducers as inputs.
speculative execution Hadoop doesn't try to diagnose and fix slow-running tasks; instead, it tries to detect when a task is running slower than expected and launches another equivalent task as a backup.